Get 1 on 1 mentorship with me! ▶▶▶calendly.com/c... In this video I demonstrate how to use worker treads to unblock and speed up your node applications
This was great. After spending some time going through various tutorials, I found some that had typos in the code, others where the demo was filled with very complex js, and yours was simple and to the point.
good question. I think the answer is that a promise does not make a sync operation happen async, instead it gives you a way to deal with an already async operation. At the end of the day this is just a simple for loop and it will always happen synchronously
@@CodingWithChaim you can promisify this for loop Edit: I recreated the same environment you used in this video and spent 5 hours playing with worker threads. My original comment is wrong. While/for loop is still blocking even if it’s wrapped in a promise unless there is a promise within its conditional or within its block.
Now, can you tell me the limitations of using this worker thread ? Like how many workers can we spawn ? does it depend on the CPU ? can it be used as promise.all() ?
Are there any threading concerns (race conditions, deadlocking, etc.) that can arise with Workers? For instance, is the registering of the worker.on('message'...) code guaranteed to execute before the worker calls its postMessage()?
Hey , I wanted to run some queries on a worker thread and passed in sequelize instnace to the thread but it failed , is there any way how we can pass sequelize instance to the thread and execute the queries on the difrent thread itself on nodejs ,it would have been much easier if it was GO
Are worker threads able to do the same requests as the main thread? I'm aggregating 5.2M documents on Mongodb. It's no problem aggregating on the main thread but it's taking for ever on the worker thread..?
Hi chaim ! It was a nice explanation, but one thing i want to know about, if we make a worker thread then after it's process , will it not be an orphan task, so have we kill that child_process or not after completion?
great question! I actually dont know the answer off hand. My assumption would be that the worker will get cleaned up after its done the work, but I would have to verify to know for sure
Question on cleanup - when working with a singular worker instance to we want to be instantiating a new worker for every call to that url? Is it a better practice to reuse them?
I would say its better to create a new worker for each request because this way we can scale. If we only had one worker that we kept re using, we would run into the same bottleneck we do as with the main thread
@@CodingWithChaim can't believe you got back so quick! Last question - do we need to cleanup the old worker? This reminds me of RXJS observable patterns and I know that failure to cleanup observables can add up over time. Should we delete these workers after the message?
thats a great question! My hunch is that after they get used they will be garbage collected, but I would say you should verify that because I dont know that for sure
@@CodingWithChaim launching my "covid-project" in two weeks with some decently heavy computational work. I had a thought like 'wait, does this block my event loop?' two days later I find your excellent , succinct solution. Thanks again for the help. I'll circle back if I find anything down the road.
If im running the exact same app not locally but on GCP Cloud Run, would it still hang? I assumed each request spun up a new instance of your app thua nullifying any thread issues from the heavy endpoint.
While the explanation on how to use worker's is good,the example is really really bad. If this is an Endpoint that is exposed to the client, there will be potentially hundreds of thousands of requests to that endpoint, so the node application will spawn that numbers of new threads with it's corresponding node instances, event loops etc, which is a nightmare in terms of performance. So keep that in mind..use workers for heavy duty, yes, but not if it will be requested an inmensurable number of times
So, if You have to do some chained awaits that update a database. How do you handle correctly the endpoint for anlot of users? I thought about doing nothing and let AWS spawn more tasks copies behind a las balancer. AM i right?
wow a jewish youtuber, thats rare, as rare as seeing a rainbow. not trying to be mean or anything just abit surprised, thats all, im jewish too so the more the better. Hello bytheway Chaim, really great english too, without any hebrew accent as well. thanks for the video, i'v found about worker threads through google ofc, while searching for cool things about nodejs. didn't quite understand how to use them so i got here, very well explain and articulated. i'll save for future reference.
![🎤 จับฉลากร้องเพลง x P’ #XXSIVK [EP.1] (2/2) เพื่อนจับได้คำไหน..ต้องร้องคำนั้น เริ่ม! 💓🎶 #MXFRUIT](http://i.ytimg.com/vi/1bJvMJXmeS4/mqdefault.jpg)
Can't believe how clean is this explanation. Thank you sir.
I've always known I needed to know Workers but am always so overwhelmed with how complicated it seemed thanks for making me understand this better
This was great. After spending some time going through various tutorials, I found some that had typos in the code, others where the demo was filled with very complex js, and yours was simple and to the point.
Perfect example, perfect explanation & presentation. Thanks for this video!
Chaim, thank you so much for the video. Your explanation is clear and awesome
Someone at my job gave me this exact explanation, excellent content
One of the most interesting and very beautifully explained topic.
Thank you very much.
can we define the pool size for this ?
Well thanks, i Will try this today. Just curious about what happens when you Open several Windows causing going out of threads in your system
Amazing example. very sorted and easy. Thanks and love from India
Thanks for another great video! I'm not a node developer but wouldn't you be able to accomplish this by wrapping the loop in a promise?
good question. I think the answer is that a promise does not make a sync operation happen async, instead it gives you a way to deal with an already async operation. At the end of the day this is just a simple for loop and it will always happen synchronously
@@CodingWithChaim you can promisify this for loop
Edit: I recreated the same environment you used in this video and spent 5 hours playing with worker threads. My original comment is wrong. While/for loop is still blocking even if it’s wrapped in a promise unless there is a promise within its conditional or within its block.
This is really good, seen it here first! Thank you man, keep up the great work.
סרטון מאוד מעניין ומעשיר ידע תודה לך חיים
Wonderful explanation!!!! Thank You!
Question, can I pass just a function inside the worker threads,
or does it really need to be a file ?
Now, can you tell me the limitations of using this worker thread ? Like how many workers can we spawn ? does it depend on the CPU ? can it be used as promise.all() ?
Are there any threading concerns (race conditions, deadlocking, etc.) that can arise with Workers? For instance, is the registering of the worker.on('message'...) code guaranteed to execute before the worker calls its postMessage()?
thank you very helpful
Hey , I wanted to run some queries on a worker thread and passed in sequelize instnace to the thread but it failed , is there any way how we can pass sequelize instance to the thread and execute the queries on the difrent thread itself on nodejs ,it would have been much easier if it was GO
Thanks Chaim. This was very helpful.
Awesome video
great video - helped my solve a problem - thanks Chaim!
Great video man thanks a lot 🙏🏻💯 I run some bots written in JS, would this make them faster?
It wouldn’t make them faster. But it would allow them to run in parallel if each of them has their own thread
Should we always use worker thread instead of fork method of child process?
Thank you very much! This was exacly what I was looking for. :)
Straight to the point. Great.
Simple and crisp, thanks
Thank you very much! More videos on the node! :)
awesome video, well presented and explained
Are worker threads able to do the same requests as the main thread?
I'm aggregating 5.2M documents on Mongodb. It's no problem aggregating on the main thread but it's taking for ever on the worker thread..?
okay, understood
brilliant, very good example!
That's a great explanation! Thank you. 💯
Glad it was helpful!
Hi chaim ! It was a nice explanation, but one thing i want to know about, if we make a worker thread then after it's process , will it not be an orphan task, so have we kill that child_process or not after completion?
great question! I actually dont know the answer off hand. My assumption would be that the worker will get cleaned up after its done the work, but I would have to verify to know for sure
@@CodingWithChaim Thanks for answer
Thanks you, very clear explanation 👍
Question on cleanup - when working with a singular worker instance to we want to be instantiating a new worker for every call to that url? Is it a better practice to reuse them?
I would say its better to create a new worker for each request because this way we can scale. If we only had one worker that we kept re using, we would run into the same bottleneck we do as with the main thread
@@CodingWithChaim can't believe you got back so quick! Last question - do we need to cleanup the old worker? This reminds me of RXJS observable patterns and I know that failure to cleanup observables can add up over time.
Should we delete these workers after the message?
thats a great question! My hunch is that after they get used they will be garbage collected, but I would say you should verify that because I dont know that for sure
@@CodingWithChaim launching my "covid-project" in two weeks with some decently heavy computational work. I had a thought like 'wait, does this block my event loop?' two days later I find your excellent , succinct solution. Thanks again for the help. I'll circle back if I find anything down the road.
glad you found the video, hope it helps
is there a way for the worker to run local function instead of another script?
nodejs.org/api/worker_threads.html#new-workerfilename-options seems like it has to be a file
Thanks you very much!
If im running the exact same app not locally but on GCP Cloud Run, would it still hang? I assumed each request spun up a new instance of your app thua nullifying any thread issues from the heavy endpoint.
Each request will not spin up a new instance of the app. I’m pretty sure that would only happen if you’re using a severless approach
Great content! Thank you!!! :D
Watch at .75x speed
While the explanation on how to use worker's is good,the example is really really bad.
If this is an Endpoint that is exposed to the client, there will be potentially hundreds of thousands of requests to that endpoint, so the node application will spawn that numbers of new threads with it's corresponding node instances, event loops etc, which is a nightmare in terms of performance.
So keep that in mind..use workers for heavy duty, yes, but not if it will be requested an inmensurable number of times
So, if You have to do some chained awaits that update a database. How do you handle correctly the endpoint for anlot of users? I thought about doing nothing and let AWS spawn more tasks copies behind a las balancer. AM i right?
🔥🔥🔥
gg sir
wow a jewish youtuber, thats rare, as rare as seeing a rainbow.
not trying to be mean or anything just abit surprised, thats all, im jewish too so the more the better.
Hello bytheway Chaim, really great english too, without any hebrew accent as well.
thanks for the video, i'v found about worker threads through google ofc, while searching for cool things about nodejs.
didn't quite understand how to use them so i got here, very well explain and articulated. i'll save for future reference.
Andrew Tate now teaches programming
it's ilegal bruh :"
Node Worker Threads, beautifully explained. Thanks.
{2024-01-22}
Wonderful explanation!!!! Thank You!
Beautiful explanation, thank you.