Just a tip: instead of modifying the fetch function to use channels and go routines, should probably just encapsulate the function call inside ankther function and handle channel communication from there go func() { fetch(...) ... } This way you don't change the fetch function behaviour and it can be used both with and without go routines
Great video for those unfamiliar with goroutines! One way to estimate how much faster this will make your code is like this: the first way of running it just one after another in a loop will take the time to call a + the time to call b + time to call c etc. When using goroutines, these are all running at the same time, however you're waiting for the collection of them to finish (using wg.Wait()). This means you will be held back by the *slowest* single call whether its a or b or c etc.
@@guhkunpatata3150 no its not an issue at all. It's the whole point of using goroutines. They allow you to take something that say runs 5 times each x amount of seconds and go from 5x to just 1x of time.
Amazing! I recently joined a new company and right now I see the code written to process 1.3G of json data is done with single thread in Java! We are switching to golang and migrating our etl process too, go routines will definitely cut short the processing time from 8 hours to ~2 hours or less. Already did some POC and just like you mentioned, I am seeing more than 3 times better performance in prod environment!
Today I had to make an application that parses a tsv file with 25k+ lines and make some requests to get other data using apis over an interval of time and caches the results. as a newbie to Go I thought it's time to learn goroutines and then I opened Firefox going to Go tour I found your video In the 1st Row lol what a timing. Thanks for the video 😁.
I am a total novice in Go, but watching this video showd me so much cool grifs , forexample that anonymous function to close the wait group, or the fact of being abel to put struct into struct, it was awesome, thanks.
I’ve heard (and maybe I misunderstood the point) that one of the main advantages of go routines over other languages utilizing async is that you don’t have to rewrite your functions. You just add go at the beginning at that’s it. But from what I’m seeing it’s applicable only to functions that do not return anything. If you function returns any value, any you want to run it with go, you’d need to add a channel as an argument to be able to pass the return value back to the caller. Is that correct?
Sorry, I do not know if someone has already mentioned it but in the second case you do not print the temp of the city but just the name of the city ;) I hope this is fixed in the code example if you are sharing it.
So what does the wait group and channels do? If that only needed if you return something from the function. When I look up go routines I get an example with the go keyword but no mention of wait group or channels.
goroutines communicate between each other using channels. this way we can send and receive data from different goroutines running concurrent or parallel. when it comes to wait group, it is usually used to synchronise all goroutines since some of them may execute faster than another. for example, when running a program with go funcs() in it, main function may execute faster than all the goroutines and you will simply end up doing nothing. you can fix this by using sync.WaitGroup{} so the main function will wait for the group to execute before shutting down the application. those are also used to prevent deadlocks in your app
"concept of executing multiple tasks simultaenously by utilizing all available resources more effectively" - isn't this parallelization and not concurrency?
It seems to defeat the purpose of the wait group which is to ensure synchronization. Synchronizing in a separate go routine can produce unpredictable results.
what i would like to see is advanced looping structures, more specifically related to algos or solving algos using some of these crazy loops and 2/3 pointer techniques with for loops.
I am not sure if your example with reading the data from huge csv file using goroutines will make it any faster. At this point you are limited by your CPU power, if you create a few goroutines, CPU power will be split amongst them. It is nice to spawn a goroutine in order to not block completely your app, so you can read csv file in the background but sorry, unless you give me a code example I am not convinced about that
Seeing 2 different terminal outputs for each implementation 6:31 NON go routine: This is the data {{292.9}} This is the data{{288.84}} … go routine: This is the Paris This is the London … Is this expected? Not seeing the data from get request..
Go's concurrency is awesome, literally the main feature, makes up for some of the ugly sides of Go for sure. Liked the video, I would maybe just add a larger array of cities so the difference is more noticeable. Probably a multiple of the number of your logical cores, to see if it scaled down linearly.
I prefer Rust's error handling over that of Go, although Go's error as value is better than throwing and catching exceptions imho. But nothing beats Go's concurrency (maybe in performance but not in syntax)
"Concurrency is executing tasks at the same time", "good example if it is to read a bunch of rows" - that's insane, it's so well covered topic, it takes 10 seconds to google and briefly check a sum up and pick up some example, why didn't you do it? you kinda want to teach other people, but you don't care enough to even google?
Concurrency is handling multiple tasks at once, NOT executing them. You can have one CPU core execute tasks in consecutive order or which their queue up. Please learn first then make videos Thanks
To piggyback on this, with a single CPU tasks run concurrently in any modern OS. A task is run for a bit, another task for a bit, and so on and forth. The thing is the CPU scheduler switches between tasks fairly often and quickly, which gives the impression the tasks are executed at the same time. However at any given point in time only a single task is running on a given CPU. When it comes to parallelism, first tasks must be broken doen to run in a concurrent fashion and then you must have the hardware to run them, i.e., tasks running at exact the same time in different CPUs. Without hardware support, i.e., more CPUs, you cannot have parallelism. Since tasks must be able to run concurrently if you want them to run in parallel (assuming you've enough CPUs), we can say that parallelism is a subset of concurrency. Thus, all parallel tasks are concurrent but not all concurrent tasks are parallel. Any good OS book goes over this. I personally recommend Ramzi's OS: Three Easy Pieces. For a more practical book, I'm finding Cutajar's "Learn Concurrent Programming with Go" quite good.
@@neneodonkorHANDLING multiple tasks AT ONCE is concurrency, EXECUTING multiple tasks AT ONCE is parallelism. Handling means that instead of executing a task to completion on a CPU, followed by another task, etc., the CPU scheduler runs a task for slices of time and thus interleaving their execution time.
Here are precise definitions of concurrency and parallelism in computer science: Concurrency: Concurrency refers to the ability of different parts or units of a program, algorithm, or system to be executed out-of-order or in partial order, without affecting the final outcome. It allows for multiple tasks to be in progress simultaneously, but not necessarily executing at the exact same time. Concurrent execution may involve interleaving the execution of multiple tasks on a single processor or core. Key points: - Tasks can start, run, and complete in overlapping time periods - Does not necessarily imply simultaneous execution - Focuses on dealing with multiple tasks at once Parallelism: Parallelism refers to the simultaneous execution of multiple tasks or processes at the same instant of time. It requires hardware with multiple processing units, such as multi-core processors or distributed systems. Parallel execution aims to divide a task into smaller subtasks that can be processed simultaneously, thus reducing the overall processing time. Key points: - Tasks are literally executed at the same time - Requires hardware support (multiple processors/cores) - Focuses on simultaneous execution to improve performance The main difference is that concurrency is about structure and dealing with multiple tasks, while parallelism is about execution and performing multiple tasks simultaneously.
@@MelkeyDev if it takes ~400ms to complete 4 operations consecutively, then one request takes about ~100 ms so if we make 4 requests in parallel that should take ~100ms as well
go is going to build a lot of tech in the future, its the easiest language to integrate with an LLM u just need to understand a bit of architecture and debugging and its still fast, python with jit to asm/c is the current most used option and imo the best but cant deny the facts
I hope you all enjoyed this GO LANG video!
Your comments mean the world to me. We are SO CLOSE to 5k subs!
nice video, can I use it in my page in a future as an example of the difference between normal process vs using go routine and channels?
Just a tip: instead of modifying the fetch function to use channels and go routines, should probably just encapsulate the function call inside ankther function and handle channel communication from there
go func() {
fetch(...)
...
}
This way you don't change the fetch function behaviour and it can be used both with and without go routines
Great video for those unfamiliar with goroutines! One way to estimate how much faster this will make your code is like this: the first way of running it just one after another in a loop will take the time to call a + the time to call b + time to call c etc. When using goroutines, these are all running at the same time, however you're waiting for the collection of them to finish (using wg.Wait()). This means you will be held back by the *slowest* single call whether its a or b or c etc.
Thanks for the info!
great insight. is it safe to assume that it is an issue ? if yes, is there any way to overcome this issue?
@@guhkunpatata3150 no its not an issue at all. It's the whole point of using goroutines. They allow you to take something that say runs 5 times each x amount of seconds and go from 5x to just 1x of time.
This is insightful. Are there better alternatives?
Great video.
Just mentioning that the speed gain is much more visible if you don't only test for 4 API calls, but for a 100 or 1000.
Good point!
2:06 I'm so used to Prime screaming TOKKKIIOOOOO I half expected Melkey to do that here...
I seriously love and value these videos about Go's main features.
I should do more then??
@@MelkeyDev Definitely please! Really useful for someone who is currently learning Go like me
@@MelkeyDev yes please, mi lord
Amazing! I recently joined a new company and right now I see the code written to process 1.3G of json data is done with single thread in Java! We are switching to golang and migrating our etl process too, go routines will definitely cut short the processing time from 8 hours to ~2 hours or less. Already did some POC and just like you mentioned, I am seeing more than 3 times better performance in prod environment!
Today I had to make an application that parses a tsv file with 25k+ lines and make some requests to get other data using apis over an interval of time and caches the results. as a newbie to Go I thought it's time to learn goroutines and then I opened Firefox going to Go tour I found your video In the 1st Row lol what a timing. Thanks for the video 😁.
Hell yeah my pleasure
why do you execute the wg.Wait and the channel close in a goroutine?
Came to learn to do exactly this. Wanted to call the same api endpoint for multiple inputs and not wait for them all sequentially.
Hopefully this was helpful!
I am a total novice in Go, but watching this video showd me so much cool grifs , forexample that anonymous function to close the wait group, or the fact of being abel to put struct into struct, it was awesome, thanks.
My favourite part was when Go said it's Going time and went way too fast
Great video
Glad you liked it!
7:37, dude i scared me
very quickly and simply explained Goroutines, appreciate it!
Glad it was helpful!
I’ve heard (and maybe I misunderstood the point) that one of the main advantages of go routines over other languages utilizing async is that you don’t have to rewrite your functions. You just add go at the beginning at that’s it. But from what I’m seeing it’s applicable only to functions that do not return anything. If you function returns any value, any you want to run it with go, you’d need to add a channel as an argument to be able to pass the return value back to the caller. Is that correct?
The only way to communicate between GO routines is through channels
Hey, Melkey this is really great one, enjoyed it! I wonder it would be great if one day you could build a complete demo project with goroutines 🙂
Sorry, I do not know if someone has already mentioned it but in the second case you do not print the temp of the city but just the name of the city ;) I hope this is fixed in the code example if you are sharing it.
Most important thing the return data in weather function was useless.
The video starts at 1:28
WE WANT MORE
Great video, I'd like to ask you to do more big projects in the Go programming language. This would be a great experience for language learners !
So what does the wait group and channels do? If that only needed if you return something from the function. When I look up go routines I get an example with the go keyword but no mention of wait group or channels.
goroutines communicate between each other using channels. this way we can send and receive data from different goroutines running concurrent or parallel. when it comes to wait group, it is usually used to synchronise all goroutines since some of them may execute faster than another. for example, when running a program with go funcs() in it, main function may execute faster than all the goroutines and you will simply end up doing nothing. you can fix this by using sync.WaitGroup{} so the main function will wait for the group to execute before shutting down the application. those are also used to prevent deadlocks in your app
Basically in his example it is used only for making sure when to close the channel. Not closing might (but not necessarily) lead to data leak
- saw this video
- was working
....
Alright, time for a break and watch MelkyDev
Its melkey time baybay
simple and on point. great videos bro!
it would be nice if you brought a bit less of drama, you are overplaying
Did you put this in the correct video?
great video, and What keyboard do you have?
I use the Moonlander
"concept of executing multiple tasks simultaenously by utilizing all available resources more effectively" - isn't this parallelization and not concurrency?
where can I review the full code bro?
is that same of eventloop concept in js?
You may want to capture common errors that don't get caught as errors, like leaving routines asleep, negative allocation, and deadlocks.
I'm subscribing.
Amazing
god dang- mans is fr making me wanna switch to go from rust 😖 awesome vid!!! 2x a week bayBEE 🔥
Bruh RUST is SO old
what is the purpose of the anonymous func that tells the wg to wait, and to then close the channel? doesn't it work the same without the func?
i guess so that it doesn't block the main function because we need to print the results as they enter the channel
It seems to defeat the purpose of the wait group which is to ensure synchronization. Synchronizing in a separate go routine can produce unpredictable results.
Gj 👏👏 What about making a tutorial where you build a simple cli tool in GO?
He's not wrong when he started off off with "We all like go" lmao
entertaining and informative thanks
Glad you enjoyed it
Sweet video my dude! Got a link to that tasty concurrency fire file?
Like the code it self?
Yes 😸
what i would like to see is advanced looping structures, more specifically related to algos or solving algos using some of these crazy loops and 2/3 pointer techniques with for loops.
can you limit how many resources you use instead of all or nothing?
Consider keeping the code zoomed in, it's really hard to follow this way, and the recycling between the zoom in and out is causing me headache.
Hey even for basic examples, posting a Github link to your working example would be 🔥. Thanks for the video.
Thanks for the idea!
WE LOVE GO
You stopped returning the data and just returned the names of the cities you passed in?
Thank you!!!
I am not sure if your example with reading the data from huge csv file using goroutines will make it any faster. At this point you are limited by your CPU power, if you create a few goroutines, CPU power will be split amongst them. It is nice to spawn a goroutine in order to not block completely your app, so you can read csv file in the background but sorry, unless you give me a code example I am not convinced about that
It can set you up to parallelize the tasks if you have more than 1 CPU
Seeing 2 different terminal outputs for each implementation 6:31
NON go routine:
This is the data {{292.9}}
This is the data{{288.84}}
…
go routine:
This is the Paris
This is the London
…
Is this expected? Not seeing the data from get request..
Ignore, I didn’t scroll down far enough. Looks like this was already mentioned
Tks for the video! But the concurrency definition is not that.
why dont u make a course on go, it will be lot helpful. Thanks!
cool video)
Hard on the selection bias of that sample of size 3...
Go's concurrency is awesome, literally the main feature, makes up for some of the ugly sides of Go for sure. Liked the video, I would maybe just add a larger array of cities so the difference is more noticeable. Probably a multiple of the number of your logical cores, to see if it scaled down linearly.
I think we need a part two then no?
@@MelkeyDev Thinking? That was your first mistake just GO ♿♿♿
I prefer Rust's error handling over that of Go, although Go's error as value is better than throwing and catching exceptions imho. But nothing beats Go's concurrency (maybe in performance but not in syntax)
@@BboyKeny 100%, rust's error handling is the best out there
ok the glitch sound effect at 7;38 is waaay to loud holy shit scared me Haah
"Concurrency is executing tasks at the same time", "good example if it is to read a bunch of rows" - that's insane, it's so well covered topic, it takes 10 seconds to google and briefly check a sum up and pick up some example, why didn't you do it? you kinda want to teach other people, but you don't care enough to even google?
I didn't see any effect of goroutines in this video😅
I think the data need to be sent to the chan and remove the return data in fetchWeather func. And it don't need the anonymous func
Please make a video about error handling. Address the `if err != nil` issue
It's not an issue
It’s a feature bro
Concurrency is handling multiple tasks at once, NOT executing them. You can have one CPU core execute tasks in consecutive order or which their queue up. Please learn first then make videos Thanks
So handling multiple task AT ONCE IS WHAT?
@@neneodonkorParallelism
To piggyback on this, with a single CPU tasks run concurrently in any modern OS. A task is run for a bit, another task for a bit, and so on and forth. The thing is the CPU scheduler switches between tasks fairly often and quickly, which gives the impression the tasks are executed at the same time. However at any given point in time only a single task is running on a given CPU.
When it comes to parallelism, first tasks must be broken doen to run in a concurrent fashion and then you must have the hardware to run them, i.e., tasks running at exact the same time in different CPUs. Without hardware support, i.e., more CPUs, you cannot have parallelism. Since tasks must be able to run concurrently if you want them to run in parallel (assuming you've enough CPUs), we can say that parallelism is a subset of concurrency. Thus, all parallel tasks are concurrent but not all concurrent tasks are parallel.
Any good OS book goes over this. I personally recommend Ramzi's OS: Three Easy Pieces. For a more practical book, I'm finding Cutajar's "Learn Concurrent Programming with Go" quite good.
@@neneodonkorHANDLING multiple tasks AT ONCE is concurrency, EXECUTING multiple tasks AT ONCE is parallelism.
Handling means that instead of executing a task to completion on a CPU, followed by another task, etc., the CPU scheduler runs a task for slices of time and thus interleaving their execution time.
Here are precise definitions of concurrency and parallelism in computer science:
Concurrency:
Concurrency refers to the ability of different parts or units of a program, algorithm, or system to be executed out-of-order or in partial order, without affecting the final outcome. It allows for multiple tasks to be in progress simultaneously, but not necessarily executing at the exact same time. Concurrent execution may involve interleaving the execution of multiple tasks on a single processor or core.
Key points:
- Tasks can start, run, and complete in overlapping time periods
- Does not necessarily imply simultaneous execution
- Focuses on dealing with multiple tasks at once
Parallelism:
Parallelism refers to the simultaneous execution of multiple tasks or processes at the same instant of time. It requires hardware with multiple processing units, such as multi-core processors or distributed systems. Parallel execution aims to divide a task into smaller subtasks that can be processed simultaneously, thus reducing the overall processing time.
Key points:
- Tasks are literally executed at the same time
- Requires hardware support (multiple processors/cores)
- Focuses on simultaneous execution to improve performance
The main difference is that concurrency is about structure and dealing with multiple tasks, while parallelism is about execution and performing multiple tasks simultaneously.
Isn't that supposed to take 100ms after using goroutine?
Hm, not sure what you mean!
@@MelkeyDev if it takes ~400ms to complete 4 operations consecutively, then one request takes about ~100 ms
so if we make 4 requests in parallel that should take ~100ms as well
Is powing Melkey's version of brofist
yes - yes it
is there a place we can get the code?
I think I should add it to the description
go is going to build a lot of tech in the future, its the easiest language to integrate with an LLM u just need to understand a bit of architecture and debugging and its still fast, python with jit to asm/c is the current most used option and imo the best but cant deny the facts
100%
race conditions in the final run xd
what where
You should really create a short intro hmmm when will you be live ? It’s already the weekend
MORE GO CONCURRENCY
ALL MORE WE WANT
Nah i'd go🗿
i dont like go
but you like me
I am desperate to get wifi working for the esp32 board so I can use tinygo for my micro controllers
I think
Hi, my unique doubt is how we print the data from the api? i see that only printing the city.
Lets goooo
2:51 it’s how hot?!
Don’t show him Event loop guys 😂
Screw benchmarks, tell us ho much you benchPRESS!
Niceee
Kuvalis Alley
Not for high trading
Congratulations! This is the worst explanation I've seen so far and I've seen a lot.
Concurrenthy
Bro I just did a fat line of PHP
we need to get you help
tbh i understand nothing.. i played with c++ modern and js stuff but still understands 0. looks like good content, but not for me
please put in some more effort next time... it does not even print the temperatures in the end 🙄
I genuinely don't understand this meme
GOCURRENCY
Call that Gorrency