@@crimsonbit Correct me if I'm wrong, but I believe Go uses an M:N task model. That means Goroutines are _always concurrent_ but _sometimes parallel_ (depending on whether the Go runtime decides to schedule your task on the main OS thread or on a different OS thread inside a work-stealing threadpool). For the sake of contrast, Rust's async/await is always concurrent, but may be single-threaded or M:N task parallel (just like Go) depending on the choice of async runtime used.
They probably only coded at a highly abstract service level never having used or profiled low-level read/write calls and seeing with their own eyeballs the time their thread spent blocked on IO. It might lead someone to (wrongly) believe the optimal case is 1 CPU to 1 thread.
"Everyone does things the convoluted serverless way that I do things, and I don't find goroutines useful, therefore goroutines are useless to everyone."
I think he misunderstood the take entirely. I am not saying the take is not stupid, but it was about that you would rather have 12 1-core instances in a 3-node Kubernetes Cluster, 4 per K8s-Node for example, instead of a „big“ 12-core instance of a go-program/-backend. This is because adding another or removing a pod is easier than changing the limits for that pod. And it is better in probable availability. The take from the podcast had nothing to do with serverless.
@@jonas_badstuebner I rewatched the original clip. The second guy accepts the statement that in-process concurrency is important, but in-process parallelism is unimportant specifically for web servers. The first guy didn't seem to know what he was trying to say and just agreed with the second guy. Anywho, I think they are still probably wrong with their implicit assertion that half-hyperthread Kunernetes pods were more scalable per dollar than larger servers with more resources. In the long term, it is cheaper and easier to scale vertically than horizontally, so why use a tech stack that guarantees you'll be unable to scale vertically if needed? Go's concurrency model is good in both scenarios afaik.
It is even worse, when Rob Pike has a whole video talking about this exact thing that those guys in the orig vid should have watched before uttering something so asinine
lightweight threads had 10,000 use cases before anyone was running anything in containers, other than BSD jails, or any of their hardware had multiple cores.
Goroutines are completely useless if you go fully serverless for every function. however that isn't an issue because why in the actual flying fuck would you do that for anything more than a trivial application?
What are the guys in the orig vid talking about? Goroutines are not for parallelism, they are for concurrency in applications that are IO bound like your standard webapp when it talks to a db. "And if you use k8s, you scale at the Pod level": Kubernetes, which is written in, ... golang. Ironically, one of the key advantage of goroutines over vanilla threads is precisely in single core machines that need concurrent IO.
Goroutines *can* be for parallelism (you can have multiple cores each handle their own distinct goroutines in parallel), but yeah, they use one core by default, so are really kind of meant, out of the box, to be used for concurrency.
I love the box I live in. I think inside my box. It's a comfortable box. Nothing exists outside of my box. Outside of my box is the Satan. Therefore, anything outside of my box either does not exist or is evil.
Their argument is two step process: 1) goroutines are stack based concurrency 2) there is no hardware capability of runing parallel code Conculsion: all parallel code on sigle core computer has to be of the form of cooperative multi-tasking (event based concurrency). Thus goroutines - a stack based semantics - is usless. They are not wrong in philosophical sense. They are completly wrong in practical sense. Programming is not philosopy, implementation details matter. Explanation of point 2: They are not saying 1 request by 1 process. They are not using apache's mod_php. They are deploying to containers which use single threaded code only. And given that modern CPU-s are SMT threaded - cores ar virtualized at hardware level - at system level you see more cores than there are physical instances. So your single thread process is not even running on one full core.
Goroutines are so good Java actually introduced their own variant with virtual threads (Project Loom). They feel nicer to program in compared to async in JS and looks like procedural code, the runtime does the magic. Async just slowly creeps into the entire codebase and is a huge PITA if the framework you are using doesn't support it.
Goroutines aren't just for parallelism, it's for concurrency in general. Concurrent programs can run on one threads or more. And goroutines are green threads, so you can have multiple goroutines on a single hardware thread. This allows you to have multiple concurrent things going on, even if you just have one hardware thread.
You need both kinds of threads, native and virtual, which is what Java gives you. golang on the other hand doesn't have native threads (yet another deficiency in that language).
That podcast is called JavaScript Jabber, and it was started years ago by a Rubyist. The Rubyist has a network of podcasts about his programming focus, like Ruby and JS and other stuff. Now you do understand where their opinion stems, their lack of understanding of coding on more than one core, they are defending Ruby.
That's how AWS Lambda works - one request per process at any given time (afterwards the process gets reused though).It definitely comes with advantages, for example it eliminates a lot of "accidentally sharing state between requests" type of bugs. However,. even when working with Lambda I found Goroutines useful. E.g. when getting results from multiple upstream APIs.
@@nickfarley2268yes, but Lambdas still get only one event at a time. Lambdas can also get a whole list of records from e.g. SQS, Kafka. In that case you may still process them in parallel. Or processing the event may be costly and require multiple CPU cores. Btw, Lambdas usually have less than one CPU core in my practical experience
Goroutines are useful for creating servers with multiprocessing input and output data. Imagine processing several data received at the same time by the client one by one, goroutines allow parallelize this task.
Goroutines are very useful. Lots of horrible system designs ignore in-process parallelism at the expense of performance. Performance matters. The problem I have with them is being a built-in language feature instead of a library. There are lots of concurrency patterns, we shouldn't be tied to a programming language on that specific need. Yeah, it is nice that it is standard and part of learning Go, but not to the level that it should be a language feature. Instead, the language should have generic facilities that can be used in any library.
@@tokiomutex4148I think you can. I've seen some pretty powerful language features before that allow this. For example, F# computational expressions. Yes, more language features can make a language more complicated. But it also makes it more useful by allowing for creating libraries that the language designers never thought of.
@@gritcrit4385 Yes, there are some advantages of it being built in -- everything has tradeoffs. But I think it could have been done in a library which also is simple to use. Hell, C++ made strings a library (not saying that extreme is a good thing). The best languages are the ones that allow you to get your job done in ways the authors never envisioned.
I totally disagree. If you make it part of the language you can make it the best and everything will work together. Also, if you want to make a library anyway then whats stopping you.
Ahhhh this was the stream we were joking about top and bottom devs. I couldn't remember what prompted that but I'd already changed my linkedin from "Full stack" to "Switch"
Goroutines of course make sense. Consider this example of a web crawler, you can have multiple coroutines running "at the same time" some of them are blocked waiting for a server response, and some of them are processing. And just because your server might have a single core, you still have multiple threads within that single core.
i hate when somebody talk out of thier a** but those podcast idiots takes the cake : 1) goroutines model is for concurrency problems not parallelism and if you don't know the difference ...then i feel sorry for your clients and team 2) goroutines tasks are optimised for IO bound applications with the possiblity to await ,synchronise and return access to data when needed using channels, it has nothing to do with serveless functions 3) what is more efficient ? runing multiple pods and maintaining an exspensive kubernetes system OR spawning multiple lightweight goroutines that can scale in both horizental and vertical . 4) i hate the word "skill issue" but in this case those 2 definitely have it when coding backend system
I'm just happy I'm getting far enough along in my journey that I at least feel like I understand what is happening when you switch to vim and start throwing some code at us❤
Using a fan out pattern(throw a go routine to every request) gives you the possibility to utilize the way that go works and how it manages the different types of work
So, this is coming from a sysadmin perspective: I feel like the guy's take is from about a decade ago. Everything is being "optimized" (shoehorned?) to run on AWS micro nodes (because cheapest) and scale-out is being handled by "spin up another VM and add it to the load balancer". He says "everything runs on one hyperthread" (even quoting that hurts my soul) is because 1 hyperthreaded core = 2 cores in the hypervisor, and the VM has 1 core assigned. So, it's running with not even a whole dedicated core.
I think you may have misunderstood this. They do not mean 1 process per request. They probably mean, A single instance of the application is allowed to use maximum 1 core on the machine and you spawn more processes to handle more traffic. It's still a weird take because Horizontal scalers are not instant and the request latency will shoot up for a few seconds to minutes until there are more instances available to handle the traffic. Go's ability to use multiple cores works out much better. I can set it to request 1cpu core and set the limit to 4 cpu core so if there is a more traffic, the same application can use multiple cores to serve it while horizontal scaler creates more instances if it has to. And what happens if the nodejs application does some thing weird with a request and pins the only core it can use to 100% ? That instance is still reporting healthy state and all the traffic directed to it gets delayed responses(HS will spawn more instances but it is not instant and the instance will continue to get more traffic). With Go, that go routine will pin a core to a 100% but it has other cores it can use to serve traffic. Their original point around this just seems silly to me, node's event loop's inability to use multiple cores is a short coming and does not render other async designs "useless".
Its not really a weird take when it comes to a cloud computing environment. Vertical scaling (assuming you have implemented some sort of scaling method to your core level abstraction) is considerably slower than Horizontal scaling. As you say, not all applications and use cases can utilize horizontal scaling, but most applications could!
Yea, try doing some mapping of 1000 DB rows in JS. Your entire site will freeze because its single thread will be stuck on this CPU-heavy task and V8 won't give a hecc about muh async
No i think you misunderstood him, even though i dont agree with his statement either. Hes saying the entire long running server is running on one core / thread / hyperthread. so your cpu has no additional resources to run go routines in parallel with your application. the core is either processing your application or processing a go routine. Where as on say a full core , you have the hyperthread as well as other resources that wont get in the way of running in parallel. but i think his argument falls apart especially if youre doing a whole lot of reading and writing like in most web applications (theyre discussing js in the backend afterall). sure parallelism is great for multiple cpu heavy tasks, but for io, concurrency is much more effective for achieving high throughput.
@@caedenwan application can handle multiple requests. Running on a single core. The app keeps running. What's said in the video is that for every request a new instance is started on a single core (or half-vthread), which would be very inefficient. and then it would die after the request is finished.
? Last that I remember you absolutely have to mark every function with async if you want to use await inside of it. And you also need to return Task from the function for it to be awaitable.
BEAM (VM) is still the greatest engine for running concurrent tasks in a parallel environment (since it actually gobbles up those cores baby [and can be run across machines!!]), simply because it's way too easy to do so using languages such as Elixir or Erlang. It's actually very hard to fuck up because the virtual processes in them don't share memory [except for message passing if you want since they are usually run as actors] and are independently garbage collected (no STW GC).
Being able to utilize multiple cores without tools like docker swarm, kubernetes is a huge win for simplicity. He also doesn't realize go combines concurrency and parallelism. Also confusing them. skill_issue/10 podcast
This is a bad take, scheduling is separate from the processes it runs on. We ran computers on single cores before 2006, and without concurrent scheduling, computers would've been bricks. Scheduling is what enabled concurrency on single core machines. The guy from the pod cast is confusing concurrency and parallelism. Prime is correct here, what the guy said is factually incorrect.
Automated await defeats the purpose of reifying concurrency: it's to tell developers "your transaction interleaves here, you need to recheck your assumptions". When promises were invented, we called these "stale stack frames", and making concurrency explicit removed entire classes of bugs.
This is like running a program that was 32 bit on a 64-bit computer and it only uses half of your CPUs and then the program crashes when it break 50% CPU
c# tasks also let you synchronously wait for the value with .Result property blocking the current thread until the task completes and then giving you the value, writing multithreaded code in c# is honestly not that bad and function coloring is relatively low which is quite nice. and before anyone asks, no, i never had the opportunity to try go yet, or rust, my experience is with c#, c, cpp, python, js, and ts (if you could even count them seperately) and a tiny tiny bit of assembly oh and obviously the best language, scratch, scratch is a perfect programming language and you cant convince me otherwise
Not wrong about Scratch TBH. Although, I may be biased, the first language I learned in tech school was ladder logic, which is a fully concurrent, fully visual dataflow language. It was *hard* for me (giggity) to learn, but now I've got that box (giggity) in my head that just makes concurrency so easy (giggity) to understand.
idk what's worse: a board ripping out the icon of a company (for good or bad reasons) or a board that didn't even have the gumption to stick with. their choice. they were screwed in the public eye for firing Sam... but now they are double screwed for walking back on it they should definitely be replaced
A junior even being interested in discussing/thinking about such concepts is a great sign on its own. I guess that's a humorous comment but I'd love to be surrounded by such juniors, tbh. Not sure what your criteria for juniors are, but that's actually a great sign.
ThePrimeagen praises the junior dev (at 0:28). The attitude of this junior dev is clearly good. He comes across something that he finds surprising, and asks for the opinion of others to help him understand. He shows a willingness to learn, with no sign of the Dunning-Kruger effect. If he ends up understanding that goroutines are useful, and why, then it would really be absurd to throw him out of the team. Delete this comment before your team sees it and decides that you are the one who should be thrown out.
According to this guy in the JavaScript Jabber podcast (which is part of Top End Devs network) Kubernetes is "writing Go to avoid writing Go" just like HTMX is writing JavaScript to avoid writing JavaScript
I…sigh…I hate how little the average developer knows about modern computers. Absolutely clueless. What are they even talking about? No idea what they are talking about.
The goroutines are amazing for how easy it could be to run parallel processes and control them, terminate, or wait until they complete their tasks. I'm amazed constantly when I run one more... I love it. Cheers! P.S> I use them, for example, to download 20 movies simultaneously since my internet uplink capacity is sufficient. This significantly shortens the downloading time. There are numerous examples of how incredible goroutines can be.
I made a test awhile back, making blocking single requests to an outside API, or spawn 25 workers to send these requests simultaneously, one goroutine for each. To no surprise, the latter was more than 50 times faster. Of course that in a single core single thread, the OS scheduler will do its magic
The guys in the video also don't understand that Concurrency is not Parallelism. Concurrency is a way of describing work, not an assumption on parallelism (though in go, concurrency patterns are easy to parallelize).
I don't know if I am wrong, but goroutines don't depend on the number of cores, because they are not running in parallel. They are "virtual threads"(not exactly, but I think this is the easiest way to understand it) and they are running asynchronously (without waiting for some routine to finish before starting another one), but not in parallel. That's why even if you have just one core, you can use multiple goroutines.
That's the thing with goroutines, it's concurrency which can also be run in parallel, depending on the number of cores used by the program (which is why you can have race conditions, which wouldn't happen with a single thread concurrency)
You can call them "tasks" (comming from c#) or fibers (from java), or green threads (generic); if I remember, virtual thread are emulating OS threads; so multitasking without concurrency. The structure itself has a bit more depth, because if your application has a threadpool, the task can be executed by different threads. In sum, it's an abstraction that combines the best of both worlds.
They are both the parallelism and concurrency primitive in go. If you have more cores available, they'll be run in parallel and concurrently. If not, they'll be run concurrently. It's the reason why they're lauded so highly, and rely on channels (message passing) to handle data sharing... Independent processes necessitate it (Erlang paved the way).
Goroutines are like TOKIO threads. They may or may not run in parallel or just concurrently. Since actually moving stuff between threads can be costly, it's virtual threads where stuff may be moved between threads if beneficial but not guaranteed.
That might be true for serverless functions/lambda infra. Batch runners (Azure Batch) and containerized workloads could have arbitrary scale per worker. It kind of depends on how the workers are set up. For Azure Batch, the cores scale with mem, so for 16gb of ram, there is a minimum of 4c per node, I think. Each job is remarkably simple copy code/binary to node and run powershell/bash command -> so we're able to take advantage of async stuffs.
I kinda understood the thing the podcast people said differently? I feel like what they're saying is that they're not running 1 thread per request, but *1 thread per logical core*. As in, if you have 24 logical core CPU, you just run 24 containerized copies of the single-threaded server and load balance it, instead of running 1 golang process with tons of goroutines that utilize all the cores. It's like GNU parallel that Primeagen loves so much: single-threaded program + external parallelization = program that utilizes all the resources efficiently. And no, you don't need goroutines for IO, all you need is regular async/await, even in the single-threaded way like how JS does it. And all the internal complexity of the goroutines and all these green threading and stuff becomes useless if you can just parallelize with an external tool and use single-threaded async for IO (which is much simpler internally and also faster). There is that function coloring deal with async/await, sure, but there are ways to solve it (like whatever Zig does with color-blindness or a proper effect polymorphism that some experimental functional languages have)
Javascript schedules async functions. Go schedules goroutines (functions). One of those two can utilize multiple CPU cores and provides more ergonomic abstractions.
Re: change my mind - As an alternative: Fibers as the execution model for functional effect systems (Scala's ZIO and Cats effect, or TypeScript's effect.ts). Fibers allow for concurrency while solving the function coloring problem, and the effect system gives type-safe, pure composability and explicit, typed failure-handling. Writing concurrent programs in ZIO is syntactically no different from writing non-concurrent programs - it's almost trivial. Without some extra work go will have the smaller footprint of course, but with Scala native compilation you can also get high performance, small executables and tiny memory overhead.
Java 21 just released yesterday with support for virtual threads, so the issue is basically moot now. More work coming too like support for structured concurrency.
Goroutines, and generally speaking, concurrency, sound really cool on paper. In backend development I've had the experience that the complexity isn't worth the minimal gains. The complexity being that where you could previously pass around crosscutting context (for example, request information or user infos for logging/tracing) by simply making them Threadlocal (aka every thread gets it's own version), you now have to possibly pass it around by hand to every single function all the way down your call stack. The gains are in theory that you can wait for multiple IO bound things in "parallel" or even just "give" the CPU to someone else that's not waiting on IO. In my personal experience, most things in the backend end up being highly sequential. You get the user auth, THEN get the user info, THEN get the user settings, THEN get the users Todo List (or whatever). Rarely are two steps actually independent enough. Caveat: This is all just from working with Java, so if Go (or other languages with coroutines) have a smart solution to the complexity and maybe other experiences with the potential gains, I'd loove to hear it.
Your claim about mostly sequential IO is only true in the classic backend case. If you have some kind of microservice mesh (also known as bad design), you will definitely have concurrent reads and sometimes even writes (the worst thing that could possibly happen) per request. The real problem is that instead of a tightly integrated system, where you have the web server, the database, the cache server, all in close proximity latency wise, you have to work with a system distributed between multiple so-called "cloud providers" with insane latency between each component. It's no wonder you will want to do overlapping IO on some kind of M:N threading model, because each request takes its sweet time on the order of seconds, instead of (sub-)milliseconds. OS threads are always better than userspace threads if your design is good.
aws lambda is in a similar situation with reactive/async code. aws is going to block on a req/resp, it's not going to reuse that 'vm' for the next request as aws will give that request its own 'instance' of the 'vm' (could be code, image, etc). Java virtual threads don't color your functions. I know, right?
It is more expensive (cost wise especially) to create multiple server instances than utilizing all of the cores and threads of a single server instance.
I don't think they're neccesarily talking about I/O or putting each request on its own core, instead I think they're saying for workload processing. If your goal is to use a go-routine to split processing and workload tasks to distribute them between cores... we just do that at the infrastructure layer now. Just have a single nodejs process where you pass it some subset of the array and it does stuff.. then spawn like 50 copies of it with kubernetes by scaling up the pods, and round robin your requests between them. Each pod is isolated and will basically run parrallell using not only different threads or cores, but entirely different machines.
I run my async node apps multithreaded. You fork out to like 60% of available cores to account for uncontrollable node threads like io and gc stuff but otherwise you can totally use multiple threads with both Cluster which even allows you to share entities like a server handle across workers and then actual worker threads if you need to do anything cpu heavy. Like no one else does this though, maybe like 1% of developers. Lol
They aren't suggesting a process per request. Just that a process only handles a single request at a time. The lifecycle of the process is not coupled to that of the request.
Yes, a single-thread CPU can only do one thing at a time, but by using routines you split your program up into smaller tasks/processes. That ONE CPU/Core/Thread now sees 2 processes and can schedule which one needs more resources/execution time. It's more efficient and thus faster. That even works in real life. Your Brain is a "Single-Core" machine so to speak, but it is still easier to tackle a to-do list, that has many small points, than one that has only a single point encompassing everything to do (Or maybe that is just because of my ADHD Brain)
So the way they propose this should be done instead of doing go-routines makes some sense to me. Generally when i write backend code in a microservice architecture the services are designed to be small and have as little CPU and memory footprint as possible. In those scenarios, instead of scaling out "internally" in the microservice by using go-routines. You should scale within the Kubernetes cluster instead. And from a architecture standpoint, that makes sense I think. That said, I don't agree with them that go-routines are useless. I've had plenty of use for them even within that type of architecture. Every single time reach out to multiple different external source, where the calls are not dependent on each other, I of course reach for go-routines. Every time I create a API server and I want the ping endpoint to live on a separate port, I use go-routines. There are still plenty of scenarios where go-routines makes a lot of sense.
But what is the benefit of having extremely small memory footprint processes if you have to use more processes? Because total CPU/memory footprint of the whole system is what actually matters...
also since you brought up serverless, that's not how serverless works either. so for this example I'm going to specifically use aws lambda and golang, but the exact same thing applies no matter what serverless system you were using. so basically what happens is that when a request comes in, the serverless system checks to see if there is a function that is available to handle the request. if not it creates a new one. so let's say that it's a service that only gets one request per hour, so everything is scaled to zero and there are no currently running functions whenever a request comes in. what will happen is it will always spin up a new instance, it will handle the one request, then it will get torn down. but if we say it gets two requests back to back every hour, it will still only spin up one instance of the function. this is because it keeps the function around for a bit. and with golang in particular, but you can also do the same thing with Python, you can actually set up all of your database connections and such and then share them between requests. to get a bit more specific about what happens in between requests, this is AWS lambda specific by the way, what happens is that you get an initial grace period during startup that lets you do things like set up database connections, global variables, etc and you get about 100 milliseconds to do all that. then your process gets sent a sig stop, so it can't run on the CPU anymore. then a request comes in and AWS lambda sends your process a sig cont allowing it to be on the processor again. this is how AWS enforces their billing and keeps you from basically getting free background compute. while your process is not sigstop'd, your program is free to do whatever it wants. so as long as requests are coming in at some rate, you can maintain all of your database connections and stuff. you can actually even do true multithreading if they give you enough CPU.
what they talk about is kubernetes which is gnu parallel on steroids. I still disagree with this take because of what Rob Pike said about them in the first place, that designing with concurrency in mind, even if it won't run in parallel is useful due to the organization it provides for the application.
This has nothing to do with K8s per se; as you can run concurrent code on 1/10 of cpu if you desire. Concurrency is not bound by the amount of CPU cores you have, parallelism on the other hand, is.
Using golang you shouldn't immediately worry about the number of core or cpu-threads there are. The biggest advantage to go routines is the time saving (concurrency) on things not just isolating inbound requests. But if you have nothing that can benefit from concurrency then there isn't a use for them, but it's hard to stretch that into "useless".
Here's the thing - lambda is already one generation behind.... for better scaling and resource utilization you should be using AppRunner and CloudRun.... Yet even on lambda green thread is still super useful, e.g. if you are connecting to database in lambda, an async model will allow you to do that in a non blocking manner... .... concurrency has (pretty much) nothing to do with parallelism, which is what I do not understand what they are talking about....
You can treat serverless JS lambdas (or really any language lambdas) as a single process. They do indeed serve one request at a time. Goroutines make the program crash if they modify a map at the same time without a mutex. Not sure if that's great. Channels are great in theory, not very useful in practice. They also have some pitfalls. I also wonder... Why is IO as values function coloring that sucks, but Result is not? We like the benefits of errors as values, right? I also like the benefits of IO as values. I do want to know if you are going to do some random IO to compute the value you're giving me instead of entirely using things passed as arguments. Helps me know how large the surface area for errors is.
Greenthreads/async: "We don't actually need an entire OS thread to handle these things, we can just do cooperative userspace threading for this problem because we know there are no long tight loops that'll cause problems. This'll let us handle a lot more requests on a single server without the massive memory and context switch overhead of spawning an entire OS thread for each connection" These guys, apparently: "What if instead of just spawning a new thread for each connection, we spawn an entire new process for every request?" These guys, later: "Greenthreads are useless because you're spawning a new process for every request anyway" (never mind that even if you're doing that, there can be plenty of stuff for a single request that can be done concurrently like any sort of database shit)
Bit of a misunderstanding there by Prime, the guy on the podcast was saying they just run 1 process per pod, which handles requests (which is very different to spawning a process for each request). Still, I agree that it is a bad take that this is the only use case ever. Also does not really make goroutines useless. Running a single process per pod is definitely not as efficient as sharing resources(memory, cpu spikes, io) between requests, but if feasible, it is a great way to reduce maintenance effort. It also makes it really simple to scale, even though it is expensive.
It's quite an amusing argument, 'go threads are useless because of kubernetes' ... which is written in go, and is working very hard behind the scene, to save the bacon of some rather less efficient languages.
One pattern I find with go developers is they sometimes lack basic computer science knowledge. Like they don't understand the difference between blocking and nonblocking IO, the difference between parallelism and concurrency.
I don't think they're saying that a new process is start/killed for each request, the processes can be long-running, just that they only serve one request at a time. It still doesn't make much sense though.
It's not unlike how lambda stuff works. Most of the lambda runtimes on AWS reuse lambda instances, but only once the last one is done. It's just not a great way to build a real system, it's just an okay one to simply do exactly what lambda is trying to do.
Sure it is, if you want to run 100 instances of the same container on Kubernetes you can, but it is much more efficient running 10 containers and use Goroutines. Goroutines run into threads, containers are processes. Spawning processes is more expensive than threads.
As @theprimeagen said, I'm not an infrastructure expert and it is not totally clear which setup are they talking about. That being said, if goroutines are useless since some time ago, people developing Kotlin and Java should be stupid, as they developed coroutines and virtual threads afterwards. That sounds like a proof by contradiction to me (as I don't think people developing Kotlin and Java are stupid).
I feel like Prime completely misunderstood what they're saying. They're talking about a pod instance of a long running program per thread. The process itself just uses no threading. Basically, think of how NodeJS scales.
I think goroutines are not the best solution. You can call functions asynchronously but you can't get there result. To get the result, you have to pass a channel for communication. Having to pass the channel makes it colored in some sense. The way zig does it is the best of both worlds. You can call any function synchronously or asynchronously. If you call it synchronously (`const x = foo()`), it will return the normal value. If you call it asynchronously however (`const x = async foo()`), it will return a frame. This is a special zig type which encapsulates a function frame. You can await the frame afterwards to get the result. Even better is that you can specify how the frames will run (your own event loop), or you can use the one the stdlib provides. (It worked on zig 0.9, but with the new self hosted compiler they are rewriting it should work again in the next release)
![ลองสุ่มไอโฟน 990 โกงมั้ย? [ โกงมั้ยครับ ep.101 ] | DOM](http://i.ytimg.com/vi/bfsWe0j492k/mqdefault.jpg)
Concurrency vs parallelism, here we go again.
Correct. This is more deep than just async await.
Do goroutines count as concurrency or parallelism?
@@crimsonbit Correct me if I'm wrong, but I believe Go uses an M:N task model. That means Goroutines are _always concurrent_ but _sometimes parallel_ (depending on whether the Go runtime decides to schedule your task on the main OS thread or on a different OS thread inside a work-stealing threadpool).
For the sake of contrast, Rust's async/await is always concurrent, but may be single-threaded or M:N task parallel (just like Go) depending on the choice of async runtime used.
@@xplinux22 that is correct
@@young_oak Phew, thanks for the confirmation. Glad I wasn't accidentally misleading folks.
That's what happens when you spend more time whiteboarding than doing actual programming
Every LinkedIn programming guru right now
but, the formal verification on the paper says that...
@@MrAlanCristhianlol, what large programs are formally verified ? You know how expensive it is to formally verify code ??
@@lucasjames8281 I said "on the paper".
@@MrAlanCristhian right my bad I guess.
Formal verification on the paper ? What does that even mean? You formally verify code
Just from the first 2 sentences in the meta-video, makes me think that these two goobers have never ever written an IO-bound application.
I think they have done it in Javascript. But they don't know why async functions exist, what they solve.
They probably only coded at a highly abstract service level never having used or profiled low-level read/write calls and seeing with their own eyeballs the time their thread spent blocked on IO. It might lead someone to (wrongly) believe the optimal case is 1 CPU to 1 thread.
Goddamn goobers…
@@potato98326676
@@potato983266 12:04 12:05 7
"Everyone does things the convoluted serverless way that I do things, and I don't find goroutines useful, therefore goroutines are useless to everyone."
Well, I don't have a brain therefore everyone is dumb
I think he misunderstood the take entirely. I am not saying the take is not stupid, but it was about that you would rather have 12 1-core instances in a 3-node Kubernetes Cluster, 4 per K8s-Node for example, instead of a „big“ 12-core instance of a go-program/-backend.
This is because adding another or removing a pod is easier than changing the limits for that pod. And it is better in probable availability.
The take from the podcast had nothing to do with serverless.
I get the take and it has nothing to do with serverless, I still think that it is a bad take.
@@jonas_badstuebner I rewatched the original clip. The second guy accepts the statement that in-process concurrency is important, but in-process parallelism is unimportant specifically for web servers. The first guy didn't seem to know what he was trying to say and just agreed with the second guy.
Anywho, I think they are still probably wrong with their implicit assertion that half-hyperthread Kunernetes pods were more scalable per dollar than larger servers with more resources. In the long term, it is cheaper and easier to scale vertically than horizontally, so why use a tech stack that guarantees you'll be unable to scale vertically if needed? Go's concurrency model is good in both scenarios afaik.
The guy was excoriated in the comment section of the original video for his nonsensical take on Go.
Dunning kruger effect on full swing in the podcast. I feel bad for the number of people who will be led astray by such a misguided notion
Whose, though? The article or the commentary?
@KirkWaiblinger I'm reffing to the guy on the podcast. Prime's commentary is spot on
@@KirkWaiblinger yes
@@pylotlight 😂😂
Skill issue, parallelism and concurrency are not the same, is sad that that people that actually lack of knowledge are that noisy
It is even worse, when Rob Pike has a whole video talking about this exact thing that those guys in the orig vid should have watched before uttering something so asinine
lightweight threads had 10,000 use cases before anyone was running anything in containers, other than BSD jails, or any of their hardware had multiple cores.
Goroutines are completely useless if you go fully serverless for every function. however that isn't an issue because why in the actual flying fuck would you do that for anything more than a trivial application?
JS brains try to comprehend things outside of their JS bubble. If people actually used their brain we wouldn't be using JS on the server.
What are the guys in the orig vid talking about? Goroutines are not for parallelism, they are for concurrency in applications that are IO bound like your standard webapp when it talks to a db. "And if you use k8s, you scale at the Pod level": Kubernetes, which is written in, ... golang. Ironically, one of the key advantage of goroutines over vanilla threads is precisely in single core machines that need concurrent IO.
let em cope bro
Goroutines *can* be for parallelism (you can have multiple cores each handle their own distinct goroutines in parallel), but yeah, they use one core by default, so are really kind of meant, out of the box, to be used for concurrency.
You can use them for parallelism, but that's a different topic.
How do they think Kubernetes is powered... docker too. 😂
we used Go to destroy why we need Go
I love the box I live in.
I think inside my box.
It's a comfortable box.
Nothing exists outside of my box.
Outside of my box is the Satan.
Therefore, anything outside of my box either does not exist or is evil.
Their argument is two step process:
1) goroutines are stack based concurrency
2) there is no hardware capability of runing parallel code
Conculsion: all parallel code on sigle core computer has to be of the form of cooperative multi-tasking (event based concurrency). Thus goroutines - a stack based semantics - is usless.
They are not wrong in philosophical sense. They are completly wrong in practical sense. Programming is not philosopy, implementation details matter.
Explanation of point 2:
They are not saying 1 request by 1 process. They are not using apache's mod_php. They are deploying to containers which use single threaded code only. And given that modern CPU-s are SMT threaded - cores ar virtualized at hardware level - at system level you see more cores than there are physical instances. So your single thread process is not even running on one full core.
Goroutines are so good Java actually introduced their own variant with virtual threads (Project Loom). They feel nicer to program in compared to async in JS and looks like procedural code, the runtime does the magic. Async just slowly creeps into the entire codebase and is a huge PITA if the framework you are using doesn't support it.
Or worse when you have two libraries: one that supports async and the other that doesn't
The nice thing with JavaScript is that you can use the older promise syntax if you can't use async. But yeah: the async virus is pretty annoying.
Especially if it is rust 😂
@@disguysn callback hell is just as bad as the async virus. javascript is the least nice thing that's ever happened to programming
@@maruseron if you don't keep nesting, it's not callback hell. If you do it right you'll only need a couple of callbacks to end the async chain.
Goroutines aren't just for parallelism, it's for concurrency in general. Concurrent programs can run on one threads or more. And goroutines are green threads, so you can have multiple goroutines on a single hardware thread. This allows you to have multiple concurrent things going on, even if you just have one hardware thread.
Accurate. Respect for standing up for Golang, too many people shit on it nowadays for no apparent reason
Yes, lightweight threading is the way to go. It's nice to see the model spreading with Java implementing it.
Yeah, it coming into Java 21 means it is going to be way more popular since Java is still one of the most popular languages.
@@IvanRandomDude Maybe after at least 5 years when you finally get to use 21. The majority are on Java 11, and a lot even use 8 still.
@@Rakkoonn Java 8 is still dominant workhorse. At least it was in 2022. Mostly because dominant platform is not JDK but JEE.
@@Rakkoonnthe tragedy of Java. You get to see what’s possible and yet never have the ability to use it
You need both kinds of threads, native and virtual, which is what Java gives you. golang on the other hand doesn't have native threads (yet another deficiency in that language).
The argument falls apart the second you do litterally anything else then a basic web service
That podcast is called JavaScript Jabber, and it was started years ago by a Rubyist. The Rubyist has a network of podcasts about his programming focus, like Ruby and JS and other stuff. Now you do understand where their opinion stems, their lack of understanding of coding on more than one core, they are defending Ruby.
That makes sense
That's how AWS Lambda works - one request per process at any given time (afterwards the process gets reused though).It definitely comes with advantages, for example it eliminates a lot of "accidentally sharing state between requests" type of bugs.
However,. even when working with Lambda I found Goroutines useful. E.g. when getting results from multiple upstream APIs.
But don’t aws lambda instances get multiple cores?
@@nickfarley2268yes, but Lambdas still get only one event at a time. Lambdas can also get a whole list of records from e.g. SQS, Kafka. In that case you may still process them in parallel. Or processing the event may be costly and require multiple CPU cores. Btw, Lambdas usually have less than one CPU core in my practical experience
Goroutines are useful for creating servers with multiprocessing input and output data. Imagine processing several data received at the same time by the client one by one, goroutines allow parallelize this task.
Sounds like the guy just doesn't understand what he's talking about. Goroutines are useful for both concurrency and parallelism.
Goroutines are very useful. Lots of horrible system designs ignore in-process parallelism at the expense of performance. Performance matters. The problem I have with them is being a built-in language feature instead of a library. There are lots of concurrency patterns, we shouldn't be tied to a programming language on that specific need. Yeah, it is nice that it is standard and part of learning Go, but not to the level that it should be a language feature. Instead, the language should have generic facilities that can be used in any library.
But having them baked into the language makes it simple as fuck to use. Which is a high priority in its design.
You can't have a concurrency model packaged as a library that's easy to use in a simple language
@@tokiomutex4148I think you can. I've seen some pretty powerful language features before that allow this. For example, F# computational expressions. Yes, more language features can make a language more complicated. But it also makes it more useful by allowing for creating libraries that the language designers never thought of.
@@gritcrit4385 Yes, there are some advantages of it being built in -- everything has tradeoffs. But I think it could have been done in a library which also is simple to use. Hell, C++ made strings a library (not saying that extreme is a good thing). The best languages are the ones that allow you to get your job done in ways the authors never envisioned.
I totally disagree. If you make it part of the language you can make it the best and everything will work together. Also, if you want to make a library anyway then whats stopping you.
Ahhhh this was the stream we were joking about top and bottom devs. I couldn't remember what prompted that but I'd already changed my linkedin from "Full stack" to "Switch"
Goroutines of course make sense. Consider this example of a web crawler, you can have multiple coroutines running "at the same time" some of them are blocked waiting for a server response, and some of them are processing. And just because your server might have a single core, you still have multiple threads within that single core.
i hate when somebody talk out of thier a** but those podcast idiots takes the cake :
1) goroutines model is for concurrency problems not parallelism and if you don't know the difference ...then i feel sorry for your clients and team
2) goroutines tasks are optimised for IO bound applications with the possiblity to await ,synchronise and return access to data when needed using channels, it has nothing to do with serveless functions
3) what is more efficient ? runing multiple pods and maintaining an exspensive kubernetes system OR spawning multiple lightweight goroutines that can scale in both horizental and vertical .
4) i hate the word "skill issue" but in this case those 2 definitely have it when coding backend system
I'm just happy I'm getting far enough along in my journey that I at least feel like I understand what is happening when you switch to vim and start throwing some code at us❤
I can feel you it is amazing feeling indeed, that makes me want to dig deeper in CS.
Using a fan out pattern(throw a go routine to every request) gives you the possibility to utilize the way that go works and how it manages the different types of work
These two are what we call highly abstracted Frameworkers drunk on the Kuber juice
So, this is coming from a sysadmin perspective: I feel like the guy's take is from about a decade ago.
Everything is being "optimized" (shoehorned?) to run on AWS micro nodes (because cheapest) and scale-out is being handled by "spin up another VM and add it to the load balancer".
He says "everything runs on one hyperthread" (even quoting that hurts my soul) is because 1 hyperthreaded core = 2 cores in the hypervisor, and the VM has 1 core assigned. So, it's running with not even a whole dedicated core.
I think you may have misunderstood this. They do not mean 1 process per request. They probably mean, A single instance of the application is allowed to use maximum 1 core on the machine and you spawn more processes to handle more traffic. It's still a weird take because Horizontal scalers are not instant and the request latency will shoot up for a few seconds to minutes until there are more instances available to handle the traffic.
Go's ability to use multiple cores works out much better. I can set it to request 1cpu core and set the limit to 4 cpu core so if there is a more traffic, the same application can use multiple cores to serve it while horizontal scaler creates more instances if it has to.
And what happens if the nodejs application does some thing weird with a request and pins the only core it can use to 100% ? That instance is still reporting healthy state and all the traffic directed to it gets delayed responses(HS will spawn more instances but it is not instant and the instance will continue to get more traffic). With Go, that go routine will pin a core to a 100% but it has other cores it can use to serve traffic. Their original point around this just seems silly to me, node's event loop's inability to use multiple cores is a short coming and does not render other async designs "useless".
Its not really a weird take when it comes to a cloud computing environment. Vertical scaling (assuming you have implemented some sort of scaling method to your core level abstraction) is considerably slower than Horizontal scaling. As you say, not all applications and use cases can utilize horizontal scaling, but most applications could!
Yea, try doing some mapping of 1000 DB rows in JS. Your entire site will freeze because its single thread will be stuck on this CPU-heavy task and V8 won't give a hecc about muh async
No i think you misunderstood him, even though i dont agree with his statement either. Hes saying the entire long running server is running on one core / thread / hyperthread. so your cpu has no additional resources to run go routines in parallel with your application. the core is either processing your application or processing a go routine. Where as on say a full core , you have the hyperthread as well as other resources that wont get in the way of running in parallel. but i think his argument falls apart especially if youre doing a whole lot of reading and writing like in most web applications (theyre discussing js in the backend afterall). sure parallelism is great for multiple cpu heavy tasks, but for io, concurrency is much more effective for achieving high throughput.
I understood it this way as well.
I don’t see how what you’re saying is any different from what he said
Basically you don't have hyper-threading except when you pay extra in the cloud!?
@@caedenwan application can handle multiple requests. Running on a single core. The app keeps running. What's said in the video is that for every request a new instance is started on a single core (or half-vthread), which would be very inefficient. and then it would die after the request is finished.
Yep thats how I understood him. Which is patently absurd.
C# async is nice that it doesn't really colour your functions and it allows both cooperative and pre-emptive multi-tasking.
? Last that I remember you absolutely have to mark every function with async if you want to use await inside of it. And you also need to return Task from the function for it to be awaitable.
BEAM (VM) is still the greatest engine for running concurrent tasks in a parallel environment (since it actually gobbles up those cores baby [and can be run across machines!!]), simply because it's way too easy to do so using languages such as Elixir or Erlang. It's actually very hard to fuck up because the virtual processes in them don't share memory [except for message passing if you want since they are usually run as actors] and are independently garbage collected (no STW GC).
There I said it, fight me
LOL, yeah, cause software is always deployed to K8s to serve HTTP requests.
Being able to utilize multiple cores without tools like docker swarm, kubernetes is a huge win for simplicity.
He also doesn't realize go combines concurrency and parallelism. Also confusing them.
skill_issue/10 podcast
it’s hilarious because docker and kubernetes are written in Go. so you literally can’t escape it
wtf is that statement
Julia implements both channels and async await (that is actually sugar syntax to await for the first value in the channel).
But all microservices and cloud architecture is built on Go
This is a bad take, scheduling is separate from the processes it runs on. We ran computers on single cores before 2006, and without concurrent scheduling, computers would've been bricks. Scheduling is what enabled concurrency on single core machines. The guy from the pod cast is confusing concurrency and parallelism.
Prime is correct here, what the guy said is factually incorrect.
What do you guys think of Kotlin's coroutines ? Functions are still colored, but calling them is identical. The compiler figures out when to await.
Too bad jvm doesn't have call/cc.
@@georgerogers1166 virtual threads were released yesterday
@@georgerogers1166 Java 21 was just released yesterday with support for virtual threads. No need for async/await.
Not needed with Java 21's virtual (green) threads.
Automated await defeats the purpose of reifying concurrency: it's to tell developers "your transaction interleaves here, you need to recheck your assumptions". When promises were invented, we called these "stale stack frames", and making concurrency explicit removed entire classes of bugs.
This is like running a program that was 32 bit on a 64-bit computer and it only uses half of your CPUs and then the program crashes when it break 50% CPU
c# tasks also let you synchronously wait for the value with .Result property blocking the current thread until the task completes and then giving you the value, writing multithreaded code in c# is honestly not that bad and function coloring is relatively low which is quite nice.
and before anyone asks, no, i never had the opportunity to try go yet, or rust, my experience is with c#, c, cpp, python, js, and ts (if you could even count them seperately) and a tiny tiny bit of assembly
oh and obviously the best language, scratch, scratch is a perfect programming language and you cant convince me otherwise
Not wrong about Scratch TBH. Although, I may be biased, the first language I learned in tech school was ladder logic, which is a fully concurrent, fully visual dataflow language. It was *hard* for me (giggity) to learn, but now I've got that box (giggity) in my head that just makes concurrency so easy (giggity) to understand.
idk what's worse: a board ripping out the icon of a company (for good or bad reasons)
or a board that didn't even have the gumption to stick with. their choice.
they were screwed in the public eye for firing Sam... but now they are double screwed for walking back on it
they should definitely be replaced
the first 30 seconds, nodejs came up at the back of my mind like right away
The kind of conversation that gets a junior dev thrown out of the team.
😂😂😂
If you throw out someone for being curious, you are the problem...
A junior even being interested in discussing/thinking about such concepts is a great sign on its own. I guess that's a humorous comment but I'd love to be surrounded by such juniors, tbh. Not sure what your criteria for juniors are, but that's actually a great sign.
ThePrimeagen praises the junior dev (at 0:28). The attitude of this junior dev is clearly good. He comes across something that he finds surprising, and asks for the opinion of others to help him understand. He shows a willingness to learn, with no sign of the Dunning-Kruger effect. If he ends up understanding that goroutines are useful, and why, then it would really be absurd to throw him out of the team. Delete this comment before your team sees it and decides that you are the one who should be thrown out.
very technically c is also non-colored functions, but that's a completely different story.
According to this guy in the JavaScript Jabber podcast (which is part of Top End Devs network) Kubernetes is "writing Go to avoid writing Go" just like HTMX is writing JavaScript to avoid writing JavaScript
I…sigh…I hate how little the average developer knows about modern computers. Absolutely clueless. What are they even talking about?
No idea what they are talking about.
They allow me to send hundreds of thousands of requests though 😈
This feels related to the "Death by 1000 MicroServices" video
The goroutines are amazing for how easy it could be to run parallel processes and control them, terminate, or wait until they complete their tasks. I'm amazed constantly when I run one more... I love it. Cheers!
P.S> I use them, for example, to download 20 movies simultaneously since my internet uplink capacity is sufficient. This significantly shortens the downloading time. There are numerous examples of how incredible goroutines can be.
when you are using apache, every request will be a fork of apache
I made a test awhile back, making blocking single requests to an outside API, or spawn 25 workers to send these requests simultaneously, one goroutine for each. To no surprise, the latter was more than 50 times faster.
Of course that in a single core single thread, the OS scheduler will do its magic
The guys in the video also don't understand that Concurrency is not Parallelism. Concurrency is a way of describing work, not an assumption on parallelism (though in go, concurrency patterns are easy to parallelize).
The original statement was insane but the casual agreement blew my mind. I bet their backend has over 10k microservices...
I don't know if I am wrong, but goroutines don't depend on the number of cores, because they are not running in parallel. They are "virtual threads"(not exactly, but I think this is the easiest way to understand it) and they are running asynchronously (without waiting for some routine to finish before starting another one), but not in parallel. That's why even if you have just one core, you can use multiple goroutines.
That's the thing with goroutines, it's concurrency which can also be run in parallel, depending on the number of cores used by the program (which is why you can have race conditions, which wouldn't happen with a single thread concurrency)
You can call them "tasks" (comming from c#) or fibers (from java), or green threads (generic); if I remember, virtual thread are emulating OS threads; so multitasking without concurrency.
The structure itself has a bit more depth, because if your application has a threadpool, the task can be executed by different threads.
In sum, it's an abstraction that combines the best of both worlds.
They are both the parallelism and concurrency primitive in go. If you have more cores available, they'll be run in parallel and concurrently. If not, they'll be run concurrently.
It's the reason why they're lauded so highly, and rely on channels (message passing) to handle data sharing... Independent processes necessitate it (Erlang paved the way).
Setting GOMAXPROCS in env variable you cab decide how many maximum go routines you want. Default value is just number of cores
Goroutines are like TOKIO threads. They may or may not run in parallel or just concurrently.
Since actually moving stuff between threads can be costly, it's virtual threads where stuff may be moved between threads if beneficial but not guaranteed.
>junior java developer
Meanwhile java:
>are virtual threads we released yesterday a joke to you?
That might be true for serverless functions/lambda infra. Batch runners (Azure Batch) and containerized workloads could have arbitrary scale per worker. It kind of depends on how the workers are set up. For Azure Batch, the cores scale with mem, so for 16gb of ram, there is a minimum of 4c per node, I think. Each job is remarkably simple copy code/binary to node and run powershell/bash command -> so we're able to take advantage of async stuffs.
I kinda understood the thing the podcast people said differently? I feel like what they're saying is that they're not running 1 thread per request, but *1 thread per logical core*. As in, if you have 24 logical core CPU, you just run 24 containerized copies of the single-threaded server and load balance it, instead of running 1 golang process with tons of goroutines that utilize all the cores. It's like GNU parallel that Primeagen loves so much: single-threaded program + external parallelization = program that utilizes all the resources efficiently.
And no, you don't need goroutines for IO, all you need is regular async/await, even in the single-threaded way like how JS does it. And all the internal complexity of the goroutines and all these green threading and stuff becomes useless if you can just parallelize with an external tool and use single-threaded async for IO (which is much simpler internally and also faster).
There is that function coloring deal with async/await, sure, but there are ways to solve it (like whatever Zig does with color-blindness or a proper effect polymorphism that some experimental functional languages have)
This is why I am satisfied when I change the like button color and call it a day
Javascript schedules async functions. Go schedules goroutines (functions). One of those two can utilize multiple CPU cores and provides more ergonomic abstractions.
Re: change my mind -
As an alternative: Fibers as the execution model for functional effect systems (Scala's ZIO and Cats effect, or TypeScript's effect.ts). Fibers allow for concurrency while solving the function coloring problem, and the effect system gives type-safe, pure composability and explicit, typed failure-handling.
Writing concurrent programs in ZIO is syntactically no different from writing non-concurrent programs - it's almost trivial. Without some extra work go will have the smaller footprint of course, but with Scala native compilation you can also get high performance, small executables and tiny memory overhead.
Java 21 just released yesterday with support for virtual threads, so the issue is basically moot now. More work coming too like support for structured concurrency.
@zhamed9587 virtual threads are great of course, but they don't spell obsolescence for effect systems.
th-cam.com/video/9I2xoQVzrhs/w-d-xo.html
Go routines are godsent even on single core instances
Goroutines, and generally speaking, concurrency, sound really cool on paper. In backend development I've had the experience that the complexity isn't worth the minimal gains.
The complexity being that where you could previously pass around crosscutting context (for example, request information or user infos for logging/tracing) by simply making them Threadlocal (aka every thread gets it's own version), you now have to possibly pass it around by hand to every single function all the way down your call stack.
The gains are in theory that you can wait for multiple IO bound things in "parallel" or even just "give" the CPU to someone else that's not waiting on IO. In my personal experience, most things in the backend end up being highly sequential. You get the user auth, THEN get the user info, THEN get the user settings, THEN get the users Todo List (or whatever). Rarely are two steps actually independent enough.
Caveat: This is all just from working with Java, so if Go (or other languages with coroutines) have a smart solution to the complexity and maybe other experiences with the potential gains, I'd loove to hear it.
Your claim about mostly sequential IO is only true in the classic backend case. If you have some kind of microservice mesh (also known as bad design), you will definitely have concurrent reads and sometimes even writes (the worst thing that could possibly happen) per request.
The real problem is that instead of a tightly integrated system, where you have the web server, the database, the cache server, all in close proximity latency wise, you have to work with a system distributed between multiple so-called "cloud providers" with insane latency between each component. It's no wonder you will want to do overlapping IO on some kind of M:N threading model, because each request takes its sweet time on the order of seconds, instead of (sub-)milliseconds.
OS threads are always better than userspace threads if your design is good.
Oh my goodness, many people criticize Go programming without even having a basic understanding of how some fundamental concepts work!
aws lambda is in a similar situation with reactive/async code. aws is going to block on a req/resp, it's not going to reuse that 'vm' for the next request as aws will give that request its own 'instance' of the 'vm' (could be code, image, etc).
Java virtual threads don't color your functions. I know, right?
"You don't know what you are calling. It's incredible! It's a non-leaky abstraction!"
Now take that same opinion with variable typing 😂
It is more expensive (cost wise especially) to create multiple server instances than utilizing all of the cores and threads of a single server instance.
just for perspective, our homebrew policy-enforcing + OIDC proxy go reverse proxy did 2000rps on 400mCPU, and it wasn't even that optimized.
I don't think they're neccesarily talking about I/O or putting each request on its own core, instead I think they're saying for workload processing. If your goal is to use a go-routine to split processing and workload tasks to distribute them between cores... we just do that at the infrastructure layer now. Just have a single nodejs process where you pass it some subset of the array and it does stuff.. then spawn like 50 copies of it with kubernetes by scaling up the pods, and round robin your requests between them. Each pod is isolated and will basically run parrallell using not only different threads or cores, but entirely different machines.
11:45 C# has Task.FromResult to avoid Async functions too
Ah yes, VM spooling and micro services > 3 lines of code running on the machine hardware direct. Big brain
I run my async node apps multithreaded. You fork out to like 60% of available cores to account for uncontrollable node threads like io and gc stuff but otherwise you can totally use multiple threads with both Cluster which even allows you to share entities like a server handle across workers and then actual worker threads if you need to do anything cpu heavy.
Like no one else does this though, maybe like 1% of developers. Lol
This is the devops equivalent of virtualizing linux on each CPU core.
They aren't suggesting a process per request. Just that a process only handles a single request at a time. The lifecycle of the process is not coupled to that of the request.
I got the impression those devs thought goroutines are purely for parallel programming.
Yes, a single-thread CPU can only do one thing at a time, but by using routines you split your program up into smaller tasks/processes. That ONE CPU/Core/Thread now sees 2 processes and can schedule which one needs more resources/execution time. It's more efficient and thus faster.
That even works in real life. Your Brain is a "Single-Core" machine so to speak, but it is still easier to tackle a to-do list, that has many small points, than one that has only a single point encompassing everything to do (Or maybe that is just because of my ADHD Brain)
So the way they propose this should be done instead of doing go-routines makes some sense to me.
Generally when i write backend code in a microservice architecture the services are designed to be small and have as little CPU and memory footprint as possible. In those scenarios, instead of scaling out "internally" in the microservice by using go-routines. You should scale within the Kubernetes cluster instead. And from a architecture standpoint, that makes sense I think.
That said, I don't agree with them that go-routines are useless. I've had plenty of use for them even within that type of architecture. Every single time reach out to multiple different external source, where the calls are not dependent on each other, I of course reach for go-routines. Every time I create a API server and I want the ping endpoint to live on a separate port, I use go-routines.
There are still plenty of scenarios where go-routines makes a lot of sense.
But what is the benefit of having extremely small memory footprint processes if you have to use more processes?
Because total CPU/memory footprint of the whole system is what actually matters...
Web developers think that programming is about the web only 😅
also since you brought up serverless, that's not how serverless works either. so for this example I'm going to specifically use aws lambda and golang, but the exact same thing applies no matter what serverless system you were using. so basically what happens is that when a request comes in, the serverless system checks to see if there is a function that is available to handle the request. if not it creates a new one. so let's say that it's a service that only gets one request per hour, so everything is scaled to zero and there are no currently running functions whenever a request comes in. what will happen is it will always spin up a new instance, it will handle the one request, then it will get torn down. but if we say it gets two requests back to back every hour, it will still only spin up one instance of the function. this is because it keeps the function around for a bit. and with golang in particular, but you can also do the same thing with Python, you can actually set up all of your database connections and such and then share them between requests.
to get a bit more specific about what happens in between requests, this is AWS lambda specific by the way, what happens is that you get an initial grace period during startup that lets you do things like set up database connections, global variables, etc and you get about 100 milliseconds to do all that. then your process gets sent a sig stop, so it can't run on the CPU anymore. then a request comes in and AWS lambda sends your process a sig cont allowing it to be on the processor again. this is how AWS enforces their billing and keeps you from basically getting free background compute.
while your process is not sigstop'd, your program is free to do whatever it wants. so as long as requests are coming in at some rate, you can maintain all of your database connections and stuff. you can actually even do true multithreading if they give you enough CPU.
what they talk about is kubernetes which is gnu parallel on steroids. I still disagree with this take because of what Rob Pike said about them in the first place, that designing with concurrency in mind, even if it won't run in parallel is useful due to the organization it provides for the application.
This has nothing to do with K8s per se; as you can run concurrent code on 1/10 of cpu if you desire. Concurrency is not bound by the amount of CPU cores you have, parallelism on the other hand, is.
Using golang you shouldn't immediately worry about the number of core or cpu-threads there are. The biggest advantage to go routines is the time saving (concurrency) on things not just isolating inbound requests. But if you have nothing that can benefit from concurrency then there isn't a use for them, but it's hard to stretch that into "useless".
Here's the thing - lambda is already one generation behind.... for better scaling and resource utilization you should be using AppRunner and CloudRun....
Yet even on lambda green thread is still super useful, e.g. if you are connecting to database in lambda, an async model will allow you to do that in a non blocking manner...
.... concurrency has (pretty much) nothing to do with parallelism, which is what I do not understand what they are talking about....
You can treat serverless JS lambdas (or really any language lambdas) as a single process. They do indeed serve one request at a time.
Goroutines make the program crash if they modify a map at the same time without a mutex. Not sure if that's great.
Channels are great in theory, not very useful in practice. They also have some pitfalls.
I also wonder... Why is IO as values function coloring that sucks, but Result is not? We like the benefits of errors as values, right? I also like the benefits of IO as values. I do want to know if you are going to do some random IO to compute the value you're giving me instead of entirely using things passed as arguments. Helps me know how large the surface area for errors is.
they meant allocation half of a CPU to a kubernetes pod. So goroutines would be concurrent but there are no parallel benefits.
Greenthreads/async: "We don't actually need an entire OS thread to handle these things, we can just do cooperative userspace threading for this problem because we know there are no long tight loops that'll cause problems. This'll let us handle a lot more requests on a single server without the massive memory and context switch overhead of spawning an entire OS thread for each connection"
These guys, apparently: "What if instead of just spawning a new thread for each connection, we spawn an entire new process for every request?"
These guys, later: "Greenthreads are useless because you're spawning a new process for every request anyway" (never mind that even if you're doing that, there can be plenty of stuff for a single request that can be done concurrently like any sort of database shit)
Bit of a misunderstanding there by Prime, the guy on the podcast was saying they just run 1 process per pod, which handles requests (which is very different to spawning a process for each request). Still, I agree that it is a bad take that this is the only use case ever. Also does not really make goroutines useless. Running a single process per pod is definitely not as efficient as sharing resources(memory, cpu spikes, io) between requests, but if feasible, it is a great way to reduce maintenance effort. It also makes it really simple to scale, even though it is expensive.
K8s is the new Apache running PHP 😂😂😂. One milli core per request?!? 😢
Kotlin has this, too and it's called Coroutines.
I want some of that kush the podcast host was smoking
It's quite an amusing argument, 'go threads are useless because of kubernetes' ... which is written in go, and is working very hard behind the scene, to save the bacon of some rather less efficient languages.
He's assuming zero waiting on IO?
One pattern I find with go developers is they sometimes lack basic computer science knowledge. Like they don't understand the difference between blocking and nonblocking IO, the difference between parallelism and concurrency.
why would go developers be the ones that don’t understand that
Funny when the basics for channel is that they are blocking LOL
I don't think they're saying that a new process is start/killed for each request, the processes can be long-running, just that they only serve one request at a time. It still doesn't make much sense though.
It's not unlike how lambda stuff works.
Most of the lambda runtimes on AWS reuse lambda instances, but only once the last one is done.
It's just not a great way to build a real system, it's just an okay one to simply do exactly what lambda is trying to do.
Sure it is, if you want to run 100 instances of the same container on Kubernetes you can, but it is much more efficient running 10 containers and use Goroutines. Goroutines run into threads, containers are processes. Spawning processes is more expensive than threads.
Ok, I cannot go longer than 2 minutes without dying of cringe overdose
That's like saying epoll is useless syscall. That's nonsense, it's base of any concurrent event loop thing
As @theprimeagen said, I'm not an infrastructure expert and it is not totally clear which setup are they talking about. That being said, if goroutines are useless since some time ago, people developing Kotlin and Java should be stupid, as they developed coroutines and virtual threads afterwards. That sounds like a proof by contradiction to me (as I don't think people developing Kotlin and Java are stupid).
I feel like Prime completely misunderstood what they're saying. They're talking about a pod instance of a long running program per thread. The process itself just uses no threading. Basically, think of how NodeJS scales.
And the guys sells an online course...
I think goroutines are not the best solution. You can call functions asynchronously but you can't get there result. To get the result, you have to pass a channel for communication. Having to pass the channel makes it colored in some sense. The way zig does it is the best of both worlds. You can call any function synchronously or asynchronously. If you call it synchronously (`const x = foo()`), it will return the normal value. If you call it asynchronously however (`const x = async foo()`), it will return a frame. This is a special zig type which encapsulates a function frame. You can await the frame afterwards to get the result.
Even better is that you can specify how the frames will run (your own event loop), or you can use the one the stdlib provides. (It worked on zig 0.9, but with the new self hosted compiler they are rewriting it should work again in the next release)