Kotlin Coroutines and Loom

Loyal viewer David had a lot of questions in the comments to the last episode ( • Composing Higher Order... ), where we learned to compose higher order functions to create http4k request handlers. He liked the idea, but was worried about the performance of calling business logic that used Kotlin coroutines from the non-suspend handlers.
This got me wondering whether project Loom and Java virtual threads make this a non-issue.
So today I’ll start by looking at why operating system threads limit the throughput of our servers, and how virtual threads solve that problem. Once we have that working, then we can use more virtual threads to invoke suspend functions from plain-old functions like http4k handlers.
With project Loom I think that we really can have the best of both worlds, and I have the benchmarks to prove it.
In this episode
00:00:55 Create an http4k server and make actual HTTP requests to it
00:05:14 Now use an executor to make the requests
00:07:42 Now submit 1000 simultaneousish requests
00:08:25 Measure just the time to process the requests
00:10:42 Shift the checking of the Responses out of the executor code
00:12:43 Separate setup from the measurement
00:15:00 Calling a slower function is much slower because we exhaust the server thread pool
00:16:42 We can use Loom virtual threads to "not have" a thread pool
00:19:30 The results are in
00:20:15 Some connection reset errors?
00:21:25 Rationalised code to record performance and errors
00:23:08 More throughput results
00:23:45 More on those connection resets
00:27:36 Now what about calling suspend funs?
00:32:59 Review
There is a playlist of http4k content • http4k
If you like this, you’ll probably like my book Java to Kotlin, A Refactoring Guidebook (java-to-kotlin.dev). It's about far more than just the syntax differences between the languages - it shows how to upgrade your thinking to a more functional style.

[@jasonportillo2984]

Does the Coroutines test case use twice as many virtual threads as the non-Coroutines one? runBlocking will block the virtual thread it's called on, and then the actual work on loomDispatcher will create and use another virtual thread. Would this impact performance in a meaningful way?

[@RefactoringDuncan]

That's a jolly good question, and I think that the answer is yes, there is an extra virtual thread to dispatch the runBlocking. Creating and switching to this can't be free, but I think that the coroutines delay code seemed generally to have better throughput than the Thread.sleep.
Roman's KotlinConf talk (th-cam.com/video/zluKcazgkV4/w-d-xo.htmlsi=2M6kOQB9J7KyTVFR) said that virtual threads pay at the point of suspension (in the real world that means IO not sleep;-), whereas coroutines pay less, but at every suspend invocation. That doesn't really help predict the relative actual costs though.
Personally if I'm running on the JVM I think that I'll plump for plain old blocking code now as opposed to the hassle of suspend functions, unless I have a real need for structured concurrency (less likely on the server than the client?). But this way I have a way of using non-blocking coroutine routines if I have code written that way, and I doubt that I will be able to detect the extra virtual thread dispatch.

[@trickyfox777]

Is this codebase available on github?

[@RefactoringDuncan]

I'm afraid not @trickfox777, because it contains code belonging to a client.

[@DavidA-fi9jy]

You said the main problem for Http4k to support coroutines is that the handler isn't a suspend fun... but really there's a bit more to that... and the fact you had to use runBlocking just makes this point even more prominent: Http4k requires the handler to **return** a Response, whereas frameworks like Ktor give you a Call object to call respond on even AFTER the handler lambda exited... this gives one the possibility to NOT use runBlocking, but rather a CoroutineScope with launch { } for handling each request... in that way you don't block the http server's event loop at all and don't have to use double the virtual threads like in the last comment.

[@RefactoringDuncan]

Apart from an extra virtual thread I don’t see the difference. I don’t know for certain, but surely all the server event loop is doing is accepting an incoming connection, creating a thread to process it, and then waiting for another connection. If we run out of threads that will block, if we don’t, and we shouldn’t with virtual threads, then it won’t?
If we have that huge pool of cheap threads, then waiting to return the Response is no skin off the server’s nose. Even in Ktor, or Undertow, something is holding the socket open waiting for the end of the content (and even the trailer) to be available. We are using resources until the whole response is available - http4k just makes that lifecycle explicitly wait until the handler returns.