Our Blog
Sharing Our Experiences
- Java project & consultancy
- Cloud Native Development
- AI & IoT
Introduction
Java 19 will be released the 20th of September and with it, will come the first preview of Virtual Threads and the first incubator of Structured Concurrency. Both features are part of Project Loom, but what is Project Loom and what can we expect of these features?
What is Project Loom?
Project Loom is one of the many OpenJDK projects and its goal is to overhaul the concurrency model of the language in which Virtual Threads and Structured Concurrency will play a big part. The main reason this project was started was because a lot of the other popular programming languages support a form of async/await while you need a third-party library in Java to do this.
Some background
In the early versions whilst designing the multithreading API, a choice had to be made between mapping every Java thread to an OS thread or using User Mode threads. At the time, benchmarks showed worse performance while using User Mode threads and they also caused a higher memory consumption.
Load requirements were a lot lower back in the day and today, the Java server concurrency has run into a few problems. The thread per request model causes the thread to be taken as long as the request has not been finished. This means that the thread cannot be used for anything else, even when it is waiting for a third-party call like an SQL query or HTTP request to complete.
We often use a thread pool with a predefined number of threads because the number of threads an OS can have, is limited and because it is a costly operation to make threads. This means that only a certain amount of request can be handled at the same time. Adding a thread to this pool will often not improve performance. Sometimes it will even take longer for a request to complete, and it will increase the time the CPU spends context switching between threads. Spawning more nodes is an option but not a cheap one, definitely not in this day and age of Cloud providers.
Coroutines to the rescue
With other programming languages having adopted different concurrency models already, it gave Java a chance to look at what others did. Whilst most chose for the Promises and async/await option, coroutines (not to be mistaken with Kotlin’s coroutines) has been chosen for Java.
Coroutines is a combination of the terms, continuations and routines:
- Continuations: when the last step of a function A is to call function B, B is considered a continuation of A.
- Routines: reusable pieces of code with an immutable set of instructions that are called multiple times during execution.
Coroutines are suspendable tasks. They can be suspended and resumed at any time because they remember their state and stack trace. It is also possible for them to give control over the carrier thread to other coroutines.
Virtual Threads
Virtual Threads
The coroutine model will be introduced with the Virtual Threads feature. To make migrations easy, the Thread class has been (re)used. The class will remain mostly the same and will use the same API, only a few extra methods will be available to make you able to switch between virtual and the old platform threads as well as the possibility to check which type of thread you are using.
The virtual thread will be able to mount and unmount from the carrier thread. The carrier thread being the platform thread that maps one to one with an OS thread. Every time the virtual thread makes a blocked call, it will be unmounted from the carrier thread, giving another virtual thread the opportunity to continue/start. Once the blocked call has been complete, the virtual thread will again be mounted on a platform thread and be able to continue.
This first implementation is not yet perfect though. It is still possible for the virtual thread to sometimes be pinned to the carrier thread:
- While executing a native method or foreign function.
- While executing inside a synchronized block or method.
The virtual thread will stay on the carrier thread until completion and be blocking. To somewhat circumvent the performance decrease this would cause, an extra platform thread would temporarily be created. If a synchronized block or method is taking a really long time to complete, using a ReentrantLock instead, could fix the blocking.
Why learn if you can unlearn?
With the Thread class being used for Virtual Threads, we do not really need to learn new things. We do need to unlearn a few things, however.
When using virtual threads, we should never use a thread pool for them as they are very cheap to create and will be garbage collected once they are no longer needed. If you want to cap the number of requests that at a given time can, for example, use your SQL database, it would still not be a good idea to use a thread pool. Instead, you should use the Semaphore class as it will better show your intent.
We should also avoid using thread local variables. With a high number of virtual threads, they might cause memory problems. Some people at Oracle even say that thread local variables should have never been exposed to the end user, so it is probably best to only use them after careful considerations.
Advantages
Virtual Threads will bring with it, a bunch of advantages:
- Context switching will be managed by the JVM and no longer by the CPU.
- It will be cheap to start and stop threads as you do not have to make a syscall to create and a syscall to delete a thread with syscalls being such a costly operation.
- The hard upper limits imposed by the OS only being able to handle so many threads, will be removed and millions of (virtual) threads will be a possibility.
- Threads will also have a resizable stack that is not fixed to 1mb. A virtual thread will often spawn other virtual threads that have to take care of a single operation, so their stack can and will be relatively small.
- In the future, it is possible that Tail/Call optimization will be made and that the virtual thread will be optimized/optimizable for the task it has to complete.
When not to switch
In some cases, there will be no point in switching to virtual threads.
- Applications with a small load will not benefit from virtual threads as there is a performance cost to managing virtual threads.
- CPU intensive apps will not benefit much either as the virtual thread will stay on the carrier thread during the entire time mathematical computations are made.
- When your bottleneck is not the thread count but for example your database, using virtual threads will not fix that problem as they do not make requests complete faster.
Interesting links
Interesting links
If you are interested in Project Loom and want to know more. Below are links to the Virtual Threads and Structured Concurrency JEP’s, an article and a video. I would definitely recommend checking out the video as it shows the virtual threads in action:
