RESTful services: reactive vs blocking

In my previous blog post I described advantages of reactive programming. Since then I wanted to compare how much would we gain if we tested similar blocking and reactive applications. Well, below are my results.

Tools and test

I wrote two sample applications on stacks with similar complexity:

  • reactive: Java + Spring WebFlux (Netty) + Spring Data R2DBC (reactive MySQL client)
  • blocking: Java + Spring MVC (Tomcat) + Spring Data JDBC (blocking MySQL client)

All sources are available on this github repo.

To perform load tests I used tool called wrk. It allows to specify number of threads and amount of open connections over specified time period to generate load on provided endpoint. The command below will keep 50 HTTP connections open using 10 threads over period of 2 minutes:

wrk -t10 -c50 -d2m http://localhost:8080/api/v1/version

For my benchmark I used

  • 100 threads, 100 connections, 5 minutes
  • 200 threads, 200 connections, 5 minutes
  • apps were run on Oracle Java 8 and 11

To see what happens on JVM side I used Visual VM with Visual GC plugin.

To further investigate how native memory was used I run the applications with VM parm -XX:NativeMemoryTracking=summary and used jcmd tool:

jcmd [PID] VM.native_memory summary

Java 8

I started my tests on Java 8. I know it is old but let’s be honest - there are still many companies using it.

100 concurrent users, 5 minutes

During 5 minute long test blocking application was able to process 45978 requests (153.21 requests/sec) with average latency of 651.94 ms.

Reactive application in the same time processed 82746 requests (275.73 requests/sec) with average lateny of 363.48 ms.

200 concurrent users, 5 minutes

For 200 concurrent users the blocking application processed 44649 requests (148.78 requests/sec) with average latency at 1.34 s. During this test there were 160 timeouts.

Reactive application in similar conditions processed 82943 requests (276.39 requests/sec) with latency of 722.97 ms and no timeouts.

Thread and memory usage

The most noticable (and expected) was much lower thread usage by reactive application. Number of threads spawned by applications for 100 concurrent users Number of threads spawned by applications for 100 concurrent users

Number of threads spawned by applications for 200 concurrent users Number of threads spawned by applications for 200 concurrent users

It is worth to mention that reactive application spawned the same number of threads when handling 100 and 200 concurrent users.

Looking at heap memory usage isn’t very useful - it will basically give us just an overview of how much memory is used and we’ll just see how often GC is called.

Memory usage of applications for 100 concurrent users Memory usage of applications for 100 concurrent users

Memory usage of applications for 200 concurrent users Memory usage of applications for 200 concurrent users

From charts above e can see that reactive application caalled GC fewer times - to be precise it was called 38 times which is 18 less than number of GC calls made by blocking application. Number of concurrent users did not affect number of GC calls - was at similar or same level for both tests.

The devil is in details. If we check how native memory was used then we’ll see more differences.

Below native memory usage for test with 100 concurrent users:

Blocking applicationReactive application
458.240kbHeap501.248kb
72.583kbClass72.054kb
141.195kbThread60.699kb
34.071kbCode37.130kb
151.856kbGC151.971kb
472kbCompiler246kb
14.726kbInternal226.387kb
12.297kbSymbol12.226kb
20.68kbMemory Tracking2.051kb
867kbArena chunk7.626kb
888.375kbTotal1.071.638kb

And for 200 concurrent users:

Blocking applicationReactive application
508.416kbHeap538.112kb
72.872kbClass72.392kb
243.994kbThread60.731kb
36.373kbCode37.036kb
151.956kbGC152.043kb
286kbCompiler242kb
16.461kbInternal226.433kb
12.331kbSymbol12.236kb
2.107kbMemory Tracking2.052kb
5.421kbArena chunk1.093kb
1.050.217kbTotal1.102.370kb

Java 11

Java 11 is not that widely used (yet), but perhaps it is a good time to switch? There is already Java 15 and 16 is right around the corner (March 2021). Let’s see how did both applications do on this JVM.

100 concurrent users

When running same test as described above I got following results:

  • blocking application: 45448 requests (151.44 requests/sec), avg. latency 659.50 ms, 1 timeout
  • reactive application: 82897 requests (276.23 requests/sec), avg. latency 362.49 ms, no timeouts

200 concurrent users

  • blocking application: 44318 requests (147.68 requests/sec), avg. latency 1.35 s, 189 timeouts
  • reactive application: 83192 requests (277.21 requests/sec), avg. latency 720.68 ms, no timeouts

Resource consumption

Here also we see very similar memory usage regardless technology used. Memory usage of applications for 100 concurrent users Memory usage of applications for 100 concurrent users

Memory usage of applications for 200 concurrent users Memory usage of applications for 200 concurrent users

Here also we can see that the reactive application called GC fewer times: 39, the blocking one: 50 times. For test with 200 concurrent users reactive application called GC 42 times, blocking application: 51 times.

And how about native memory usage for 100 concurrent users? Here you are:

Blocking applicationReactive application
312.320kbHeap312.320kb
65.929kbClass66.934kb
11.925kbThread4.398kb
33.882kbCode32.423kb
69.115kbGC68.557kb
1.049kbCompiler435kb
2.123kbInternal1.330kb
865kbOther213.680kb
12.995kbSymbol12.984kb
3.066kbMemory Tracking2.909kb
5.063kbArena chunk22.585kb
4kbLogging4kb
23kbArguments25kb
498kbModule261kb
518.857kbTotal738.845kb

And for 200 concurrent users:

Blocking applicationReactive application
312.320kbHeap312.320kb
66.202kbClass67.205kb
17.897kbThread4.221kb
34.753kbCode33.702kb
69.351kbGC68.563kb
603kbCompiler403kb
2.678kbInternal1.427kb
1.658kbOther213.681kb
13.003kbSymbol12.998kb
3.142kbMemory Tracking2.920kb
2.442kbArena chunk1.798kb
4kbLogging4kb
23kbArguments25kb
500kbModule262kb
524.576kbTotal719.529kb

And of course there were much less threads spawned by reactive application. Number of threads spawned by applications for 100 concurrent users Number of threads spawned by applications for 100 concurrent users

Number of threads spawned by applications for 200 concurrent users Number of threads spawned by applications for 200 concurrent users

Summary

It was a bit surprising for me to see that memory consumption was almost similar for both apps on both JVMs (slightly smaller on Java 11). I was expecting reactive application to consume less memory than blocking one, especially when runninig on Java 8. I thought I would notice the memory savings that would come from fewer thead usage. When I dwelved deeper into VM to see how native memory was used I was able to confirm that memory used by threads was much lower. Unfortunately it was used elsewere: Other area for Java 11 and Internal for Java 8.

Fortunately the performace was not a surpise.

Best overall performance was achieved by reactive application when run on Java 11. On the other end, the worst performance had a blocking application running on Java 11. I really thought it’s high time to switch at least to Java 11 - looks it is not the case for traditional backend applications. Still, the performance gain is huge - reactive application was able to process nearly twice the requests than blocking application on both JVMs.

I noticed it also when performing similar tests on t2.micro instance in AWS EC2. The test was spawning 1000 concurrent users that perform requests over 1 minute. Reactive application was able to process 3113 requests (no timeouts) while blocking just processed 1841 requests, 351 had timed out. Average response times for reactive and blocking applications during 1000 concurrent users test, application running on t2.micro in EC2 Average response times for reactive and blocking applications during 1000 concurrent users test, application running on t2.micro in EC2

I can’t wait till Fibers are fully implemented. It may be a game changer for reactive backends. Unfortuantely Java will still have to wait for it’s own “virtual threads” as Project Loom most proably won’t be completely delivered by September 2021.

I’d like to mention one more thing - there is still no reactive version of JPA. I think this is the biggest blocker for reactive backend applications. Quarkus may be first to do something about it as they are working on hibernate-reactive extension. I’ll definitely take a closer look at it.

Are you ready to take a special offer?

Pick our brain and learn how you can benefit from serverless, cloud computing and Amazon Web Services.

Schedule a free reconnaissance call