How Java 12 has made microbenchmarking easier

This month we are focusing on Java 12 new features you should know about. In other articles, we looked at improvements to Garbage Collection, Java 12’s JVM Constants API explained in 5 minutes, and Java 12 Switch Expressions explained in 5 minutes

In this article, we will be covering the introduction of a Microbenchmark suite in JEP 230 and what this means.

What is MicroBenchMarking

Microbenchmarking (for brevity we shall call this MBM) is similar to benchmarking but on a smaller scale. Whilst Benchmarking (for brevity we shall call this BM) would test a system based on real-world scenarios and workloads, MBM tests a small section of a system against artificial scenarios.

The theory behind MBM is that you can test the performance of sections of a system to determine what changes if any are appearing in those sections. This means MBM goes hand in hand with BM when BM shows a change in performance, MBM will enable you to pinpoint where the change was introduced.

If you do some reading around MBM you will find that it is considered difficult to handle correctly. As the benefits of MBM is not the topic of this blog I will not go into too much detail. The basics you need to know is that languages like Java can compile the code to optimize which can give misleading results. Once you include the JIT and multiple levels of compilation the results from MBM can be varied.

Until we get access to JEP 230 we will not know the full extent of these tests but these possible complications should be accounted for.

What is JEP 230

JEP 230 can be a little unclear and has lead to some confusion on some forums as to what it will do.

JEP 230 is the proposal to introduce microbenchmarking tests to Java 12 so that the JVM performance can easily be tested with pre-existing tests. This would be very useful for anyone wishing to work on the JVM itself. The tests that are being added are built using the Java Microbenchmark Harness (JMH). These tests will allow for continuous performance testing on a JVM using these tests. JEP 230 proposes the introduction of around 100 tests with new tests being introduced as new Java versions are released.

What isn’t JEP 230

As mentioned previously, initially there was some confusion around these tests. To help clarify I will also describe what JEP 230 is not.

These tests are not a part of the JVM builds, however they are a part of the source code. This will allow developers if they are required by building the JVM with these tests in place. The tests also do not include JMH in the JDK. JMH will remain separate to the JVM and the JVM will require access to its binaries in order to run the Microbenchmarking tests.

Thoughts

JEP 230 looks to be an interesting addition to Java 12. It will definitely prove useful for JVM developers and those wishing to create a customized JVM. Your average Java user and many developers may find that they will never need to use this new feature.