This post provides some experiences from making a simple Java app (well, the actual app does something this is just a demo) that connects to Bitfinex to get the funding rates run as a GraalVM native image.
xChange is one of the most famous Java libraries for integrating with a large number of crypto exchanges. It provides a simple API that behind the scenes uses reflection heavily, so it seemed like an interesting challenge. The challenge? Create a simple application that gets the last three 15-minute candles for funding offers. The Java code is as simple as the following:
final Exchange bitfinex = ExchangeFactory.INSTANCE.createExchange(BitfinexExchange.class);
final BitfinexMarketDataServiceRaw bitfinexMarketDataServiceRaw = (BitfinexMarketDataServiceRaw) bitfinex.getMarketDataService();
final List<BitfinexCandle> bitfinexCandles = bitfinexMarketDataServiceRaw.getFundingHistoricCandles("15m", "fBTC", 2, 3);
bitfinexCandles.stream().forEach(System.out::println);
Iteration 1: Compile as Java and run on GraalVM JVM in Docker
Native images have many restrictions, so the first attempt was to compile it to Java, create a Docker image and run it on AWS Fargate as a scheduled task to run once every 15 minutes. The Fargate resource allocation is 512MB of Task Memory and 0.25 of vCPU.
Using the IntelliJ New Quarkus project wizard also generates a set of Docker files for fast-jar/native docker images. Creating a fast-jar for Quarkus is as simple as the following:
# Before building the container image run: # # ./gradlew build -Dquarkus.package.type=fast-jar # # Then, build the image with: # # docker build -f src/main/docker/Dockerfile.fast-jar -t quarkus/quarkus-xchange-native-demo-fast-jar .
The initial attempt created a docker image that was 150MB in size, the application took 9.3 seconds to startup and CPU usage spiked to about 100% and memory to 14% .
[io.quarkus] (main) demo on JVM (powered by Quarkus 1.11.1.Final) started in 9.307s
CPU Usage Docker Image (JVM – Version)
Memory Usage Docker Image (JVM – Version)
Not bad, but since this is a very short lived application, it consumes rather a lot of resources. Let’s try to use GraalVM native image generation. For this experiment we used GraalVM version: 21.0.0.2.r11.
Iteration 2: Compile as Native Image and Run in Docker
Creating a native image is a rather challenging process when you include third party libraries, especially if they use reflection. It is important that such features of standard JVM are defined in configuration files as mentioned in the GraalVM documentation.
Native images are built ahead of runtime and their build relies on a static analysis of which code will be reachable. However, this analysis cannot always completely predict all usages of the Java Native Interface (JNI), Java Reflection, Dynamic Proxy objects (java.lang.reflect.Proxy), or class path resources (Class.getResource). Undetected usages of these dynamic features need to be provided to the native-image tool in the form of configuration files.
Luckily GraalVM provides an agent with the ability to automatically generate the GraalVM configuration files.
Iteration 2 – Step 1: Use the Agent to automatically generate the GraalVM native generation configuration files
The xChange library uses reflection a lot. In GraalVM you can explicitly specify which classes use reflection. It provides a method to automatically detect reflection items on a simple run:
java -agentlib:native-image-agent=config-output-dir=./agent-lib-config -jar ./build/quarkus-app/quarkus-run.jar
This code creates a set of files in the agent-lib-config directory that we defined above. GraalVM searches for those files in the src/resources/native-image directory, so we copy/paste all the automatically generated files there.
Iteration 2 – Step 2: Run the native image generation using the generated files
Trying to run the native image generation:
./gradlew clean build -Dquarkus.package.type=native
The first error that we get is:
Could not resolve com.sun.proxy.$Proxy23 for reflection configuration. Reason: java.lang.ClassNotFoundException: com.sun.proxy.$Proxy23
We need to get rid of those erroneous reflection classes in reflect-config.json. Now we try again the native image generation command above. This time the native-image generation succeeds! Now let’s try running the native image:
./build/quarkus-xchange-native-demo-1.0-SNAPSHOT-runner
This time we get this error:
2021-03-08 02:15:44,947 ERROR [io.qua.run.Application] (main) Failed to start application (with profile prod): java.lang.NoSuchMethodException: javax.ws.rs.QueryParam$$ProxyImpl.value()
at java.lang.Class.getMethod(DynamicHub.java:1089)
at si.mazi.rescu.AnnotationUtils.getValueOrNull(AnnotationUtils.java:45)
It appears the problem is in the rescu library that xChange is using, as it was using the class of the annotation for reflection. Patching it was relatively easy. The patched release is tagged here: https://github.com/cyrus13/rescu/commit/aeb94695a7d8073b020a65e08df9ca6b2e17076c
Then I created a version of xChange that uses this version of the library: https://github.com/cyrus13/XChange/releases/tag/native-image-friendly
Using those versions seemed to solve the issue. Now let’s try again:
Iteration 2 – Step 3: Results of execution on AWS
We replaced the docker image of the fast-jar with the native image on AWS. It was running on the same AWS task definition as the JVM image (i.e. allocating the same resources). Consequent runs showed that the native application docker image started in 33ms
[io.quarkus] (main) demo on JVM (powered by Quarkus 1.11.1.Final) started in 0.033s
Also CPU usage spiked to around 45% with an average spike of 25%, and memory usage spiked to 4.6%!
Comparison Table
| Metric | GraalVM JVM | Native-Image |
|---|---|---|
| Docker Image Size | 158 MB | 56 MB |
| Image Start Time | 9.307 seconds | 0.033 seconds |
| CPU Spikes | 50 - 100 % | 7 - 35 % |
| Memory Spikes | 5 - 13 % | 0.8 - 4.8 % |
Please note that fargate has a minimum charge of 1 minute so this whole optimization is useless from a cost saving point of view, it’s just for the nerdiness of it!
A lamda would be a more appropriate solution from a cost saving perspective. Will try that fairly soon.
Source Code
Source code is available on: https://github.com/cyrus13/anastasakis-net-sample-code/tree/master/quarkus-xchange-native-demo