🗒️ l-lin

Search

SearchSearch

✏️ 2024-07-12 🗓️️ Log

GraalVM and AWS Lambda or solving Java cold start problem

src: GraalVM + AWS Lambda or solving Java cold start problem | by Aleksandr Filichkin | Medium - 2021-08-14

Abstract

GraalVM can solve the Java cold start issue where the lambda billing is based on the runtime duration. So the faster the application starts, the cheaper the lambda will cost.

Do you also like Serverless and Java, but hate a Java code start? In this article, we are going to demonstrate how to solve Java cold start issue in AWS Lambda.

Let’s write some book microservice and measure the performance.

As usual all code you can see in my Github: https://github.com/Aleksandr-Filichkin/java-graalvm-aws-lambda

Our test architecture

API-Gateway → AWS Lambda → DynamoDb

Version 1 (plain Java without improvements)

  • Java 11
  • AWS SDK-V2 for DynamoDB(extended DynamoDb client)
  • No DI (Spring, Dagger, etc)
  • No special frameworks

Code is here: https://github.com/Aleksandr-Filichkin/java-graalvm-aws-lambda/tree/main/lambda-v1

Our handler is:

public class BookHandler implements RequestHandler<APIGatewayProxyRequestEvent, APIGatewayProxyResponseEvent> {
 
    private static final String TABLE_NAME = "books";
    private final RequestDispatcher requestDispatcher;
 
    public BookHandler() {
        DynamoDbEnhancedClient dynamoDbEnhancedClient = DynamoDbEnhancedClient.create();
        DynamoDbTable<Book> dynamoDbTable = dynamoDbEnhancedClient.table(TABLE_NAME, TableSchema.fromBean(Book.class));
        requestDispatcher = new RequestDispatcher(new EnhancedClientBookStorage(dynamoDbTable), new ObjectMapper());
    }
 
 
    @Override
    public APIGatewayProxyResponseEvent handleRequest(APIGatewayProxyRequestEvent apiGatewayProxyRequestEvent, Context context) {
        return requestDispatcher.dispatch(apiGatewayProxyRequestEvent);
    }
}

Result

REPORT RequestId: e89b743e-bf08–4c8a-9783–0772102d4e90 Duration: 10845.21 ms Billed Duration: 10846 ms Memory Size: 256 MB Max Memory Used: 168 MB Init Duration: 2650.86 ms

Cold start for version 1 (256 Mb)

Version 2 (plain Java with improvements)

  • Java 11
  • AWS SDK-V2 for Dynamodb
  • No DI (Spring, etc)
  • No special frameworks
  • Utilize CPU burst on startup (move everything to static, warm-up dynamoDB client)
  • Reduce dependencies(exclude Netty)
  • Specify AWS Regions
  • Specify Credential Provider

Code is here: https://github.com/Aleksandr-Filichkin/java-graalvm-aws-lambda/tree/main/lambda-v2

Handler:

public class BookHandler implements RequestHandler<APIGatewayProxyRequestEvent, APIGatewayProxyResponseEvent> {
 
    private static final String TABLE_NAME = "books";
 
    private static final EnhancedRequestDispatcher ENHANCED_REQUEST_DISPATCHER = initDispatcher();
    static {
        ENHANCED_REQUEST_DISPATCHER.warmUp();
    }
 
 
    private static EnhancedRequestDispatcher initDispatcher() {
        DynamoDbEnhancedClient dynamoDbEnhancedClient = DynamoDbEnhancedClient.builder()
                .dynamoDbClient(DynamoDbClient.builder()
                        .credentialsProvider(EnvironmentVariableCredentialsProvider.create())
                        .region(Region.US_EAST_1)
                        .build())
 
                .build();
        DynamoDbTable<Book> dynamoDbTable = dynamoDbEnhancedClient.table(TABLE_NAME, TableSchema.fromBean(Book.class));
        return new EnhancedRequestDispatcher(new EnhancedClientBookStorage(dynamoDbTable), new ObjectMapper());
    }
 
 
    @Override
    public APIGatewayProxyResponseEvent handleRequest(APIGatewayProxyRequestEvent apiGatewayProxyRequestEvent, Context context) {
        return ENHANCED_REQUEST_DISPATCHER.dispatch(apiGatewayProxyRequestEvent);
    }
}

Result

REPORT RequestId: 236850d2–3b85–4b98–9a8f-15dee608e212 Duration: 4037.08 ms Billed Duration: 4038 ms Memory Size: 256 MB Max Memory Used: 170 MB Init Duration: 3604.04 ms

Billable time reduced 2.5 times!

X-Ray trace for the first call

What is AWS Lambda Custom runtime?

  • Just a function.zip with a bootstrap shell script or binary executable file (compiled for Amazon Linux)
  • Any programming language
  • Not a new feature (since 2018)

How can we compile Java code to a binary file? Right, GraalVM Native

GraalVM

GraalVM JIT and AOT

AOT vs JIT: Startup Time

JIT:

  • Load JVM executables
  • Load classes from the file system
  • Verify bytecodes
  • Start interpreting
  • Run static initializers
  • First-tier compilation
  • Gather profiling feedback
  • Second tire compilation(C2 or GraalVM)
  • Finally, run with the best machine code

AOT:

  • Load executable with a prepared heap
  • Immediately start with the best machine code

AOT vs JIT

Version 3 (AWS Lambda Custom Runtime + GraalVM)

Code is here https://github.com/Aleksandr-Filichkin/java-graalvm-aws-lambda/tree/main/lambda-v3

To build a native binary I’m using Docker:

FROM ghcr.io/graalvm/graalvm-ce:latest
 
RUN gu install native-image
WORKDIR /tmp/dist
ADD ./lambda-v3/target/lambda-v3-1.0-SNAPSHOT.jar ./app.jar
RUN native-image -jar ./app.jar --verbose --no-fallback
RUN  microdnf install zip
 
ADD bootstrap bootstrap
 
RUN chmod +x bootstrap
RUN chmod +x ./app
 
RUN zip -j function.zip bootstrap app
 
 
ENTRYPOINT ["./app"]

Result

REPORT RequestId: 982648d4–2a68–49fa-9a89-bb7c36519be9 Duration: 372.73 ms Billed Duration: 704 ms Memory Size: 256 MB Max Memory Used: 90 MB Init Duration: 330.61 ms

GraalVM X-Ray tracing for 256MB

Graal native drawbacks:

  • Manual/explicit mapping for reflections
  • Not all libraries can be compiled(closed-world assumption)
  • Too slow (CPU intensive) build time
  • Big size of binary file (for our service the jar size is 8.8MB, GraalVM binary is 56MB)
  • Only Serial GC is supported for GraalVM CE version(Enterprise has G1)

Useful GraalVM tips:

  • Use JVM agentlib to tracks all usages of dynamic features of an execution on a regular Java VM:
$JAVA_HOME/bin/java -agentlib:native-image-agent=config-output-dir=path

Summary

Cold start comparison

What about the warmed-up state?

I sent ~10.000 requests to the single instance of Lambda V2 (Java optimized) and Lambda V3(GraalVM). GraalVM has constant great performance ~7ms, for Java, we have a bad performance at the beginning and then it becomes ~ 15 ms(look like the Second-tier JIT optimization still was not applied)

Orange is GraalVM, red is Java

Conclusion

  • GraalVM solves Java cold start issue and warm-up performance is great
  • GraalVM requires additional explicit configuration
  • Not all libraries can be compiled

Recent Notes

Graph View

Backlinks