Is it possible to prevent Localstack from downloading the Gremlin server every time?

Question

Each time I start localstack with testcontainers, when I create the Neptune server instance localstack is downloading and installing a Gremlin server to emulate Neptune. This slows down very much my integration tests. Can I avoid this? I can't find any info on this in the official documentation.

Answer 1

This is a bit tricky. In general it's possible to configure Testcontainers and define a mount for /var/lib/localstack (which is what you would do when running LocalStack in standalone, see also https://docs.localstack.cloud/references/filesystem/#localstack-volume).

Testcontainers provide the function withFileSystemBind for that purpose.

Your code would look something like this:

 @Rule
 public LocalStackContainer localstack = new LocalStackContainer(localstackImage)
        .withExposedPorts(4510, 4511, 4512, 4513, 4514) // TODO the port can have any value between 4510-4559, but LS starts from 4510
        .withEnv("LOCALSTACK_API_KEY", api_key) // TODO add your API key here
        .withFileSystemBind("tmp-testcontainer", "/var/lib/localstack", BindMode.READ_WRITE);

 @Test
 public void testNeptune() {
        NeptuneClient neptune = NeptuneClient
                .builder()
                .endpointOverride(localstack.getEndpointOverride(LocalStackContainer.EnabledService.named("rds")))
                .credentialsProvider(StaticCredentialsProvider.create(
                        AwsBasicCredentials.create(localstack.getAccessKey(), localstack.getSecretKey())
                )).region(Region.of(localstack.getRegion())).build();

        software.amazon.awssdk.services.neptune.model.CreateDbClusterResponse response  = neptune.createDBCluster(software.amazon.awssdk.services.neptune.model.CreateDbClusterRequest.builder().engine("neptune").databaseName("hello").dbClusterIdentifier("testa").build());
        software.amazon.awssdk.services.neptune.model.DescribeDbClustersRequest request = software.amazon.awssdk.services.neptune.model.DescribeDbClustersRequest.builder().dbClusterIdentifier("testa").build();
        software.amazon.awssdk.services.neptune.model.DescribeDbClustersResponse describedb = neptune.describeDBClusters(request);

        // wait for db to be ready
        while(! describedb.dbClusters().get(0).status().equalsIgnoreCase("available")) {
            describedb = neptune.describeDBClusters(request);
        }
        assert describedb.dbClusters().get(0).status().equalsIgnoreCase("available");

    }

Using this bind-mount you will see a subfolder lib where the gremlin/tinkerpop server will be loaded. On re-run it will not be downloaded anymore because it's already there.

But:

• what I found is that TestContainers kills the container rather than stopping it (there is also an open github issue)
• this prevents LocalStack of doing some proper cleanup before shutting down
• the tmp folder potentially has some left over data that will interfere with subsequent test runs.

So you would need to manually cleanup the tmp folder in the mounted directory (in my sample it's a subdirectory of tmp-testcontainer).

Hope that helps!