How to use aerospike with testcontainers on github actions

Question

I am trying to run aerospike on testcontainers with github actions,

I am sometimes getting an error and sometimes not when I run this on github actions, the error i got is below,

integration_test.go:124: error getting aerospike client: error creating cluster client: aero client failed to connect to [127.0.0.1]:32772 timeout:10s: ResultCode: INVALID_NODE_ERROR, Iteration: 0, InDoubt: false, Node: <nil>: Failed to connect to hosts: [127.0.0.1:32772]

My aerospike config is below,

service {
    user root
    group root
    # Number of nodes where the replica count is automatically reduced to 1.
    # paxos-single-replica-limit 1
    # pidfile /var/run/aerospike/asd.pid
    proto-fd-max 15000
    query-threads-limit 100
}

logging {
    # Log file must be an absolute path.
    file ${LOGFILE} {
        context any critical
    }

    # Send log messages to stdout
    console {
        context any info
    }
}

network {
    service {
        address any
        port 3000

        # Uncomment the following to set the `access-address` parameter to the
        # IP address of the Docker host. This will the allow the server to correctly
        # publish the address which applications and other nodes in the cluster to
        # use when addressing this node.
        # access-address <IPADDR>
    }

    heartbeat {
        address any
        # mesh is used for environments that do not support multicast
        mode mesh
        port 3002

        # use asinfo -v 'tip:host=<ADDR>;port=3002' to inform cluster of
        # other mesh nodes

        interval 150
        timeout 10
    }

    fabric {
        address any
        port 3001 
    }
}

namespace persistent00 {
  replication-factor 1
  memory-size 20M
  storage-engine memory
  nsup-period 86400
}

This is how i connect via using test containers,

func startContainer(ctx context.Context) (*aerospikeContainer, error) {
    req := testcontainers.ContainerRequest{
        Image:        "aerospike/aerospike-server:6.2.0.12",
        ExposedPorts: []string{"3000/tcp", "3001/tcp", "3002/tcp"},
        Files: []testcontainers.ContainerFile{
            {
                HostFilePath:      "aerospike.conf",
                ContainerFilePath: "aerospike.conf",
            },
        },
        Cmd: []string{"--config-file", "aerospike.conf"},
        WaitingFor: wait.ForAll(
            wait.ForListeningPort("3000/tcp"),
            wait.ForLog("{persistent00} migrations: complete").WithOccurrence(2),
        ),
    }

    container, err := testcontainers.GenericContainer(ctx, testcontainers.GenericContainerRequest{
        ContainerRequest: req,
        Started:          true,
    })
    if err != nil {
        return nil, err
    }

    return &aerospikeContainer{Container: container}, nil
}

What should I do to make it work?

I tried to use heartbeats and increased withOccurence but it still seems to not work

Answer 1

This thread suggests to upgrade the client (in your case Go client) to the latest version when encountering INVALID_NODE_ERROR: https://discuss.aerospike.com/t/invalid-node-error-code-3-after-some-writes/6037/9

JDogMcSteezy's comment makes sense if it happens sometimes and not always (client is trying to connect to Aerospike before Aerospike is reachable), waitingFor should help in this case.

I'm not familiar with Go, but you can see a stable Java example of using Aerospike with testcontainers which utilizes Playtika's public library to use testcontainers with Aerospike here: https://github.com/aerospike/spring-data-aerospike

Playtika's testcontainers dependency: https://github.com/aerospike/spring-data-aerospike/blob/main/pom.xml#L256

This is a configuration file for testcontainers, it only contains Aerospike's version by default: https://github.com/aerospike/spring-data-aerospike/blob/main/src/test/resources/bootstrap.properties

And this is the configuration class in tests: https://github.com/aerospike/spring-data-aerospike/blob/main/src/test/java/org/springframework/data/aerospike/config/BlockingTestConfig.java#L30