Starting a Grizzly HTTP server on a fargate container

Question

String baseUri = http://127.0.0.1:8080/abc
HttpServer grizzlyServer = startServer(baseUri);
while (grizzlyServer.isStarted())
{
    TimeUnit.SECONDS.sleep(1);
    loopCount++;
    loopCount = loopCount%60;
    if (loopCount == 0)
    {
        logger.finest("I am still Running");
    }
}
try {
    grizzlyServer.shutdown().get();

} catch (Exception e) {
    logger.warning("Exception stopping HTTP Server: " + e.getLocalizedMessage());       
}

/**
 * Starts the grizzly HTTP server for providing the REST capabilities
 *
 * @param baseUri the base uri of the server
 * @return the grizzly http server
 */
public static HttpServer startServer(String baseUri) 
{
    
    // create a resource config that scans for JAX-RS resources and providers in the given package
    final ResourceConfig rc = new ResourceConfig().packages("com.api");
        
    rc.property(ServerProperties.WADL_FEATURE_DISABLE, true); // disable WADL as we don't use it and it generates runtime warnings
        
    // Register the provider binder - implements jakarta.ws.rs.core.Feature. Then we have some other classes with @Path and @GET annotations, and the provider here is @Inject-ed, so we have access to it.
    rc.register(new ServiceProviderBinder());
        
    // create and start a new instance of grizzly http server
    // exposing the Jersey application at BASE_URI
    HttpServer server = GrizzlyHttpServerFactory.createHttpServer(URI.create(baseUri), rc);
    Logger l = Logger.getLogger("org.glassfish.grizzly.http.server.HttpHandler");
    l.setLevel(Level.FINE);
    l.setUseParentHandlers(false);
    ConsoleHandler ch = new ConsoleHandler();
    ch.setLevel(Level.ALL);
    l.addHandler(ch);
    return server;
}

I can use this approach to start up the server locally, or in a docker container than is run locally using docker-compose up.

If, however, I then deploy this same container to fargate through an ECS Task Definition and service, I get the following in Cloudwatch:

jakarta.ws.rs.ProcessingException: Failed to start Grizzly HTTP server: Address already in use

I get the same if I use 0.0.0.0 instead of 127.0.0.1.

My Task definition configuration contains a port mapping 8080:8080 -

[![task definition mapping][1]][1]

and my Dockerfile is as follows (with the details passed in from docker-compose):

FROM eclipse-temurin:21.0.1_12-jdk
ARG SERVICE_NAME
ARG APP_DIR=/opt/app
RUN mkdir -p ${APP_DIR}
COPY ${SERVICE_NAME}.sh ${APP_DIR}
COPY dependencies ${APP_DIR}/dependencies
COPY ${SERVICE_NAME}.jar ${APP_DIR}
COPY ${SERVICE_NAME}.awstest.properties ${APP_DIR}

USER root

RUN chmod -R 777 ${APP_DIR}

# When the container starts, run the svt startup script
ENV SERVICESCRIPT=${APP_DIR}/${SERVICE_NAME}.sh
CMD exec $SERVICESCRIPT

Task definition:

{
    "family": "appname",
    "containerDefinitions": [
        {
            "name": "appnameQP",
            "image": "012345678901.dkr.ecr.eu-west-2.amazonaws.com/appname:appnameQP",
            "cpu": 0,
            "portMappings": [
                {
                    "name": "8080",
                    "containerPort": 8080,
                    "hostPort": 8080,
                    "protocol": "tcp",
                    "appProtocol": "http"
                }
            ],
            "essential": false,
            "environment": [],
            "environmentFiles": [],
            "mountPoints": [],
            "volumesFrom": [],
            "dependsOn": [
                {
                    "containerName": "initkafka",
                    "condition": "COMPLETE"
                }
            ],
            "ulimits": [],
            "logConfiguration": {
                "logDriver": "awslogs",
                "options": {
                    "awslogs-create-group": "true",
                    "awslogs-group": "/ecs/appname",
                    "awslogs-region": "eu-west-2",
                    "awslogs-stream-prefix": "ecs"
                },
                "secretOptions": []
            },
            "systemControls": []
        },
        {
            "name": "appnamerd",
            "image": "012345678901.dkr.ecr.eu-west-2.amazonaws.com/appname:appnamerd",
            "cpu": 0,
            "portMappings": [],
            "essential": false,
            "environment": [],
            "environmentFiles": [],
            "mountPoints": [],
            "volumesFrom": [],
            "dependsOn": [
                {
                    "containerName": "initkafka",
                    "condition": "COMPLETE"
                }
            ],
            "logConfiguration": {
                "logDriver": "awslogs",
                "options": {
                    "awslogs-create-group": "true",
                    "awslogs-group": "/ecs/appname",
                    "awslogs-region": "eu-west-2",
                    "awslogs-stream-prefix": "ecs"
                },
                "secretOptions": []
            },
            "systemControls": []
        },
        {
            "name": "appnamesm",
            "image": "012345678901.dkr.ecr.eu-west-2.amazonaws.com/appname:appnamesm",
            "cpu": 0,
            "portMappings": [],
            "essential": false,
            "environment": [],
            "environmentFiles": [],
            "mountPoints": [],
            "volumesFrom": [],
            "dependsOn": [
                {
                    "containerName": "initkafka",
                    "condition": "COMPLETE"
                }
            ],
            "logConfiguration": {
                "logDriver": "awslogs",
                "options": {
                    "awslogs-create-group": "true",
                    "awslogs-group": "/ecs/appname",
                    "awslogs-region": "eu-west-2",
                    "awslogs-stream-prefix": "ecs"
                },
                "secretOptions": []
            },
            "systemControls": []
        },
        {
            "name": "appnameRT",
            "image": "012345678901.dkr.ecr.eu-west-2.amazonaws.com/appname:appnameRT",
            "cpu": 0,
            "portMappings": [],
            "essential": false,
            "environment": [],
            "environmentFiles": [],
            "mountPoints": [],
            "volumesFrom": [],
            "dependsOn": [
                {
                    "containerName": "mosquitto",
                    "condition": "START"
                },
                {
                    "containerName": "initkafka",
                    "condition": "COMPLETE"
                }
            ],
            "logConfiguration": {
                "logDriver": "awslogs",
                "options": {
                    "awslogs-create-group": "true",
                    "awslogs-group": "/ecs/appname",
                    "awslogs-region": "eu-west-2",
                    "awslogs-stream-prefix": "ecs"
                },
                "secretOptions": []
            },
            "systemControls": []
        },
        {
            "name": "appnamePS",
            "image": "012345678901.dkr.ecr.eu-west-2.amazonaws.com/appname:appnamePS",
            "cpu": 0,
            "portMappings": [],
            "essential": false,
            "environment": [],
            "environmentFiles": [],
            "mountPoints": [],
            "volumesFrom": [],
            "dependsOn": [
                {
                    "containerName": "mosquitto",
                    "condition": "START"
                },
                {
                    "containerName": "initkafka",
                    "condition": "COMPLETE"
                }
            ],
            "logConfiguration": {
                "logDriver": "awslogs",
                "options": {
                    "awslogs-create-group": "true",
                    "awslogs-group": "/ecs/appname",
                    "awslogs-region": "eu-west-2",
                    "awslogs-stream-prefix": "ecs"
                },
                "secretOptions": []
            },
            "systemControls": []
        },
        {
            "name": "appnameFF",
            "image": "012345678901.dkr.ecr.eu-west-2.amazonaws.com/appname:appnameFF",
            "cpu": 0,
            "portMappings": [],
            "essential": false,
            "environment": [],
            "environmentFiles": [],
            "mountPoints": [],
            "volumesFrom": [],
            "dependsOn": [
                {
                    "containerName": "initkafka",
                    "condition": "COMPLETE"
                }
            ],
            "logConfiguration": {
                "logDriver": "awslogs",
                "options": {
                    "awslogs-create-group": "true",
                    "awslogs-group": "/ecs/appname",
                    "awslogs-region": "eu-west-2",
                    "awslogs-stream-prefix": "ecs"
                },
                "secretOptions": []
            },
            "systemControls": []
        },
        {
            "name": "kafka",
            "image": "012345678901.dkr.ecr.eu-west-2.amazonaws.com/appname:kafka",
            "cpu": 0,
            "portMappings": [
                {
                    "name": "9092",
                    "containerPort": 9092,
                    "hostPort": 9092,
                    "protocol": "tcp"
                }
            ],
            "essential": true,
            "environment": [
                {
                    "name": "KAFKA_CFG_NUM_PARTITIONS",
                    "value": "1"
                },
                {
                    "name": "KAFKA_CFG_LISTENER_SECURITY_PROTOCOL_MAP",
                    "value": "CONTROLLER:PLAINTEXT,EXTERNAL:PLAINTEXT,PLAINTEXT:PLAINTEXT"
                },
                {
                    "name": "KAFKA_CFG_DELETE_TOPIC_ENABLE",
                    "value": "true"
                },
                {
                    "name": "KAFKA_CFG_NUM_IO_THREADS",
                    "value": "8"
                },
                {
                    "name": "KAFKA_CFG_SOCKET_SEND_BUFFER_BYTES",
                    "value": "102400"
                },
                {
                    "name": "KAFKA_CFG_GROUP_INITIAL_REBALANCE_DELAY",
                    "value": "3000"
                },
                {
                    "name": "KAFKA_CFG_SOCKET_REQUEST_MAX_BYTES",
                    "value": "104857600"
                },
                {
                    "name": "KAFKA_CFG_NUM_NETWORK_THREADS",
                    "value": "8"
                },
                {
                    "name": "KAFKA_CFG_ZOOKEEPER_CONNECT",
                    "value": "127.0.0.1:2181"
                },
                {
                    "name": "KAFKA_CFG_LISTENERS",
                    "value": "PLAINTEXT://:9092,CONTROLLER://:9093,EXTERNAL://:9094"
                },
                {
                    "name": "KAFKA_CFG_ADVERTISED_LISTENERS",
                    "value": "PLAINTEXT://127.0.0.1:9092,EXTERNAL://localhost:9094"
                }
            ],
            "environmentFiles": [],
            "mountPoints": [],
            "volumesFrom": [],
            "dependsOn": [
                {
                    "containerName": "zookeeper",
                    "condition": "START"
                }
            ],
            "logConfiguration": {
                "logDriver": "awslogs",
                "options": {
                    "awslogs-create-group": "true",
                    "awslogs-group": "/ecs/appname",
                    "awslogs-region": "eu-west-2",
                    "awslogs-stream-prefix": "ecs"
                },
                "secretOptions": []
            },
            "systemControls": []
        },
        {
            "name": "initkafka",
            "image": "012345678901.dkr.ecr.eu-west-2.amazonaws.com/appname:initkafka",
            "cpu": 0,
            "portMappings": [],
            "essential": false,
            "entryPoint": [
                "/bin/sh",
                "-c"
            ],
            "command": [
                "# blocks until kafka is reachable",
                "kafka-topics --bootstrap-server 127.0.0.1:9092 --list",
                "echo -e 'Creating appname Kafka Topics'",
                "/opt/bitnami/kafka/bin/kafka-topics.sh --bootstrap-server 127.0.0.1:9092 --create --if-not-exists --topic topicnamen1 --replication-factor 1 --partitions 1 --config cleanup.policy=compact",
                "delete --config retention.ms=604800000 --config delete.retention.ms=604800000",
                "/opt/bitnami/kafka/bin/kafka-topics.sh --bootstrap-server 127.0.0.1:9092 --create --if-not-exists --topic topicnamen2 --replication-factor 1 --partitions 1 --config cleanup.policy=compact",
                "delete --config retention.ms=604800000 --config delete.retention.ms=604800000",
                "/opt/bitnami/kafka/bin/kafka-topics.sh --bootstrap-server 127.0.0.1:9092 --create --if-not-exists --topic topicnamen3 --replication-factor 1 --partitions 1 --config cleanup.policy=compact",
                "delete --config retention.ms=604800000 --config delete.retention.ms=604800000",
                "/opt/bitnami/kafka/bin/kafka-topics.sh --bootstrap-server 127.0.0.1:9092 --create --if-not-exists --topic topicnamen4 --replication-factor 1 --partitions 1 --config cleanup.policy=compact",
                "delete --config retention.ms=604800000 --config delete.retention.ms=604800000",
                "/opt/bitnami/kafka/bin/kafka-topics.sh --bootstrap-server 127.0.0.1:9092 --create --if-not-exists --topic topicnamen5 --replication-factor 1 --partitions 1 --config cleanup.policy=compact",
                "delete --config retention.ms=172800000 --config delete.retention.ms=172800000",
                "/opt/bitnami/kafka/bin/kafka-topics.sh --bootstrap-server 127.0.0.1:9092 --create --if-not-exists --topic topicnamen6 --replication-factor 1 --partitions 1 --config retention.ms=200000000 --config delete.retention.ms=200000000",
                "/opt/bitnami/kafka/bin/kafka-topics.sh --bootstrap-server 127.0.0.1:9092 --create --if-not-exists --topic topicnamen7 --replication-factor 1 --partitions 1 --config cleanup.policy=compact",
                "delete --config retention.ms=172800000 --config delete.retention.ms=172800000",
                "/opt/bitnami/kafka/bin/kafka-topics.sh --bootstrap-server 127.0.0.1:9092 --create --if-not-exists --topic topicnamen8 --replication-factor 1 --partitions 1 --config cleanup.policy=compact",
                "delete --config retention.ms=60480000 --config delete.retention.ms=60480000",
                "/opt/bitnami/kafka/bin/kafka-topics.sh --bootstrap-server 127.0.0.1:9092 --create --if-not-exists --topic topicnamen9 --replication-factor 1 --partitions 1 --config cleanup.policy=compact",
                "delete --config retention.ms=604800000 --config delete.retention.ms=604800000",
                "/opt/bitnami/kafka/bin/kafka-topics.sh --bootstrap-server 127.0.0.1:9092 --create --if-not-exists --topic topicnamen10 --replication-factor 1 --partitions 1 --config cleanup.policy=compact",
                "delete --config retention.ms=604800000 --config delete.retention.ms=604800000",
                "/opt/bitnami/kafka/bin/kafka-topics.sh --bootstrap-server 127.0.0.1:9092 --create --if-not-exists --topic topicnamen11 --replication-factor 1 --partitions 1 --config cleanup.policy=compact",
                "delete --config retention.ms=604800000 --config delete.retention.ms=604800000",
                "/opt/bitnami/kafka/bin/kafka-topics.sh --bootstrap-server 127.0.0.1:9092 --create --if-not-exists --topic topicnamen12 --replication-factor 1 --partitions 1 --config cleanup.policy=compact",
                "delete --config retention.ms=604800000 --config delete.retention.ms=604800000",
                "/opt/bitnami/kafka/bin/kafka-topics.sh --bootstrap-server 127.0.0.1:9092 --create --if-not-exists --topic topicnamen13 --replication-factor 1 --partitions 6 --config cleanup.policy=compact",
                "delete --config retention.ms=60480000 --config delete.retention.ms=60480000",
                "/opt/bitnami/kafka/bin/kafka-topics.sh --bootstrap-server 127.0.0.1:9092 --create --if-not-exists --topic topicnamen14 --replication-factor 1 --partitions 1 --config retention.ms=60480000 --config delete.retention.ms=60480000",
                "/opt/bitnami/kafka/bin/kafka-topics.sh --bootstrap-server 127.0.0.1:9092 --create --if-not-exists --topic topicnamen15 --replication-factor 1 --partitions 1 --config retention.ms=604800 --config delete.retention.ms=604800",
                "/opt/bitnami/kafka/bin/kafka-topics.sh --bootstrap-server 127.0.0.1:9092 --create --if-not-exists --topic topicnamen15 --replication-factor 1 --partitions 1 --config retention.ms=604800 --config delete.retention.ms=604800",
                "/opt/bitnami/kafka/bin/kafka-topics.sh --bootstrap-server 127.0.0.1:9092 --create --if-not-exists --topic topicnamen16 --replication-factor 1 --partitions 1 --config retention.ms=604800 --config delete.retention.ms=604800",
                "/opt/bitnami/kafka/bin/kafka-topics.sh --bootstrap-server 127.0.0.1:9092 --create --if-not-exists --topic topicnamen17 --replication-factor 1 --partitions 1 --config cleanup.policy=compact",
                "delete --config retention.ms=604800000 --config delete.retention.ms=604800000",
                "/opt/bitnami/kafka/bin/kafka-topics.sh --bootstrap-server 127.0.0.1:9092 --create --if-not-exists --topic topicnamen18 --replication-factor 1 --partitions 1 --config cleanup.policy=compact",
                "delete --config retention.ms=604800000 --config delete.retention.ms=604800000",
                "/opt/bitnami/kafka/bin/kafka-topics.sh --bootstrap-server 127.0.0.1:9092 --create --if-not-exists --topic topicnamen19 --replication-factor 1 --partitions 6 --config retention.ms=172800000 --config delete.retention.ms=172800000",
                "/opt/bitnami/kafka/bin/kafka-topics.sh --bootstrap-server 127.0.0.1:9092 --create --if-not-exists --topic topicnamen20 --replication-factor 1 --partitions 100 --config retention.ms=6048000 --config delete.retention.ms=6048000",
                "/opt/bitnami/kafka/bin/kafka-topics.sh --bootstrap-server 127.0.0.1:9092 --create --if-not-exists --topic topicnamen21 --replication-factor 1 --partitions 1 --config retention.ms=604800 --config delete.retention.ms=604800",
                "/opt/bitnami/kafka/bin/kafka-topics.sh --bootstrap-server 127.0.0.1:9092 --create --if-not-exists --topic topicnamen22 --replication-factor 1 --partitions 6 --config cleanup.policy=compact",
                "delete --config retention.ms=172800000 --config delete.retention.ms=172800000",
                "/opt/bitnami/kafka/bin/kafka-topics.sh --bootstrap-server 127.0.0.1:9092 --create --if-not-exists --topic topicnamen23 --replication-factor 1 --partitions 1 --config cleanup.policy=compact",
                "delete --config retention.ms=172800000 --config delete.retention.ms=172800000",
                "/opt/bitnami/kafka/bin/kafka-topics.sh --bootstrap-server 127.0.0.1:9092 --create --if-not-exists --topic topicnamen24 --replication-factor 1 --partitions 1 --config retention.ms=60480000 --config delete.retention.ms=60480000",
                "/opt/bitnami/kafka/bin/kafka-topics.sh --bootstrap-server 127.0.0.1:9092 --create --if-not-exists --topic topicnamen25 --replication-factor 1 --partitions 1 --config cleanup.policy=compact",
                "delete --config retention.ms=172800000 --config delete.retention.ms=172800000",
                "/opt/bitnami/kafka/bin/kafka-topics.sh --bootstrap-server 127.0.0.1:9092 --create --if-not-exists --topic topicnamen26 --replication-factor 1 --partitions 1 --config retention.ms=200000000 --config delete.retention.ms=200000000",
                "echo -e 'successfully created the following topics:'",
                "/opt/bitnami/kafka/bin/kafka-topics.sh --bootstrap-server 127.0.0.1:9092 --list"
            ],
            "environment": [],
            "environmentFiles": [],
            "mountPoints": [],
            "volumesFrom": [],
            "dependsOn": [
                {
                    "containerName": "kafka",
                    "condition": "START"
                }
            ],
            "logConfiguration": {
                "logDriver": "awslogs",
                "options": {
                    "awslogs-create-group": "true",
                    "awslogs-group": "/ecs/appname",
                    "awslogs-region": "eu-west-2",
                    "awslogs-stream-prefix": "ecs"
                },
                "secretOptions": []
            },
            "systemControls": []
        },
        {
            "name": "zookeeper",
            "image": "012345678901.dkr.ecr.eu-west-2.amazonaws.com/appname:zookeeper",
            "cpu": 0,
            "portMappings": [
                {
                    "name": "2181",
                    "containerPort": 2181,
                    "hostPort": 2181,
                    "protocol": "tcp"
                }
            ],
            "essential": false,
            "environment": [
                {
                    "name": "ALLOW_ANONYMOUS_LOGIN",
                    "value": "yes"
                }
            ],
            "environmentFiles": [],
            "mountPoints": [],
            "volumesFrom": [],
            "logConfiguration": {
                "logDriver": "awslogs",
                "options": {
                    "awslogs-create-group": "true",
                    "awslogs-group": "/ecs/appname",
                    "awslogs-region": "eu-west-2",
                    "awslogs-stream-prefix": "ecs"
                },
                "secretOptions": []
            },
            "systemControls": []
        },
        {
            "name": "mosquitto",
            "image": "012345678901.dkr.ecr.eu-west-2.amazonaws.com/appname:mosquitto",
            "cpu": 0,
            "portMappings": [
                {
                    "name": "1883",
                    "containerPort": 1883,
                    "hostPort": 1883,
                    "protocol": "tcp"
                }
            ],
            "essential": false,
            "environment": [],
            "environmentFiles": [],
            "mountPoints": [],
            "volumesFrom": [],
            "logConfiguration": {
                "logDriver": "awslogs",
                "options": {
                    "awslogs-create-group": "true",
                    "awslogs-group": "/ecs/appname",
                    "awslogs-region": "eu-west-2",
                    "awslogs-stream-prefix": "ecs"
                },
                "secretOptions": []
            },
            "systemControls": []
        }
    ],
    "taskRoleArn": "arn:aws:iam::012345678901:role/appname_Fargate_ECS_Role",
    "executionRoleArn": "arn:aws:iam::012345678901:role/ecsTaskExecutionRole",
    "networkMode": "awsvpc",
    "requiresCompatibilities": [
        "FARGATE"
    ],
    "cpu": "8192",
    "memory": "16384",
    "runtimePlatform": {
        "cpuArchitecture": "X86_64",
        "operatingSystemFamily": "LINUX"
    }
}```

where appnameQP is the container that is failing.

What do I need to change in my configuration, either in code, Docker or AWS, in order to get this working? Why is port 8080 already in use on the fargate container?


  [1]: https://i.stack.imgur.com/DClEW.png