Connection refused when executing tests using Selenium, Docker (docker-compose.yaml) and cloud build (GCP)

Question

I want to execute the selenium tests in cloud using Docker containers and Cloud Build. Tried veries remote URL's but nothing seems to make the connection.

** docker-compose.yaml file** SE_NODE_GRID_URL and the remote URL need to be same, if not then there is also a error (even in localhost).

version: "3"
services:
  chrome:
    image: selenium/node-chrome:4.16.1-20231219
    shm_size: 2gb
    volumes:
      - /dev/shm:/dev/shm
    depends_on:
      - selenium-hub
    links:
      - selenium-hub:selenium-hub
    environment:
      - SE_EVENT_BUS_HOST=selenium-hub
      - SE_EVENT_BUS_PUBLISH_PORT=4442
      - SE_EVENT_BUS_SUBSCRIBE_PORT=4443
      - SE_NODE_OVERRIDE_MAX_SESSIONS=true
      - SE_NODE_MAX_SESSIONS=5
      - SE_NODE_GRID_URL=http://selenium-hub:4444/wd/hub

  selenium-hub:
    image: selenium/hub:4.16.1-20231219
    container_name: selenium-hub
    ports:
      - "4442:4442"
      - "4443:4443"
      - "4444:4444"

cloudbuild.yaml used:

steps:
  # Load the artifactory and sonar properties
  - name: 'docker/compose:1.22.0'
    args: [ '-f', 'docker-compose.yaml', 'up', '-d' ]

  # Decrypt artifactory and sonar properties
  - name: 'gcr.io/xxx/gcloud'
    args:
      - kms
      - decrypt
      - --ciphertext-file=xxx
      - --plaintext-file=xxx
      - --location=global
      - --keyring=xxxx
      - --key=build-key

  # Append gradle properties file
  - name: 'gcr.io/xx/yy'
    entrypoint: 'bash'
    args: [ '-c', 'cat xxx.properties >> xx.properties' ]

  - name: gradle:jdk11
    entrypoint: 'bash'
    args: [ './gradlew','clean','cucumber' ]
    waitFor:

options:
  machineType: 'E2_HIGHCPU_8'
substitutions:
  _BUILD_FILES_BUCKET_PATH: 'xxx-build-files'

The tests are working fine if connected within local machine using selenium and docker with url http://localhost:4444. but the same is not working using cloudbuild.yaml

The following is the different remote urls that was tried, but nothing seems to be working

http://localhost:4444
http://selenium-hub:4444/wd/hub
http://selenium-hub:4444/
http://chrome:4444/wd/hub
http://chrome:4444/
http://0.0.0.0:4444/wd/hub
http://172.27.0.2:4444