Nginx docker variable substitution does not work

Question

I'm struggling with Nginx docker environment variable substitution, it somehow does not work for me. I'm using nginx version 1.25.3 in a custom dockerfile where I copy my template nginx config file like that:

ARG NGINX_VERSION=1.25.3
FROM nginx:${NGINX_VERSION}

ARG SCRIPTS_DIRECTORY=data/scrypts
COPY ["./${SCRIPTS_DIRECTORY}/99-autoreload.sh", "/docker-entrypoint.d/99-autoreload.sh"]
RUN chmod 755 /docker-entrypoint.d/99-autoreload.sh

ARG TEMPLATES_DIRECTORY=data/nginx
COPY ["./${TEMPLATES_DIRECTORY}/server.conf.template", "/etc/nginx/templates/server.conf.template"]

CMD ["nginx", "-g", "daemon off;"]

Then in my docker compose file I set some nginx environment variables like that:

   environment:
      - NGINX_CONFIG_SERVER_NAME:${NGINX_CONFIG_SERVER_NAME}
      - NGINX_CONFIG_LINUX_HOST:${NGINX_CONFIG_LINUX_HOST}
      - NGINX_CONFIG_WINDOWS_HOST:${NGINX_CONFIG_WINDOWS_HOST}
      - LINUX_EXAMPLE_SERVICE_ROUTE:${LINUX_EXAMPLE_SERVICE_ROUTE}
      - LINUX_EXAMPLE_SERVICE_PORT:${LINUX_EXAMPLE_SERVICE_PORT}
      - WINDOWS_EXAMPLE_SERVICE_ROUTE:${WINDOWS_EXAMPLE_SERVICE_ROUTE}
      - WINDOWS_EXAMPLE_SERVICE_PORT:${WINDOWS_EXAMPLE_SERVICE_PORT}

Then when I run docker-compose up I use env file with values for that. I do not updated command for nginx in docker-compose (since that can broke nginx startup scripts).

I checked out the logs of container and it stated nginx did run variable substitution:

...
/docker-entrypoint.sh: Launching /docker-entrypoint.d/20-envsubst-on-templates.sh
20-envsubst-on-templates.sh: Running envsubst on /etc/nginx/templates/server.conf.template to /etc/nginx/conf.d/server.conf
...
2023/12/11 14:46:34 [emerg] 1#1: unknown "nginx_config_linux_host" variable
nginx: [emerg] unknown "nginx_config_linux_host" variable

When I inspected the container, variables are set there correctly, but it looks like nginx does not see environment variables and not set it, as when I inspected the config file it was exctracted from template, but the content was untouched.

My config for nginx:

server {
    listen 80;
    listen [::]:80;

    server_name ${NGINX_CONFIG_SERVER_NAME};

    #For test purposes only, update later
    location ${LINUX_EXAMPLE_SERVICE_ROUTE} {
        add_header Content-Type text/html;
        proxy_pass http://${NGINX_CONFIG_LINUX_HOST}:${LINUX_EXAMPLE_SERVICE_PORT}/;
    }

    #For test purposes only, update later
    location ${WINDOWS_EXAMPLE_SERVICE_ROUTE} {
        proxy_pass http://${NGINX_CONFIG_WINDOWS_HOST}:${WINDOWS_EXAMPLE_SERVICE_PORT}/;
    }
}

Any clues why it didn't work? My best guess is envsubst command somehow does not see my variables. What am I missing?