How can I add custom labels promtail?

Question

http_listen_address: 0.0.0.0

  http_listen_port: 9080

positions:

  filename: "/var/promtail/positions_${HOST_HOSTNAME}.yaml"

clients:

  - url: http://loki:3100/loki/api/v1/push

scrape_configs:

- job_name: containers

  static_configs:

  - targets:

      - localhost

    labels:

      job: containers_logs

      path: /var/lib/docker/containers/*/*log

  pipeline_stages:

  - json:

      expressions:

        compose_project: attrs."com.docker.compose.project"

        compose_service: attrs."com.docker.compose.service"

  - regex:

      expression: "^/var/lib/docker/containers/(?P<container_id>.{12}).+/.+-json.log$"

      source: filename

  - timestamp:

      format: RFC3339Nano

      source: time

  - labels:

      compose_project:

      compose_service:

  - output:

      source: log

This config is needed for collecting logs in docker containers. But I have my app auth_service. In that I set logging middleware and I get the following log in each request and response: {'X-REQUEST-ID': '7344981b-6d6f-4b72-b4df-f26efc31c677', 'request': {'user_id': 'unknown', 'method': 'POST', 'path': '/auth/api/v1/login', 'ip': '192.168.16.12'}, 'response': {'status': 'successful', 'status_code': '200', 'time_taken': '0.3172s'}} I want to set labels on json keys in order to filter their on grafana. These logs are catching in console inside docker container and are showing in grafana, but I cannot filter their nohow. I have corrected this config:

server:

  http_listen_address: 0.0.0.0

  http_listen_port: 9080

positions:

  filename: "/var/promtail/positions_${HOST_HOSTNAME}.yaml"

clients:

  - url: http://loki:3100/loki/api/v1/push

scrape_configs:

- job_name: containers

  static_configs:

  - targets:

      - localhost

    labels:

      job: containers_logs

      path: /var/lib/docker/containers/*/*log

  pipeline_stages:

  - json:

      expressions:

        compose_project: attrs."com.docker.compose.project"

        compose_service: attrs."com.docker.compose.service"

  - regex:

      expression: "^/var/lib/docker/containers/(?P<container_id>.{12}).+/.+-json.log$"

      source: filename

  - timestamp:

      format: RFC3339Nano

      source: time

  - labels:

      compose_project:

      compose_service:

  - output:

      source: log

  - match:

      selector: '{compose_service="auth_service"}'

      stages:

      - json:

          expressions:

            request_id: X-REQUEST_ID

            user_id: request.user_id

            method: request.method

            path: request.path

            ip: request.ip

            status: response.status

            status_code: response.status_code

            time_taken: response.time_taken

      - labels:

          request_id:

          user_id:

          method:

          path:

          ip:

          status:

          status_code:

          time_taken:

      - output:

          source: log`

And it doesn't work, these logs still don't have labels. And grafana does not talk anything.

I would be glad if you help me. P.S. compose_service is the same label that have auth_service, that have this middleware.