OpenTelemetry for PHP fails to associate services across API requests

Question

When I utilize Opentelementry for PHP, I encounter an issue where I am unable to visualize the connection between two microservices within the same network in Jaeger.

In one microservice, I initiate a call to an endpoint, "service1/ping," and within the corresponding method, I make an API request to the second service using Guzzle, specifically to the endpoint "service2/api/products/1".

However, when examining Jaeger, I observe that the two microservices are displayed separately, and there is no apparent connection between them.

Has anyone else encountered a similar issue?

Answer 1

After two days of investigation and reading troubleshooting guides in OpenTelemetry and Jaeger, I identified the problem and would like to share my setup with you. Perhaps it could assist someone else facing a similar situation.

Firstly, I am conducting tests on a Symfony 6.3 project using the original library from OpenTelemetry for PHP. The documentation on how to set it up can be found here: https://opentelemetry.io/docs/instrumentation/php/getting-started/

Subsequently, I came across an article that referred to this example: https://github.com/ecotoneframework/php-ddd-cqrs-event-sourcing-symfony-laravel-ecotone, which did not start properly on my machine. Consequently, I began reverse engineering the implementation of OpenTelemetry for PHP.

Here are the extensions I had to install in my Docker environment where PHP is running:

FROM php:8.2-fpm
....
RUN pecl install protobuf && docker-php-ext-enable protobuf
RUN pecl install opentelemetry && docker-php-ext-enable opentelemetry

Note: The preferred method for sending information to Jaeger is gRPC, but unfortunately, I couldn't install it via pecl. I attempted the installation several times, but each attempt took over 2000 seconds, leading me to interrupt the process. Currently, I am using http/protobuf as the protocol - the default one.

Following that, I had to include additional environment variables in my docker-compose.yml file when starting the web server.

  - OTEL_SERVICE_NAME=${PROJECT_NAME}
  - OTEL_PHP_AUTOLOAD_ENABLED=true
  - OTEL_TRACES_EXPORTER=otlp
  - OTEL_METRICS_EXPORTER=otlp
  - OTEL_LOGS_EXPORTER=otlp
  - OTEL_EXPORTER_OTLP_PROTOCOL=http/protobuf
  - OTEL_EXPORTER_OTLP_ENDPOINT=http://jaeger:4318
  - OTEL_EXPORTER_OTLP_TRACES_ENDPOINT=http://jaeger:4318/v1/traces
  - OTEL_EXPORTER_OTLP_METRICS_ENDPOINT=http://jaeger:4318/v1/metrics
  - OTEL_EXPORTER_OTLP_LOGS_ENDPOINT=http://jaeger:4318/v1/logs
  - OTEL_PROPAGATORS=tracecontext,baggage
  - OTEL_LOG_LEVEL=debug
  - OTEL_TRACES_SAMPLER=parentbased_always_on

You should verify that port 4318 is open for Jaeger. This port is utilized for http/protobuf, while port 4317 is reserved for gRPC.

Note: I attempted to add these environment variables to the .env file of my Symfony project, but it proved ineffective. After half a day of investigating the issue, I concluded that OpenTelemetry only initializes properly when the variables are set in the environment of the web server. It might be something worth investigating further to understand why it didn't work from the .env file. If anyone has experience with this and can share insights, it would be greatly appreciated.

The last step is to add the needed libraries for open telemetry in composer.json.

....

"open-telemetry/exporter-otlp": "*",
"open-telemetry/opentelemetry-auto-psr15": "^1.0",
"open-telemetry/opentelemetry-auto-psr18": "^1.0",
"open-telemetry/opentelemetry-auto-symfony": "*",
"guzzlehttp/guzzle": "^7.8",
"guzzlehttp/psr7": "^2.6",

....

"config": {
  "allow-plugins": {
    "php-http/discovery": true,
   },
},

Addressing my question and its corresponding answer: When establishing communication between two services, it is essential to utilize libraries that are compatible with PSR-18 for Request Output and PSR-15 for Request Input. In PHP this would be Guzzle. For further details, you can refer to the documentation available here: https://opentelemetry.io/docs/instrumentation/php/propagation/

In conclusion, when initiating a request from one service to another, you should expect to observe something similar in your Jaeger trace:

Enjoy exploring distributed tracing with OpenTelemetry and Jaeger!