I have a microservice deployed on GKE. I was setting up logging and trace, and logs are correctly displayed on Google Cloud Logging, but the traces are not showing up in Cloud Trace. It's a maven Spring Boot 3 project, so Sleuth is no longer supported and I figured I should use OpenTelemetry due to how much documentation there is on the subject, and I've been tinkering around with the configuration.
Since the traces work fine for local logging exporter, I figure I must be missing some component for the traces to propagate correctly in deployment (using a different exporter). It's definitely not a ServiceAccount IAM issues since the ServiceAccount should have all the necessary privileges to write traces and logs.
Here're the dependencies I use for tracing:
<dependency>
<groupId>io.opentelemetry.instrumentation</groupId>
<artifactId>opentelemetry-spring-boot-starter</artifactId>
</dependency>
<dependency>
<groupId>io.opentelemetry.instrumentation</groupId>
<artifactId>opentelemetry-spring-webmvc-6.0</artifactId>
</dependency>
<dependency>
<groupId>io.opentelemetry.instrumentation</groupId>
<artifactId>opentelemetry-spring-webflux-5.3</artifactId>
</dependency>
<!-- OpenTelemetry exporter -->
<!-- replace this default exporter with your OpenTelemetry exporter (ex. otlp/zipkin/..) -->
<dependency>
<groupId>io.opentelemetry</groupId>
<artifactId>opentelemetry-exporter-logging</artifactId>
</dependency>
<dependency>
<groupId>com.google.cloud.opentelemetry</groupId>
<artifactId>exporter-auto</artifactId>
<version>0.27.0-alpha</version>
</dependency>
And here's the application.yaml configs added for configure traces
otel:
service:
name: profile-service
traces:
sampler:
probability: 1.0
exporter: logging
metrics:
exporter: logging
logs:
exporter: logging
This works fine locally, and I'm seeing traces/spans/metrics, but in deployment the exporter fields are otel.traces.exporter=google_cloud_trace and otel.metrics.exporter=google_cloud_monitoring, as noted by the first resource I list below.
Here are the resources I used: Google Cloud Trace Exporter Spring Boot Instrumenting with OpenTelemetry and other Github discussions related to the topic which've led me to the current situation. I'd appreciate any guidance/feedback.