I'd like to ask if there is any way to use tracing incoming requests with the X-Ray SDK for quarkus reactive (Netty) application?
To be more specific, I am trying to trace all hits to dynamoDB.
Following AWS documentation (https://docs.aws.amazon.com/xray/latest/devguide/xray-sdk-java-filters.html) I can see solutions only for tomcat/spring.
Example solution is to provide bean for javax.servlet.Filter:
@Configuration
public class WebConfig {
@Bean
public Filter TracingFilter() {
return new AWSXRayServletFilter(SegmentNamingStrategy.dynamic("MyApp", "*.example.com"));
}
}
Since Netty doesn't relay on servlets, is there any overcome available?
What I've done so far is adding:
- dependencies:
implementation("io.quarkus:quarkus-opentelemetry-exporter-otlp")
implementation("io.opentelemetry:opentelemetry-extension-aws")
implementation("io.opentelemetry.contrib:opentelemetry-aws-xray:1.+")
implementation("io.opentelemetry.contrib:opentelemetry-aws-xray-propagator:1.+")
implementation("com.amazonaws:aws-xray-recorder-sdk-aws-sdk-v2:2.8.0")
- properties:
quarkus.opentelemetry.enabled=true
quarkus.opentelemetry.propagators=xray
quarkus.dynamodb.telemetry.enabled=true
quarkus.dynamodb.interceptors=com.amazonaws.xray.interceptors.TracingInterceptor
- config-code for idGenerator:
import io.opentelemetry.contrib.awsxray.AwsXrayIdGenerator
import io.opentelemetry.sdk.trace.IdGenerator
import jakarta.enterprise.context.ApplicationScoped
class AwsOpenTelemetryIdGeneratorProducer {
@ApplicationScoped
fun idGenerator(): IdGenerator {
return AwsXrayIdGenerator.getInstance()
}
}
- aws-otel-collector definition in terraform for aws_ecs_task_definition:
{
name : "aws-otel-collector",
image : "public.ecr.aws/aws-observability/aws-otel-collector:latest",
cpu : 0,
portMappings : [],
essential : true,
command : [
"--config=/etc/ecs/ecs-xray.yaml"
],
environment : [],
mountPoints : [],
volumesFrom : [],
logConfiguration : {
logDriver : "awslogs",
options : {
awslogs-create-group : "true",
awslogs-group : "/ecs/ecs-aws-otel-sidecar-collector",
awslogs-region : "eu-central-1",
awslogs-stream-prefix : "ecs"
}
}
}
- IAM policy:
statement {
actions = [
"xray:PutTraceSegments",
"xray:PutTelemetryRecords"
]
resources = ["*"]
}
With this configuration I am able to trace requests for my ECS. The problem is I cannot trace any dynamoDb hits as following exception occurs "Suppressing AWS X-Ray context missing exception (SegmentNotFoundException): Failed to begin subsegment named 'DynamoDb': segment cannot be found.".
Exception's message is quite meaningful, it cannot determine any parent-segment for DynamoDb sub-segment, what is understandable.
So I decided to add following vertex.core.Handler:
import com.amazonaws.xray.AWSXRay
import com.amazonaws.xray.contexts.SegmentContextExecutors
import com.amazonaws.xray.entities.Segment
import io.quarkus.runtime.annotations.RegisterForReflection
import io.vertx.core.Handler
import io.vertx.ext.web.RoutingContext
@RegisterForReflection
class XRayTracingFilter : Handler<RoutingContext> {
override fun handle(context: RoutingContext) {
val request = context.request()
val segment: Segment = AWSXRay.beginSegment("TestSegment")
segment.putHttp("request", request.method())
SegmentContextExecutors.newSegmentContextExecutor(segment)
context.addEndHandler { segment.close() }
context.next()
}
}
which then I register as follows:
import io.quarkus.vertx.http.runtime.filters.Filters
import jakarta.enterprise.context.ApplicationScoped
import jakarta.enterprise.event.Observes
class XrayConfig {
@ApplicationScoped
fun registerFilters(@Observes filters: Filters) {
filters.register(XRayTracingFilter(), 1)
}
}
After adding XRayTracingFilter exception SegmentNotFoundException doesn't occur anymore but unfortunately I'm still unable to see any tracing coming from AWS::DynamoDB::Table node. All I still see comes from AWS::ECS::Fargate.
I cannot understand at this point why the problem still occurs and which part of my configuration it is related to. Is it a problem with Otel-collector, my Handler or IAM policy?
STACK INFO:
- Quarkus 3.6.6
- Java 17 Corretto
- Gradle 8.5
I would be super glad for any feedback/ideas for my issue.
Thank you in advance!