How to obfuscate/redact OpenTelemetry logs using transform/attribute processor?

26 Views Asked by At

I have been struggling to redact/obfuscate logs using tranform processor.

Source: https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/v0.87.0/pkg/ottl/ottlfuncs/README.md

This is Splunk Otel distribution implementation on EKS cluster.

The logs looks like this:

2024-03-11T21:04:41.411025006Z stdout F {"time": "2024-03-11T21:04:41+00:00", "upstream_namespace":"system-monitoring", "remote_user": "sample-user"}

Processor Configuration( tested individually, just clubbed here for ease):

processors:
  attributes/upsert:
   actions:
   - key: upstream_namespace
     action: upsert
     value: "REDACTED_NS"
  transform:  
   log_statements:
   - context: log
     statements:
      - replace_all_patterns(attributes,"value","upstream_namespace", "REDACTED_NS")
      - replace_all_patterns(attributes,"key","upstream_namespace", "REDACTED_NS")
      - replace_match(attributes["upstream_namespace"], "*" , "REDACTED_NS")
      - replace_match(attributes["upstream_namespace"], "system-monitoring" , "REDACTED_NS")
      - delete_key(attributes,"upstream_namespace")
      - delete_key(resource.attributes,"upstream_namespace")
      - replace_all_patterns(attributes["upstream_namespace"],"value","upstream_namespace", "REDACTED_NS")
      - replace_all_patterns(attributes["upstream_namespace"],"value","system-monitoring", "REDACTED_NS")`

The attribute/upsert however adds REDACTED_NS value along with the original.

upstream_namespace: REDACTED_NS
                    system-monitoring

Any suggestions to achieve this logs transformation?

0

There are 0 best solutions below