How to tranform logs into a table schema in OTEL Collector

14 Views Asked by At

I have created an OTEL Collector with following configuration:-

processors:
      filter:
        error_mode: ignore
        logs:
          log_record:
            - 'not(IsMatch(body, "customerFailureLogs *"))'
    exporters:
      azuredataexplorer:
        cluster_uri: ""
        application_id: ""
        application_key: ""
        tenant_id: ""
        db_name: ""
        logs_table_name: "OTELLogs"
        logs_table_json_mapping: "OTELLogs"
    service:
      pipelines:
        
        logs:
          receivers: [otlp]
          processors: [filter]
          exporters: [azuredataexplorer]
    

  This collector is able to read the logs associated with customer and forward to ADX database.

The Customer logs show up in OTELLogs table in the body columns. Question:- In the collector I want to read the logs coming in the body and flatten it into individual columns and send it to a new table "Customer Logs" (the table that has the same columns as the logs in body) using Azuredataexplorer exporter

I tried to use transform processor but I don't how to send the transformed data to ADX https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/processor/transformprocessor#parsing-json-logs

0

There are 0 best solutions below