How to parse json array input to basic json using Logstash and can be send to Loki as input for visualization

Question

Input-

[{"id": 23232, "user_name__v": "[email protected]", "data_type__v": "full__v", "token_id__v": 43434, "last_login__v": "2023-10-18T09:19:41.000Z", "active__v": true, "profile_type__v": "us_document_bulk_action_users_with_creat__c"}, {"id": 17685499, "user_name__v": "[email protected]", "data_type__v": "full__v", "token_id__v": 566, "last_login__v": "2023-10-18T09:18:56.000Z", "active__v": true, "profile_type__v": "busineess"}, {"id": 343434, "user_name__v": "[email protected]", "data_type__v": "full__v", "token_id__v": 37431, "last_login__v": "2023-11-01T07:39:00.000Z", "active__v": true, "profile_type__v": "it_actions__c"}]

Configuration File Filter -

filter {
  json {
    source => "message"
    target => "message"
  }
  mutate {
    add_field => {
      "app" => "TestingApp"
      "job" => "cronData"
    }
  }  
  mutate {
    remove_field => [ "event" ]
  }  
}

I am unbale to convert this json array to json object I tried to split it but didn't worked as expected.

Desired Output:Json format which can be used in Loki to create table data/timeseries using basic transformation.

I tried split but didn't worked as expected.

Answer 1

You just need to apply a json filter on the input

filter {
    json {
        source => "array_input"
        target => "parsed_array"
        remove_field => "array_input"
    }
    ruby {
        code => "event.get('parsed_array').each { |kv| event.set(kv['Field'], kv['Value']) }"
    }
}