%%4416 %%4417 %%4" /> %%4416 %%4417 %%4" /> %%4416 %%4417 %%4"/>

Logstash Split Issue

36 Views Asked by At

Below statement is simple incoming message and AccessList should be parsed by Logstash(v8.12.1):

<EventData>
    <Data Name="AccessList">%%4416 %%4417 %%4418 %%4419 %%4420 %%4423 %%4424 %%1538 </Data>
</EventData>

I scrape this text with following configuration but split in mutate filter plugin is not working correctly:

filter {
  xml {
    xpath => ["//Data[@Name='AccessList']/text()","access_text"]
  }
  mutate {
    gsub => ["access_text" ,"\s+", ""]
    gsub => ["access_text" ,"%%", ","]
    gsub => ["access_text","^,",""] #Delete first comma
  }
  mutate {
    convert => {
      "access_text" => "string"
    }
  }
  mutate {
    split => {"access_text" =>  ","}
  }
}

This is a output splitless text on Kibana:

4416,4417,4418,4419,4420,4423,4424,1538

My expected output is a array like this on Kibana:

[4416,4417,4418,4419,4420,4423,4424,1538]
2

There are 2 best solutions below

0
siwm On BEST ANSWER

Solution: I added force_array attribute in xml filter plugin like below:

filter {
  xml {
    source => "message"
    store_xml => false
    force_array => false
    # other statements ...
  }
  mutate {
    gsub => ["access_text" ,"\s+", ""]
    gsub => ["access_text" ,"%%", ","]
    gsub => ["access_text","^,",""]
    split => {"access_text" => ","}
  }
}
1
Ankit On

When executing the transform mutation, the conversion of the entry_text category to a string may be causing the subsequent separate mutation to deviate from anticipated outcomes.

Try this.

filter {
  xml {
    source => "message"
    target => "xml_content"
    xpath => ["/EventData/Data[@Name='AccessList']/text()", "access_text"]
  }
  mutate {
    gsub => ["access_text" ,"\s+", ""]
    gsub => ["access_text" ,"%%", ","]
    gsub => ["access_text","^,",""] # Delete first comma
  }
  mutate {
    split => {"access_text" => ","}
  }
}