Multiline parsing in Fluent-Bit for pentaho jobs on kubernetes

Question

I am attempting to get fluent-bit multiline logs working for my jobs running on kubernetes. These are pentaho jobs. Is there a way to use a custom multiline parser to get the logs in elastic? I am attempting to use the date format along with other fields as the start of the multiline parser, and giving condition to capture the next line that should be included in the first log, rather than broken up into different. Here are the config files with the input, filter, and parsers:

[MULTILINE_PARSER]
    name multiline_logs
    type regex
    flush_timeout 1000
    rule       "start_state"    "^(?<timestamp>[^ ]* [^ ]*) (?<log_level>[^ ]*) (?<step_name>[^>]*\>) (?<job_name>[^]]*\])"  "cont"
    rule       "cont"    "(?<message>[\s\S]+)(?=\d{4}-\d{2}-\d{2} \d{2}:\d{2}:\d{2}.\d{3})"  "cont"

[SERVICE]
    Flush         5
    Log_Level     info
    Daemon          off
    Parsers_File  parsers.conf
[INPUT]
    Name              tail
    Tag               multiline_logs
    Path              log file
    read_from_head    true
    multiline.parser  multiline_logs
[FILTER]
    Name record_modifier
    Match *
    Record __clp_index container
    Record __clp_type log
    Record source ${HOSTNAME}

log examples:

2024-02-28 12:30:52.956 INFO  <Thread-809> [abc/job1]  Starting entry [Set variables]
2024-02-28 12:30:52.956 INFO  <Thread-809> [abc/job1]  Starting entry [Set Database Connection]
2024-02-28 12:30:52.963 ERROR <Thread-809> [abc/job1]  Unable to run job. The Set Database Connection has an error.
2024-02-28 12:30:52.963 ERROR <Thread-809> [abc/job1]  org.pentaho.di.core.exception.KettleException:
Unable to find repository directory [abc/job1/../xyz.kjb]

        at org.pentaho.di.base.MetaFileLoaderImpl.getMetaFromRepository(MetaFileLoaderImpl.java:312)
        at org.pentaho.di.base.MetaFileLoaderImpl.getMetaForEntry(MetaFileLoaderImpl.java:165)
        at org.pentaho.di.job.entries.trans.JobEntryTrans.getTransMeta(JobEntryTrans.java:1290)
        at org.pentaho.di.job.entries.trans.JobEntryTrans.execute(JobEntryTrans.java:715)
        at org.pentaho.di.job.Job.execute(Job.java:703)
        at org.pentaho.di.job.Job.execute(Job.java:844)
        at org.pentaho.di.job.Job.execute(Job.java:844)
        at org.pentaho.di.job.Job.execute(Job.java:513)
        at org.pentaho.di.job.Job.run(Job.java:393)

2024-02-28 12:30:52.964 INFO  <Thread-809> [abc/job1]  Starting entry [Get Database details Failed]

But still getting logs into broken lines and no fields in logs. Can you please check my configs and let me know what is wrong here?