So, here is a quite mysterious problem regarding syslog-ng and a custom parser, using grouping-by.
Here is the parser as defined in the configurations (I left the line numbers to make it easier to identify the error message):
16 parser p_correlate_session_data {
17 grouping-by(
18 key("${source.ip}/${destination.ip}/${source.port}/${destination.port}")
19 scope("host")
20 where(match("ORIG" value("MESSAGE")))
21 trigger(match("DESTROY" value("MESSAGE")))
22 having( "${UNIXTIME}@2" ne "1" )
23 aggregate(
24 value("event.start" "${ISODATE}@2")
25 value("event.end" "${ISODATE}@1")
26 value("event.duration", "$(- ${UNIXTIME}@1 ${UNIXTIME}@2)")
27 value("MESSAGE" "Session completed; client='${source.ip}'; server='${destination.ip}'; destination_port='${destination.port}'; protocol='${network.transport}'; session_lenght='${event.duration}'\n")
28 inherit-mode("context")
29 )
30 inject-mode("pass-through")
31 # destroy events sometimes arrive later than 2 minutes, even when a client app is already closed (ssh, telnet)
32 timeout(600)
33 );
34 };
When I try to use this parser, I'm unable to start syslog-ng, and get the following messages in the logs:
syslog-ng[5897]: Error parsing dbparser, syntax error, unexpected ';', expecting ')' in /etc/syslog-ng/conf.d/network-ulogd2.conf>
syslog-ng[5897]: 28 inherit-mode("context")
syslog-ng[5897]: 29 )
syslog-ng[5897]: 30 inject-mode("pass-through")
syslog-ng[5897]: 31 # destroy events sometimes arrive later than 2 minutes, even when a client app is already closed>
syslog-ng[5897]: 32 timeout(600)
syslog-ng[5897]: 33----> );
syslog-ng[5897]: 33----> ^
syslog-ng[5897]: 34 };
syslog-ng[5897]: 35
syslog-ng[5897]: 36 template t_network {
syslog-ng[5897]: 37 template("$(format_json --omit-empty-values --pair host.name=$HOST --pair host.hostname=$HOST --pair>
syslog-ng[5897]: 38 };
So, it seems to be that the ; character in the end of the line 33 is not supposed to be there. I reviewed the configuration many times to be sure that I had no missing () or {}.
If I comment these lines, everything works great. Syslog-ng starts and log messages from remote servers arrive as expected. I also referred to the official documentation to see if by any chance the syntax was not correct. As far as I can tell, it's not problem.
Any suggestions?