I'm parsing our production logs and need to "join" 2 log lines, one containing an IP the other a user ID, and both have a request ID to join on. This is to compare and count and detect users signin from multiple IP and IPs used by multiple users.
It looks like the join operator is the way to do this, and I got it working for small time ranges, unfortunately it has a 50000 input message limit which is really little and makes it quite useless for production traffic. I tried using timewindow to limit how far the system needs to search in logs but that doesn't seem to change anything.
I don't think I'm doing something particularly rare, matching IP from an access log (which often has millions of lines/day) with another log line so I'm surprised by the very low limit on this. Is there some other way to achieve the same result in Sumo Logic?