I am trying to keep a database consistent with the current status of the PBS jobs on a cluster. Currently this has to be done by a heavyweight regular qstat polling & diffing between qstat and the database tables, which is a fairly terrible way to do this.
Is there a way to use the observer pattern & get a notification of job changes, or to get a log file of status changes that a process can tail for updates & stream to the database or to Kafka?