understanding sacct's maxRSS -- Why are there two rows for a job?

Question

I have the output from sacct --format="jobID,CPUTime,MaxRSS" -j 66930332_195. I know maxRSS reports a value roughly equivalent to max memory usage. However, what do the two different rows in maxRSS refer to?

  JobID    CPUTime     MaxRSS
------------ ---------- ----------
66930332_195   00:05:15
66930332_19+   00:05:15   4688356K
66930332_19+   00:05:15      2376K

Thanks in advance! I haven't been able to find this documented anywhere

Answer 1

If you use %20 like this to show the JobID in its entirety,

sacct --format="jobID%20,CPUTime,MaxRSS"

you would probably see something like this:

  JobID          CPUTime     MaxRSS
------------     ---------- ----------
66930332_195     00:05:15
66930332_195.0   00:05:15   4688356K
66930332_195.1   00:05:15      2376K

The first row corresponds to the job itself and the other ones correspond to the jobs steps. That should correspond to the number of srun calls you do in your submission script.