Unexpected results from a WHEN function

Question

I have a BWR that executes a FUNCTIONMACRO. Within the FUNCTIONMACRO I execute a FUNCTION that reads a dataset, sorts and deduplicates that dataset and returns the results. Before it is returned, I want to write out an audit record.

EXPORT FN_Read_Batch_File( string pFileName ) := FUNCTION
  rLayout := RECORD
     STRING field_01;
     STRING field_02;
  END;
  
  rAudRec := RECORD
     INTEGER06 raw_ds_cnt;
     INTEGER06 final_ds_cnt;
  END;
  
  // process the file
  ds                     := dataset(data_services.foreign_prod + pFileName, rLayout, THOR, OPT);
  INTEGER06 raw_ds_cnt   := COUNT(ds);
  
  sdd_ds                 := DEDUP( SORT( ds, field_01), field_01);
  INTEGER06 final_ds_cnt := COUNT(sdd_ds);
  
  // write the audit record
  auditRec               := output(  dataset([  {raw_ds_cnt; final_ds_cnt}
                                             ], rAudRec)
                        )
                   , ,'~temp::audit::record', compressed, overwrite, expire(1)
                  );
  RETURN  **WHEN(sdd_ds, auditRec)**;
END;

When I execute the function from within a BWR (during development), it worked just fine: I have the sorted/deduped file returned and the audit record was written. When I attempt to execute the FUNCTION from the FUNCTIONMACRO, I only get the sorted/deduped file returned - the audit record is not written (and the FUNCTIONMACRO that is expecting that audit record fails). When I look at the executed code, it seems like the trigger (the presence of the sdd_ds file) is there because it gets returned, but it doesn't seem to be triggering the auditRec action.

Am I missing something with "WHEN" or should I be doing this differently?

Answer 1

Based on your problem description, I tested this code, which works correctly. So perhaps you should try this and expand on your question if it doesn't work for you.

    MyFunc(STRING s) := FUNCTION
      mod := RANDOM() % 7 + 2: INDEPENDENT;
      rec := RECORD
        STRING f1;
        STRING f2;
      END;
      ds := DATASET(50,
                    TRANSFORM(rec,
                              SELF.f1 := 'F1-' + RANDOM() % mod,
                              SELF.f2 := s + '-' + COUNTER));   

  // write the audit record
  rAudRec := RECORD
     INTEGER06 raw_ds_cnt;
     INTEGER06 final_ds_cnt;
  END;
  // process the file
  INTEGER06 raw_ds_cnt   := COUNT(ds);
  
  sdd_ds                 := DEDUP( SORT( ds, f1), f1);
  INTEGER06 final_ds_cnt := COUNT(sdd_ds);
    
  auditRec := OUTPUT(DATASET([{raw_ds_cnt; final_ds_cnt}], rAudRec),,
                     '~temp::audit::record'+s, COMPRESSED, EXPIRE(1));  
  RETURN WHEN(sdd_ds, auditRec);
END;

MyFnMac(s) := FUNCTIONMACRO
  res := MyFunc(s);
    RETURN res;
ENDMACRO;

MyFnMac(WORKUNIT);

I removed the OVERWRITE option on the OUTPUT since I'm passing in a unique identifier (the WORKUNIT id) and there's no need to worry about the number of times it executes because the EXPIRE(1) will cause the audit files to be deleted after a day.