How can I dynamically generate and output multiple files in ECL?

Question

Let's say we want to dynamically create multiple files containing some data. How can that be achieved?

I tried the following for example, to output some data in multiple files on the fly:

file_names := DATASET([{'file_1'}, {'file_2'}], {STRING name});
ds := DATASET([{'a'}], {STRING val});
APPLY(file_names, OUTPUT(ds,, name, THOR));

But I got the following error:

Answer 1

This exact issue was reported in this JIRA ticket: JIRA HPCC-14754 which so far has not been resolved.

HTH,

Richard

Answer 2

As Richard stated above, this is a known issue and has been reported. However, you can generate multiple outputs using the code you posted but you need to change your ECL cluster target to "hthor" instead of "thor". Of course, for large files this could be problematic, but I think your use case for small records and multiple files would work for you.

Bob