How to reference parameters generated in a loop

795 Views Asked by At

I have Argo WorkflowTemplate with tasks, enclosed below. I'm passing parameters to 'A' task with using loop, so, during runtime I have 2 different 'A' tasks.

Those tasks produce 'run_id', which I have to pass to the next task 'B' as a parameter. And I'm stucked looking for how to reference 'run_id' parameter from 'B'.

entrypoint: pipeline
  templates:
    - name: pipeline
      dag:
        tasks:
          - name: A
            template: A
            arguments:
              parameters:
                - { name: model_type, value: "{{item}}" }
            withItems:
              - "emp"
              - "pos"

          - name: B
            depends: A
            template: B
            arguments:
              parameters:
                - { name: model_type, value: "{{item}}" }
                - { name: run_id, value: '{{tasks.A.outputs.parameters.run_id}}'}
            withItems:
              - "emp"
              - "pos"

    - name: A
      inputs:
        parameters:
          - name: model_type
      outputs:
        parameters:
          - name: run_id
            valueFrom:
              path: "src/run_id"
      script:
        ...

    - name: B
      inputs:
        parameters:
          - name: model_type
          - name: run_id
      script:
        ...

In the current implementation, 'run_id' value in B will be a list like:

["b4045e62ff9f49c99846614a17365b98", "b4045e62ff9f49c99846614a17365b98", "33f4d3d9932543e398f16d9f2c3fa57a", "33f4d3d9932543e398f16d9f2c3fa57a"]

If I set different names to A outputs 'run_id' like 'pos_run_id', 'emp_run_id', and try to catch them in B as:

- name: B
  depends: A
  template: B
  arguments:
    parameters:
      - { name: run_id, value: '{{tasks.A.outputs.parameters}}'}

I get a lists of key-value pairs like:

[{"emp_run_id":"3e706f3491384a46a71eeb8ddf629081"},{"emp_run_id":"3e706f3491384a46a71eeb8ddf629081"},{"pos_run_id":"f15e3781010f46ea97610474041f832e"},{"pos_run_id":"f15e3781010f46ea97610474041f832e"}]

I know how to solve the issue with using artifacts. But I'm curious, is there a way to catch those values, generated in a loop, from another task as parameters?

Thank you in advance.

0

There are 0 best solutions below