create dynamic volumes mid-workflow

13 Views Asked by At

I appreciate this question might be dumb but i am unsure where to look (obviously tried the docs, no help)

kind: Workflow
metadata:
  generateName: extr-test
  namespace: argo
spec:
  entrypoint: extr-val
  serviceAccountName: argo
  volumes:
        - name: local-volume
          hostPath: 
            path: /tmp/data/
  
        
        
        
  templates:
    - name: extraction
      script:
        
        image: docker/whalesay:latest
        imagePullPolicy: Never
        command:
         
          - cowsay
          
        args:
          - '5'
          - '10'
        volumeMounts:
        - mountPath: /mnt/
          name: local-volume
      outputs:
        parameters:
        - name: folders
          valueFrom:
            path: /mnt/folders_created.txt

    - name: validation
      inputs:
        parameters:
          - name: folder
      script:
        
        image: docker/whalesay:latest
        imagePullPolicy: Never
        command:
          
          - cowsay
        args: ['main.py', "{{inputs.parameters.folder}}", 'a,b,c']
          
          
        volumeMounts:
        - mountPath: /mnt/
          name: local-volume

    - name: hello
      inputs:
        parameters:
          - name: folder
      script:
        image: alpha-notation:latest
        imagePullPolicy: Never
        command:
          - /bin/bash 
            -  -c
            - python3 /tmp/entrypoint.py --config-file /tmp/config/a.xml 
        args: ["{{inputs.parameters.folder}}"]
          
        
   volumeMounts:
            - mountPath: /tmp/input/
              name: "{{inputs.parameters.folder}}"
            - mountPath: /tmp/output
              name: output-folder

            - mountPath: /tmp/license
              name: license-folder

            - mountPath: /aa/config
              name: config-folder

            - mountPath: /tmp/conv/
              name: converter-folder
    
    
    
    - name: val-hello
      inputs:
        parameters:
          - name: folder
      steps:
        - - name: hello
            template: hello
            arguments:
              parameters:
                - name: folder
                  value: "{{inputs.parameters.folder}}"
        
        - - name: validation
            template: validation
            
            arguments:
              parameters:
                - name: folder
                  value: "{{inputs.parameters.folder}}"


    - name: extr-val
      steps:
        - - name: extraction
            template: extraction
        
        - - name: val-hello
            template: val-hello
            withParam: "{{steps.extraction.outputs.parameters.folders}}"
            continueOn:                 
                failed: True           
            arguments:
              parameters:
                - name: folder
                  value: "{{item}}"

The issue i'm having is creating volumes for the folders that are created in the extraction step and mounting them in the next steps. Is this possible? Is there a concept i'm missing? thank you

i tried passing the folders value in volumeMounts, obv volume not defined. (i somehow need to define them before the actual mounting)

0

There are 0 best solutions below