How set name file for GCS Move using Arguments runtime with wildcards in CDAP/Data fusion pipeline

182 Views Asked by At

I need to move some files between buckets after they are processed in the pipeline, however I have come across files that contain characters like "+" or "-" within their name (example: data+1+2132121.json)

following the documentation in this How to set runtime arguments in a CDAP/DATA FUSION pipeline? response

I have set the source path to "gs://mybucket/data/${input}" so that the value of my argument can be "data+1+" and only move those files that begin with "data+1+" are the ones that will be moved from location.however when using wildcards() I cannot move the files.some idea that it may be happening?

0

There are 0 best solutions below