I am currently working on transferring data from a REST API in Azure Data Factory. The data is presented in the form of an array of nested JSONs, and my goal is to copy it into a CSV file. To achieve this, I have implemented a loop for the array and dynamic content for the mapping of the copy activity.
I am utilizing a Pipelines_Mappings variable to generate a flexible mapping. Here is part of the Pipelines_Mappings variable
"type": "TabularTranslator",
"mappings": [{
"source": {
"path": "$['results'][0]['label']"
},
"sink": {
"type": "String",
"ordinal": 1
}
}
Within the loop, a set variable activity is employed to modify the Pipelines_Mappings variable to replace the index [0] with the loop index item(). Then, the updated variable is assigned to the Modified_Mappings variable. Eventually, I utilize the Modified_Mappings variable in the copy activity and set the mapping tab of the copy activity as @json(Modified_Mappings).
But the problem is when I use the set variable to manipulate the JSON string, backslashes are added before each double quote. This causes issues with the JSON and the copy doesn't work correctly. How can I remove these backslashes?
when I used the Pipelines_Mappings without any changes in the copy activity, it works.