How to return only one pipeline using template-parameters-definition.json synapse

Question

I have over 100 pipelines in azure synapse and I want to use "template-parameters-definition.json" to return parameters for just 3 of the 100 pipelines. How can I specify what pipelines I want to return parameters from in the template-parameters-definition.json. I am currently using the sample file from here: https://learn.microsoft.com/en-us/azure/data-factory/continuous-integration-delivery-resource-manager-custom-parameters Thanks

I tried to add the name of the pipeline here:

    "Microsoft.Synapse/workspaces/pipelines": {
      name: "test1234",
        "properties": {
            "activities": [{
                "typeProperties": {
                    "waitTimeInSeconds": "-::int",
                    "headers": "=::object",
                    "activities": [
                        {
                            "typeProperties": {
                                "url": "-:-webUrl:string"
                            }
                        }
                    ]
                }
            }]
        }
    },

but this still returned all parameters for all pipelines

Answer 1

Prerequisites and Setup for Filtering Azure Synapse Pipelines with PowerShell cannot be provisioned.

To manage your CI/CD process with Azure Synapse or Azure Data Factory, which share similar concepts and configurations, you need to understand how the template-parameters-definition.json file works. This file is essential for parameterizing your pipelines in these services.

The template-parameters-definition.json file controls the behavior of parameters during the ARM template generation process. It does not filter or specify the pipelines to be included or excluded. Instead, it determines how to handle parameters for resources that are deployed or managed by ARM templates. In your case, you want to parameterize only 3 out of 100 pipelines. However, the snippet you've given tries to set the parameters for a specific pipeline by its name, which is not how the template-parameters-definition.json file works.

You want to select only a few pipelines (3 out of 100) using CICD. The template parameter definition file cannot help you with this. This JSON file only shows the parameters that are hidden by default.

To use cherry-pick for the pipelines, you should create a separate branch for the artifacts and deploy them from that branch using the validate and deploy operation.

To confirm please review the resolution provided by @BhargavaGunnam-MSFT

Reference:

https://learn.microsoft.com/en-us/azure/synapse-analytics/cicd/source-control#unsupported-features

https://techcommunity.microsoft.com/t5/azure-synapse-analytics-blog/automating-the-publishing-of-workspace-artifacts-in-synapse-cicd/ba-p/3603042

https://learn.microsoft.com/en-us/answers/questions/1527532/how-to-return-only-one-pipeline-using-template-par