Azure DevOps Release pipeline for Databricks notebook

Question

I am creating a Azure DevOps Release Pipeline for Databricks notebooks deployment.

I have written below code in powershell activity but its not working.

Requesting some pointer here.

# Variables
$workspaceUrl = "https://<databricks-instance>"
$personalAccessToken = "<databricks-personal-access-token>"
$notebookPath = "/notebooks/my_notebook.ipynb"
$targetEnvironment = "UAT"

# Authenticate with Databricks CLI
databricks configure --token
databricks configure --token "$personalAccessToken" --url "$workspaceUrl"

# Deploy Notebook to UAT Environment
databricks workspace import -l "$notebookPath" -o "/$targetEnvironment$notebookPath"

Answer 1

You don't need to configure CLI explicitly, it's just enough to expose workspace URL as DATABRICKS_HOST environment variable, and personal access token as DATABRICKS_TOKEN environment variable - then CLI will automatically pickup them. See documentation for full list of supported variables.

It's also not recommended to specify PAT directly in the code, but instead it's better to put it & workspace URL into variables and refer from the pipeline, like this (full example):

    - script: |
        echo "Checking out the $(branchName) branch"
        databricks repos update --path $(STAGING_DIRECTORY) --branch "$(branchName)"
      env:
        DATABRICKS_HOST: $(DATABRICKS_HOST)
        DATABRICKS_TOKEN: $(DATABRICKS_TOKEN)
      displayName: 'Update Staging project'