Automate unmounting and mounting step because of Service principal expiration in Databricks

Question

I need to automate task which is related to azure databricks.we have configured a job in azure databricks and suddenly my service prinicipal secrets get expired and my notebook failed. on the next day I created new secrets and unmount --> mount it again and job worked. I know the way to create new secret or get the alert before expiring using logic app and manually will change it.

But how to manage unmount --> mount step automatically? As SP can be used in multiple project. This is how I am mounting and mount_point used across notebook.

Answer 1

Here is the sample code for Unmounting and mounting notebook. Check if the path is mounted or not, If it is not mounted yet mount the path with the given condition.

def mount_blob_storage_from_sas(dbutils, storage_account_name, container_name, mount_path, sas_token, unmount_if_exists = True):
    if([item.mountPoint for item in dbutils.fs.mounts()].count(mount_path) > 0):
        if unmount_if_exists:
        print('Mount point already taken - unmounting: '+mount_path)
        dbutils.fs.unmount(mount_path)
    else:
        print('Mount point already taken - ignoring: '+mount_path)
        return
        print('Mounting external storage in: '+mount_path)
          dbutils.fs.mount(
            source = "wasbs://{0}@{1}.blob.core.windows.net".format(container_name, storage_account_name),
            mount_point = mount_path,
            extra_configs = {"fs.azure.sas.{0}.{1}.blob.core.windows.net".format(container_name, storage_account_name): sas_token })

Create a job to run the notebook with given specific time period.

Provision the notebook path and cluster to create the JOB.

Schedule a time to trigger the notebook in Add Schedule .

Select Scheduled path to provide time to trigger.

Job will trigger at given time to run the notebook.