Read the file from datalake in databricks -No such file directory error

Question

I have a problem with reading the file from ADLS gen 2.

I have dont the mounting properly as after executing dbutils.fs.ls('/mnt/bronze') I can see the file path.

the way how I did the mounting:

# dbutils.fs.mount(
#   source = "abfss://"+container_read+"@"+storage_account_name_read+".dfs.core.windows.net/",
#   mount_point = "/mnt/dataverse",
#   extra_configs = configs)

the way how I read the file:

with open('/mnt/dataverse/model.json', 'r') as f:
    data = f.read()
    manifest = json.loads(data)

and it throws the error No such file or directory

Sketchy part is that I can read the file usign different cluster ( runtime 11.3.x-scala2.12) but after switching to 12.2 cluster I cant read it.

Any idea how can I fix this?

Answer 1

I created the mount point data and got same error with your code.

Here, with open() not identifying the mount point might be reason for the reason. We need to give the dbfs path to this. To resolve the error, give the path like this.

/dbfs/mnt/<mount-point>/<filename>

To understand better about this path, Go to dbfs browser and click on file name in the mount point. You can see the File API format file path.

You can see that I am able to read the file, with the above file path.

If the issue still persists, it's better to raise a support ticket on this.