DATA_SOURCE_NOT_FOUND Failed to find data source: mlflow-experiment. Please find packages at `https://spark.apache.org/third-party-projects.html

Question

I am trying to load MLflow experiement run data on databricks using Python as suggested in this page. But when I try to do the same, I am getting error "org.apache.spark.SparkClassNotFoundException: [DATA_SOURCE_NOT_FOUND] Failed to find data source: mlflow-experiment. Please find packages at `https://spark.apache.org/third-party-projects.html".

Please help me resolve this issue.

enter image description here

I have checked the experiment id and found it correct.

Answer 1

The requirement is Databricks Runtime 6.0 ML or above.

So, create a cluster having above requirement.

Then run your code with experiment id.

df = spark.read.format("mlflow-experiment").load("1036690367521372")
display(df)