I discovered recently mlflow Databricks so I'm very very new to this Can someone explain for me clearly the steps to track my runs into the databricks API. Here is the steps I followed : 1/ install Databricks CLI 2/ I sewed up authentication between Databricks CLI and my Databricks workspaces according to instructions here text
I checked the file cat ~/.databrickscfg and everything is fine
3/ I'm using Pycharm and I'm writing a python script including mlflow and I want to track the runs on my Databricks workspace Here is a part of my code :
mlflow.autolog(
log_input_examples=True,
log_model_signatures=True,
log_models=True,
disable=False,
exclusive=False,
disable_for_unsupported_versions=True,
silent=False
)
#mlflow.login()
mlflow.set_tracking_uri("databricks")
mlflow.set_experiment("mlflowAUS")
with mlflow.start_run() as run:
bestModel.fit(X_train, y_train)
y_pred = bestModel.predict(X_test)
runId = run.info.run_id
mlflow.set_tag('mlflow.runName', datetime.datetime.now().strftime("%Y%m%d_%H%M%S"))
mlflow.log_param("model_name", str(bestModel)[: str(bestModel).index("(")])
and if I add to my code :
mlflow.login()
I got another time the same error but before the error I got : (I hided my the link with xxxx )
2024/03/30 01:40:44 INFO mlflow.utils.credentials: Successfully connected to MLflow hosted tracking server! Host: https://dbc-xxxxxx-xxxx.cloud.databricks.com.
What I don't understand if it says that I'm successfully connected after adding mlflow.login() why I still got the same error and I can't track my runs ? Please help me and I thank you in advance for the support
this code return error :
Traceback (most recent call last):
File "/Users/kevin/PycharmProjects/Projects2024/nov23_continu_mlops_meteo/src/models/bestModel.py", line 184, in <module>
mlflow.set_experiment("mlflowAUS")
File "/Users/kevin/opt/anaconda3/envs/env_2023/lib/python3.11/site-packages/mlflow/tracking/fluent.py", line 142, in set_experiment
experiment = client.get_experiment_by_name(experiment_name)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/kevin/opt/anaconda3/envs/env_2023/lib/python3.11/site-packages/mlflow/tracking/client.py", line 539, in get_experiment_by_name
return self._tracking_client.get_experiment_by_name(name)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/kevin/opt/anaconda3/envs/env_2023/lib/python3.11/site-packages/mlflow/tracking/_tracking_service/client.py", line 236, in get_experiment_by_name
return self.store.get_experiment_by_name(name)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/kevin/opt/anaconda3/envs/env_2023/lib/python3.11/site-packages/mlflow/store/tracking/rest_store.py", line 323, in get_experiment_by_name
response_proto = self._call_endpoint(GetExperimentByName, req_body)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/kevin/opt/anaconda3/envs/env_2023/lib/python3.11/site-packages/mlflow/store/tracking/rest_store.py", line 60, in _call_endpoint
return call_endpoint(self.get_host_creds(), endpoint, method, json_body, response_proto)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/kevin/opt/anaconda3/envs/env_2023/lib/python3.11/site-packages/mlflow/utils/rest_utils.py", line 220, in call_endpoint
response = verify_rest_response(response, endpoint)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/kevin/opt/anaconda3/envs/env_2023/lib/python3.11/site-packages/mlflow/utils/rest_utils.py", line 170, in verify_rest_response
raise MlflowException(f"{base_msg}. Response body: '{response.text}'")
mlflow.exceptions.MlflowException: API request to endpoint was successful but the response body was not in a valid JSON format. Response body: '<!doctype html>