I want to use mlflow at work for experiment tracking, it's an opensource library to track machine learning experiments. The problem with mlflow is that it doesn't support snowflake as database. Thus, I have decided to create my custom-mlflow library where I add snowflake support.
However, I am struggling a bit to organize my project.
Right now, the structure looks a bit like that:
project/
|-- pyproject.toml
|-- src/
What I aim to do is create the new custom-mlflow library
project/
|-- pyproject.toml
|-- custom-mlflow/
|-- src/
and add the dependancy in my pyproject.toml
[tool.poetry.dependencies]
python = "3.11.6"
custom-mlflow = {path = "./custom-mlflow"}
Where I am struggling is how organize the custom-mlflow project.Let's say for sake of simplicity that I want to use exactly mlflow and just rename it custom-mlflow
Could you please tell me how that is usually done? Also mlflow can be used in the cli, does that need additional support?
Right now, this is what I think of doing:
inside the custom-mlflow folder, create a __init__.py file and just do the following:
from mlflow import *
and that's it, but I know it is not going to work. I am confused, if I need to add more layers and/or add a setup.py file?
should I add a pyproject.toml of its own for the custom-mlflow project?