How to deploy two dependent models in azure ml

Question

I have two YOLO models. In the processing script, the output of model 1 is given as input to model 2 and model 2 gives final output after further more processing. My input to model 1 is either jpg or png image and final output from model 2 is like [(A, 2), (B, 4), (C, 1)....].

I need solution for multimodel deployment, if any two models dependent on each other

Answer 1

I think we can do this by starting to containerize each YOLO model separately using Docker. Then create a Docker image for Model 1, including the necessary YOLO model files and the processing script. Expose an API endpoint in the container to receive image input and return the output. later Create a Docker image for Model 2, and Expose an API endpoint for receiving the output of Model 1 as input and returning the final output. Deploy an orchestrator or server that coordinates the flow between Model 1 and Model

Hope this helps, thanks

Answer 2

To deploy two dependent models using Azure Machine Learning, you can create a single service that hosts both models and handles processing between them.

Below are the steps with sample code snippet for reference:

1. Register your models with Azure ML:

First, upload your models to the workspace's default datastore and register them with Azure ML.

from azureml.core.model import Model

my_model_1 = Model.register(model_path="first_model.pkl",
                            model_name="my_first_model",
                            workspace=ws)

my_model_2 = Model.register(model_path="second_model.pkl",
                            model_name="my_second_model",
                            workspace=ws)  

2. Create a score.py script for handling the inference:

This script will load both models, preprocess the input, and perform inference using the two models sequentially.

%%writefile score.py
import joblib
import json
import numpy as np

from azureml.core.model import Model

def init():
    global model_1, model_2
    # Here "my_first_model" is the name of the model registered under the workspace.
    # This call will return the path to the .pkl file on the local disk.
    model_1_path = Model.get_model_path(model_name='my_first_model')
    model_2_path = Model.get_model_path(model_name='my_second_model')
    
    # Deserialize the model files back into scikit-learn models.
    model_1 = joblib.load(model_1_path)
    model_2 = joblib.load(model_2_path)

# Note you can pass in multiple rows for scoring.
def run(raw_data):
    data = json.loads(raw_data)['data']
    data = np.array(data)
    
    # Call predict() on each model
    result_1 = model_1.predict(data)
    result = model_2.predict(result_1)
     
    return result

Once you configure the inference script according to your requirement, you can create an endpoint.

For more details, you can check this documentation and also you can check the complete code sample for multi-model deployment in this notebook.