Can't get data labels from ONNX Model exported from Microsoft Custom Vision

Question

I'm using Microsoft Custom Vision to build an image classifier. Then, I'm able to download the ONNX file and get probabilities from the Model on a WPF app.

My issue is that I'm not able to get labels for those probabilities.

Under netron.app the neural network looks like this: ONNX model inputs and outputs

but since I'm not having an output that is referencing a string I'm assuming that I can't get the labels from this ONNX. Am I right?

Another hypothesis is to assume that the index on the label file match the output index of the ONNX. But how could I validate that?

Answer 1

it seems like you're using Microsoft Custom Vision to build an image classifier and then exporting the model to ONNX format for use in a WPF app. To map the output probabilities to their corresponding labels, you need to know how the labels are associated with the output indices in the ONNX model.

To validate this assumption, you can perform a simple test. Choose an image that you used during training, run inference on it using your ONNX model, and then check the index with the highest probability in the output tensor. This index should correspond to a line number in your label file, which will give you the associated label.

Here's a simple code snippet in Python

import onnxruntime
import numpy as np

# Load the ONNX model
session = onnxruntime.InferenceSession('your_model.onnx')

# Load your label file into a list
with open('your_label_file.txt', 'r') as f:
    labels = [line.strip() for line in f.readlines()]

# Load and preprocess your image
# ...

# Run inference
input_name = session.get_inputs()[0].name
output_name = session.get_outputs()[0].name
input_data = np.array(preprocessed_image).astype(np.float32)
output = session.run([output_name], {input_name: input_data})

# Get the index with the highest probability
predicted_class_index = np.argmax(output)

# Get the label for the predicted class
predicted_label = labels[predicted_class_index]

print(f"Predicted Label: {predicted_label}")