How to interpret output value from object detection ONNX model

Question

I am making Unity project that does object detection on Hololens 2. I trained a detector model using Azure Custom Vision and exported it as ONNX file and imported in my Unity Project. I converted texture (Camera capture) to a Barracuda Tensor.

        Tensor input = new Tensor(inputTexture, channels: inputChannels);

        worker.Execute(input);

        var output = worker.PeekOutput();
        float[] outputData = output.ToReadOnlyArray();

I don't know what to do with var output and how to interpret the output data.

How do I know whether inputTexture contains tagged object in my Custom Vision Model?

Output for my model is (n:1, h:13,w:13,c:30)

I couldn't figure out what to do next.

Answer 1

It depends on what your model gives the as output.

If you model gives bounding boxes as output,

Let's say there is a model that gives bounding boxes as output and we are detecting an object lets say a TV here

1. Run the model and get the output

2. Output will be a 1-dimensional array

3. get the number of classes you are detecting, here we have only 1 class

4. In a bounding box there will be total 5 values related to a box and additional labels depending on how many classes you have (here it will be 1) as specified below -> xmin,ymin,xmax,ymax and confidence OR xcenter,ycenter,width,height and confidence and remaining will be class label confidences that will help in determining which class it belongs -> for eg. lets say there are 2 labels label0 and label1 and if a box belongs to label0 the its value will be close to 1 and label 1 value for the same box will be close to 0

5. you have the output stored in the array and one class that you are detecting so you need to divide that output array by 5+number of classes -> 5+1 -> 6 that will give you number of boxes

6. get only boxes that have confidence above a certain value in our case lets say 0.25 and ignore the remaining boxes

7. Apply NMS in case you have multi objects of same class and you will get bounding box for the object in image

for more information on barracuda Unity -> https://docs.unity3d.com/Packages/[email protected]/manual/index.html