Is it possible for Azure Custom Vision to analyze a video stream and count wheels on vehicle

Question

I have a video feed from a remote camera, we would like to have a solution that looks at the video feed and counts the wheels on any vehicles that pass through its views. From what I have been reading this would be quite easy using images, but I can find nothing about from a video feed. Any help would be appreciated. Environment in C# WinForms using Azure as its backend for processing and storage etc

Answer 1

I'm having trouble with the same problem. I have not solved it yet in code. But I can try to help explain it conceptually.

If you watch this video, they are counting the object when the centroid passes a given line. https://www.youtube.com/watch?v=WgbS_csjxhk&ab_channel=Nodeflux

The way I think you should approach your problem is similar.

1. Train a model to recognize vehicle wheels.
2. Implement that model. When you use the model, it will return the coordinates of the image bounding box. Here's an abridged output from Azure Custom Vision "fork: 98.2% [ 0.111609578, 0.184719115, 0.6607002, 0.6637112 ]"+
3. Given the polygon output in the previous step, calculate its centroid.
4. Determine the direction the wheels will pass through the video feed, up-down or left-right. Create a rectangle that covers this area.
5. Create a function that determines when the centroid crosses that rectangle in the desired direction. IE when Centroid goes from < Rectangle to > Rectangle in X or Y.

Hope that helps!

Answer 2

You could design a client-side algorithm to grab keyframes and send them to computer vision, or you could train a custom model in Azure Machine Learning.

But there's an easier solution if you don't want any of those ways: You could use video indexer to get your vehicles' keyframes and export those images to computer vision.

Here is something that could help you if you're interested:

-The idea: https://azure.microsoft.com/en-us/blog/combine-the-power-of-video-indexer-and-computer-vision/

-Docs:

https://learn.microsoft.com/en-us/azure/azure-video-indexer/

https://learn.microsoft.com/en-us/azure/azure-video-indexer/scenes-shots-keyframes

Have a nice day, greetings!