I have been looking at running Custom Vision at IoT Edge. I would like to do something similar to as what they have done in this demo https://www.youtube.com/watch?v=_K5fqGLO8us. They use a Raspberry Pi 3B+ and it works, but it is quite slow. It takes the device around 3-5 seconds to notice a change.
I would like to do something similar, but I would need a latency a and frame rate about 2-3 FPS (i.e. I would need something 10x better than the Raspberry 3B+). Are there any suitable single-board computers available for my needs? Anyone that has done something similar and can give a recommendation?
You will need a device with a dedicated GPU to run these kinds of workloads quickly. The Jetson Nano might be an excellent place to start. There are other devices in the Jetson Family that could be a better match.