I am working on 3d reconstruction of objects in real time using Stereolabs ZED X camera. The point cloud generated by the camera has a few issues and so I was not able to use it for reconstruction. I have a few ideas but I do not know if it will work. Let me summarise the ideas:
- The segmentation created by the inbuilt features in the camera was faulty. So I am thinking of using a custom segmentation model and use it to segment the objects in 2d. Then I would use the masked region and overlay it on the depth map to cut out all noises and extract only the object that I want.
- I directly create a segmentation model for point cloud then use the segmented point cloud regions and perform surface approximation using clustering algorithms like DBSCAN.
I have no idea how they will work and how efficient it will be. Can someone give me in-depth list of all the things I have to do and maybe even your own idea that can work efficiently.
I tried to use the mask generated by the ZED X camera in the Object Detection Module and it was faulty. The mask was inconsistent for each frame leading to poor point cloud generation. Also I tried to use YOLOv8 for semantic segmentation in 2d and then get the corresponding depth values for the mask cordinates (x,y). But the ZED Orin Box is unable to use the CUDA installed inside it via the Jetson SDK manager.