Visualizing multiple all point clouds with .bin format as a video from Lidar - Open3d

Question

I generated several point clouds in .bin files through velodyne and would like to view the various point clouds as a video or animation.

My files 000000.bin to 007480.bin are from a route with a LIDAR turned on until the end of the path and they are all in a directory called ../velodyne/ and I'm running a Deep learning model called OpenPCDet and it's time to run the demo.py with the following command:

python demo.py --cfg_file cfgs/kitti_models/pointrcnn.yaml --ckpt ../OpenPCDet/stev_models/pointrcnn_7870.pth --data_path ../OpenPCDet/data/ kitti/training/velodyne/ enter image description here the result I have is that it opens an image through the Open3D visualizer but I have to keep clicking Q (quit) or ESC to close the window and the code read the next image. My goal is to run demo.py and it will read all the .bin files at once and do the detection with OpenPCDet model treined.

https://github.com/open-mmlab/OpenPCDet/blob/master/tools/demo.py

I've already installed everything you need, now I have to run it as if it were a video already detecting the objects I trained...

Answer 1

You can modify the demo.py script to achive that. Mayavi is capable of saving the images as .png-s in a specific folders. Insert the following under the plot:

#import
import mayavi.mlab as mlab

#The draw scene plot already in demo.py, just to show where to insert
V.draw_scenes(points=data_dict['points'][:, 1:], ref_boxes=pred_dicts[0] 
['pred_boxes'],
ref_scores=pred_dicts[0]['pred_scores'], ref_labels=pred_dicts[0] 
['pred_labels'])
#Setting the view and distance, place this under
mlab.view(180, 85)
mlab.view(distance=40)
#Stop showing, you don't have to use Q or ESC
mlab.show(stop=True)
#Path to save the image
mlab.savefig('/home/OpenPCDet/data/prediction_results/' + 
str(i).zfill(6) + ".png")
mlab.close(all=True)

Since this is inserted in the loop of for idx, data_dict in enumerate(demo_dataset) it is going to iterate over all the bin-s you have. After this, you can connect the exported images into a video with another script. Like this: https://stackoverflow.com/a/44948030/16495145

Note, this works well if you are happy with the clouds to be "converted" to png-s.