Occlusion handling in object tracking

Question

I am implementing motion based object tracking program which is using background substraction, Kalman Filter and Hungarian algorithm. Everything is working fine except the occlusions. When two object are close enough to each other the background substraction recognizes it as one of these two objects. After they split the program recognizes these two objects correctly. I am looking for solution/algorithm which will detect occlusion like shown in point c) in the example below.

I will appreciate any references or code examples reffering to occlusion detecion problem when using background substraction.

Answer 1

Object detection using a machine learning algorithm should reliably distinguish between these objects, even with significant occlusion. You haven't shared anything about your environment so I don't know what kind of constraints you have, but using an ML approach, here is how I would tackle your problem.

import cv2
from sort import *

tracker = Sort() # Create instance of tracker (see link below for repo)

cap = cv2.VideoCapture(0)
while(True):
    ret, frame = cap.read()

    # Not sure what your environment is, 
    #but get your objects bounding boxes like this
    detected_objects = detector.detect(frame) #pseudo code
    
    # Get tracking IDs for objects and bounding boxes
    detected_objects_with_ids = tracker.update(detected_objects)
...

The above example uses this Kalman Filter and Hungarian algorithm, which can track multiple objects in real-time.

Again, not sure about your environment, but you could find pre-built object detection algorithms on the Tensorflow site.