How can I use opencv python cv2.xfeatures2d.SIFT as imput for cv2.calcOpticalFlowPyrLK?

Question

I made a stabilizer for videos and now I'm trying to compare my algorithm with others, so I wanted to implement the sift and pass the keypoints to the optical flow algorithm.

sift = cv2.xfeatures2d.SIFT_create()

then

frame.features = sift.detect(frame_img_gray,None)
new_features, _, _ = cv2.calcOpticalFlowPyrLK(prev_frame_img, frame_img, prev_frame.features, None, **lk_params)
        new_features_for_validation, _, _ = cv2.calcOpticalFlowPyrLK(frame_img, prev_frame_img, new_features, None,**lk_params)

So I get this message from the compiler:

 new_features, _, _ = cv2.calcOpticalFlowPyrLK(prev_frame_img, frame_img, prev_frame.features, None, **lk_params)
TypeError: prevPts is not a numpy array, neither a scalar

Answer 1

You can manually reshape SIFT output to be suitable for cv2.calcOpticalFlowPyrLK:

LK_pts=np.empty((0,1,2), np.float32)
for pts  in prev_frame.features:
    x, y = pts.pt
    LK_pts=np.append(LK_pts,np.array([[[x,y]]],dtype=np.float32),axis=0)