Leap Motion Gesture Detection Accuracy with LeapJS

Question

I am using a leap motion sensor with my node app to detect gestures. With my current setup, the gesture detection is working but not as accurately as I need it to be. When I perform a swipe gesture the leap motion does not always send a swipe gesture object and when it does detect a swipe it often will send multiple gestures. For example, when swiping left, it will return a left swipe as well as an up swipe. I am wondering if this is an issue with my code or my leap motion sensor.

Here is my leap controls code:

var controller = Leap.loop({ inNode: true, enableGestures: true }, function(frame) {

  // finger location tracking
  if(frame.hands.length > 0) {
    var iBox = frame.interactionBox;
    var pointable = frame.pointables[0];

    if (pointable) {
      var leapPoint = pointable.stabilizedTipPosition;
      var normalizedPoint = iBox.normalizePoint(leapPoint, true);
      let mouseControlY = (1 - normalizedPoint[1]) * appHeight; // with leap facing up

      // invert directions for mirror
      let invertMouseControlX = (1 - normalizedPoint[0]) * appWidth;

      updateMouseCords(invertMouseControlX, mouseControlY)
    }
  }

  // finger gesture tracking
  if(frame.valid && frame.gestures.length > 0){
    frame.gestures.forEach(function(gesture){
        switch (gesture.type){
          case "screenTap":
              console.log("Click");
              break;
          case "swipe":
              handleSwipe(gesture);
              break;
        }
    });
  }
});

function handleSwipe (swipe) {
  let io = getIO();
  
  if(swipe.state === 'stop'){
      if (swipe.direction[0] > 0){
        // user swiped right
        swipeDirection = 'right'
      } else {
        // user swiped left
        swipeDirection = 'left'
      }
      
      if(swipe.direction[1] > 0) { 
        // user swiped up
        swipeDirection = 'up'
      }  

      // send swipe direction to frontend
      io.emit('swipeData', swipeDirection);
  }
}

I only need to detect a left, right, or up swipes and possibly the screen tap gesture. Is there anything I can do to optimize my code or is the issue likely my sensor?

Answer 1

I am not an expert myself, but I have prevented multiple detections by using the setTimeout() function, which is set to a few milliseconds where no gesture detection can occur.

The JavaScript documentation also mentions the attributes Gesture.duration and Gesture.type. Perhaps you can achieve better results by using them.

It is also worth noting, that Gestures are unfortunately deprecated in version 3.0. However, I am not sure if this is due to the problems you described or if they will be fundamentally revised in later versions. I would also be interested in an update on this topic.