Enabling tap gestures with the TouchZone API is an effective way to allow quick interactions within your app. Tapping is fast and easy, and it’s a very familiar action to anyone who’s used a touchscreen. Here’s a brief guide on how to recognize a tap gesture, in a way that avoids needless complexity and synchronization problems.

The TouchZone API

The “touch zone” is an unique feature provided by the Leap API that creates a virtual touch screen that follows your hand around. For app developers, the most important element here is the distance of a finger or tool (collectively referred to as pointables) from this virtual surface. This value is called “touchDistance,” which is illustrated below:

image

I’ve hosted a simple JavaScript demo to visualize the touchDistance here.

What is a Tap?

Because we’re emulating a touchscreen, let’s first consider how a normal touchscreen (such as one you’d find in a smartphone) would interpret a tap.

image

Depending on the application, an action can be triggered either when you first touch the screen, or when you take your finger off the screen. Triggering the action on release is the most common response, and is the approach I will use here.

How should we extend this concept to 3D space with the Leap Motion Controller? Because the API provides the TouchDistance value, it’s practically identical – except now the ‘touch’ value is analog.

image

How to Implement Tap Detection

Implementing tap detection in either the “Press” or “Release” mode is quite simple. Since the Leap Motion API smooths the input quite well,  you don’t need to worry about filtering or timing.  All you need to do is check for the transition of the pointable in and out of the touch zone. Here’s an example:

Thanks to the Leap Motion API storing frame history, the tap detection code is completely standalone (no state needs to be maintained).

void onFrame(const Controller & controller)
{
    Frame currentFrame = controller.frame();
    Frame lastFrame = controller.frame(1);

    if (!lastFrame.isValid()) 
        //Only one frame so far - do nothing
        return; 

    for (int p = 0; p <
        currentFrame.pointables().count();
        p++)
    {
        Pointable pN = currentFrame.pointables()[p];
        Pointable pN_prev = lastFrame.pointable(pN.id());

        //Pointable has only been around for one frame
        if (!pN_prev.isValid())
            continue; 

        //Look for the transition from
        //inside to outside of the touchzone
        if (pN_prev.touchDistance() <= 0 &&
            pN.touchDistance() > 0)
        {
            printf("Leap Thread: Pointable %d tapped\n",pN.id());
        }
    }
}

However, this code comes with a strict warning: It will only work if called on the Leap API thread as a Listener. If you call it using polling, you will receive unreliable results. For example, consider an application with its own rendering loop (i.e. a game):

int main()
{
    Controller controller;
    TapDetector tapDetector;

    while (true)
    {
        tapDetector.onFrame(controller);

        //Other application logic

        //Rendering 
    }    
    return 0;
}

If the rendering loop runs at a slower pace than the Leap API (which is quite likely), then frames returned by controller.frame() may not be consecutive. This can cause frames to get skipped.While the multi-threaded approach is the simplest solution, it leads to synchronization  challenges for the rest of your application.

Fortunately, there’s an easy solution: To detect taps from your own thread, you can catch up on missed frames by going through the history. In this case the last frame processed needs to be stored so we know how far in the history to go back.

       
Frame lastFrame;
void onPolledFrame(const Controller & controller)
{
    Frame thisFrame = controller.frame();
    int MaxHistory = 100;
    for (int i=1;i < MaxHistory; i++)
    {
        Frame frame = controller.frame(i-1);
        Frame previousFrame = controller.frame(i);

        //Last frame is invalid
        if (!previousFrame.isValid())
            break;

        //We reached the frame that was processed last time
        if (frame == lastFrame)
            break;

        for (int p = 0; p < frame.pointables().count();p++)
        {
            Pointable pN = frame.pointables()[p];
            Pointable pN_prev = previousFrame.pointable(pN.id());
            //Pointable has only been around for one frame
            if (!pN_prev.isValid())
                continue; 

            //Look for the transition from
            //inside to outside of the touchzone
            if (pN_prev.touchDistance() <= 0 &&
                pN.touchDistance() > 0)
            {
                 printf("App Thread:Pointable %d tapped\n",pN.id());
            }
        }
    }
    lastFrame = thisFrame;
}

With this function, you can enable reliable detection of taps on any thread. To see these and other tap code samples, check out my github repo or the code in the JavaScript sample.

Discussion

In my next article,  I’ll discuss how to choose between fingers when many are present, and some thoughts on meaningful visual feedback. For now, let’s close with two questions – what’s your approach when integrating tap gestures within an app? And which gestures or topics would you like to hear about next on Developer Labs? Post your thoughts in the comments below, and also feel free to mention any problems you might be having with implementation.

Originally appeared on Recognizing Taps with the Leap Motion API

Recommended Posts