As any interaction designer knows, there’s rarely only one right answer, but there are usually about a million wrong answers. If you’ve ever developed for the Leap Motion Controller, you know that getting a solid interaction model down is hard. We know this better than anyone. That’s why we’ve been experimenting with a variety of interaction models and developing best practices. Fundamental research on everything from intuitive gesture flows to camera controls.

Today, I’d like to show you two camera control models that I’ve developed. These are still a work in progress, and far from polished. They’re also just the tip of the iceberg – on my GitHub demo page, you’ll find seven different models altogether! We’re still working out which models are “best” under different circumstances; let me know your favorite in the comments.

Enter THREE.CameraControls

If you’re anything like me, camera controls are the first thing you do for a Three.js project. What’s the point of seeing a pulsing space puppy if you can’t see the other side? It’s the movement that lets us know we’re operating in three dimensions, and so I want to introduce that movement as soon as possible.

It’s simple enough – all you need to do is include the script, create the controls, set the parameters, and update the controls. But what’s behind the curtain, what’s in the included script, is far from simple. The first time I ever looked at THREE.TrackballControls (back when I was working on Recursion, so I had no idea what I was doing), I think I killed a few brain cells.

There were words like quaternion and rotation matrix, and although I vaguely remembered those terms from my undergraduate physics courses, it felt a bit like staring at the Book of Kells. (The results, of course, are amazing.) Instead, I continued trying to do the simple thing: include the script, create the controls, set the parameters, and update the controls.

This, however, is just a single control, and Three.js has many. Whatever the control though, it still uses the four step process: include, create, set, update. Thus, for our Leap Motion-based Three.js camera controls, I wanted to set up the structure so it was exactly the same.

This part was easy enough. All I needed to do was mimic the Three.js camera controls file structure, and then utilize (as we normally would) camera control. The problem is that instead of being able to steal pages from the magic Book, I actually had to create some.

Spring Controls

Probably my favorite controls are the Spring Controls (go to slide 8). The way that they work is to create an “anchor,” which connects to your camera via a spring. The anchor is then moved from location to location, dragging your camera along with it. What excites me about these controls is that they let you both create “spectral” and “quantized” movement – you can either tap your fingers together to precisely place the anchor at a specific location, or hold your fingers together to slowly float through the scene.

Another cool thing about the spring controls is the ability to add a basic UI – you can place a sphere where the anchor is and where your hand is. Once you do this, it forms a sort of crosshair that lets you know which way you’re going to look when you pinch, simply by looking at the difference between the hand position and the anchor position.

Pros:

  • Smooth like butter
  • Lets you fly anywhere you want in the scene with relative ease
  • Once you let go, slowly brings you to a final resting point

Cons:

  • Moving camera near poles results in some weirdness
  • Uses pinch, which removes the ability to use it for other gestures
  • Easy to get lost in space if you have no reference points

Potential uses:

  • Space flying games
  • Plane flying games
  • A quick addition to visual experiments

It’s interesting that this UI didn’t arise from any conscious effort on my part. Rather, it was through playing with the device and creating control after control that ended up getting thrown away that it emerged. Playing around with the technology and coding constantly taught me what does and doesn’t work.

If there’s any point to this article, it’s that constantly playing and coding is the only way to build anything really great. With any luck, my experiments will provide you with the shortcuts you need to play at a higher level, just like I stole a few pages from Mr.doob’s book. It’s all part of the socially acceptable thievery of coding life.

Eye Look Controls

In the Eye Look control scheme (slide 10), you pinch and move the camera’s position with your right hand, and pinch and move the ‘lookat’ with your left hand. What feels really organic about this is the separation from the left and right hand. If I want to orbit around a point, I always put my right hand over the device. However, if I want to change the point I’m orbiting around, I use my left hand.

Usually I’m not a big proponent of having two-handed controls. However, they do provide some pretty amazing benefits, as the separation of use by hand creates a way for users to logically parse what to do next.

Pros:

  • Always looking at the same place, so it’s easy to stay in control
  • Movements feel smoothish
  • Relative movements allow for the exploration of the entire scene

Cons:

  • Moving camera near poles results in some weirdness
  • Uses pinch, which removes the ability to use it for other gestures
  • Relative movement means that you can get very far away from your target, leading to depth being difficult to judge
  • Difficult to move through an entire scene quickly

Potential uses:

  • Slowly examining a full scene
  • 3D modelling camera controls
  • A quick addition to visual experiments

Again, the creation of this control came from multiple iterations of playing around, not thinking of executing an idea. It started as Pointer Controls (slide 9), but when it felt like something was missing, the additional hand was added in.

What’s next?

Now, these are only two of the many controls that have either been created or are in the works. Here’s where you can learn more:

Tutorial: leapmotion.github.io/Leap-Three-Camera-Controls
Repo: 
github.com/leapmotion/Leap-Three-Camera-Controls

There are many ways that I could imagine these controls being improved. For example, imagine if each of the controls had a ‘fallbackControls’ parameter that would be updated when the Leap Motion Controller was plugged in. Or a flying control where your hand is a spaceship.

I’d love to see what could be created what’s already available. Please try out the controls, tell me what you think, and send along anything you make to icohen@leapmotion.com or @cabbibo. You can also post your ideas and projects in the comments.

A musician and creative researcher, Isaac explores the boundaries of space and sound with the Leap Motion JavaScript API. He has designed a variety of experimental interfaces that let you reach into music, big data, and even the fourth dimension.

Twitter