When creating new demos for the Leap Motion Controller, the first question I tend to ask myself is: ‘How can I create something that was not possible to make before?’ This line of thought tends to lead me to a place of wondering how I can make a spaceship that lets me wing suit base jump on the moons of Jupiter, but sometimes it just makes me want to create a demo. One of the things that really excites me about the Leap Motion Controller is the extra dimension of interaction, especially when it pertains to 3D objects.
All of this made me decide to create a program that let you manipulate the mesh of a sphere. If you have a Leap Motion Controller, the program is available at cabbibo.com/leap/meshGallery
As many people who I’ve talked to who are programming with the Leap Motion Controller have told me, one of the bigger issues that you have to solve is how to start and stop a motion. In the case of mesh manipulation this is especially difficult, because the way that you begin or end a motion cannot affect the placement of the vertex you are moving.
In the end there were a few things I did to try and solve this problem.
The first was to make it so that the ‘plucker’ which would attach itself to a vertex, would be a point that lay directly in the middle of the two ‘finger markers’, as seen in the screen shot below.
This lets you see the point that attaches to the objects, and also means that when you move the ‘plucker’ over a vertex, it will automatically attach to the object. I have found this to be especially useful because we as humans kind of suck at 3D (or at the very least aren’t good at telling depth on a 2D screen) This means that doing something like ‘intersect the object, then click’ becomes even more difficult because to have to overcome two obstacles, rather then just one.
Once you have attached to the object, you get to feel the true power of the Leap Motion Controller. You get to effortlessly move the vertex around in a dimension that literally was impossible to interact with before. The first time I got this to work, it was an experience that only the likes of Gandalf or Merlin could begin to comprehend. I was moving in a whole new dimension, and if felt good. It literally felt like I was interacting with the computer, and it knew something that it never had before: the Z-Axis.
That being said, there still was the problem of letting go of the object. In the end I ended up going with the event ‘when the last frame has two fingers and this frame has less then two fingers’ to trigger the release of the vertex. However, even with the specificity of this event, there are still multiple ways a user could trigger it. Do they put 2 fingers in the field, and then wildly move them away? This obviously won’t work, because then every vertex would follow the fingers to the edge of the field, making for a pretty ugly mesh. Also they could place two fingers in the field, and curl one of them until it is no longer read, but this would make the average ‘plucker’ point follow the curled finger, and move the vertex.
What I ended up saying is: ‘Make a ‘V’ with your fingers to grab a point, and to let go, close the ‘V’ your fingers make’. This seems to work pretty well, and I am able to create some pretty interesting shapes
Going forward, I would love to experiment more using the hand position as the stable point, so that I can wiggle my fingers all I want without affecting the point of the vertex, but that is something for another experiment!
A musician and creative researcher, Isaac explores the boundaries of space and sound with the Leap Motion JavaScript API. He has designed a variety of experimental interfaces that let you reach into music, big data, and even the fourth dimension.
