As a truly 3D human interface, the Leap Motion Controller opens up a lot of possibilities for developers of all stripes. For modern designers, it means that we have to constantly rethink and tinker with a new way of interacting with computers. It can be frustrating.

At this point, you might expect me to say that it doesn’t have to be frustrating. While you’d be wrong – constantly running into walls is a part of any experimental process – it is possible to lay down a solid foundation. You need to be bold. And you need a boilerplate. So put on your design hat and dive down into the rabbit hole – it’s time to get messy.

If you can’t see the boilerplate in action above, check out the demo video at the top of this post.

Modern Design

I am a modern designer. What does this mean?

• I don’t use pen, paper, or paint. I use a computer and write programs.
• I don’t do 2D or stationary or immovable. I do real-time animated 3D.
• I don’t do closed source, paid-for apps. I do open-source, free apps.
• I don’t do downloadable executables. I do cloud web-apps.

If I do my job right, there will be plenty of work for the less modern.

I am a modern designer. What does this mean?

• I am not a programmer. I don’t worry about variable types.
• I am not a QA engineer. I don’t worry about test suites.
• I am not a project manager. I don’t worry if my code will be decipherable in ten years.

If I do my job right, there will be plenty of work for the techs.

And I know that there are plenty of you out there who build demos, who get things going, who get so excited about things – that you make things happen.

And, since you are reading this post, you may well be as enthralled by the Leap Motion device as I am.

I’ve had a Leap Motion device since July. (Thank you Rodrigo.) I’ve been designing apps just about every day since then. This post is a double-click into the tools I use to build and publish Leap Motion apps with a flick of the wrist.

So what does my current toolkit look like? The first part is attitude. The second part is boilerplate.

Lesson Learned #0: Failure is your friend

Failure is lesson learned number 0 because any other number would indicate success.

The first thing to consider is that designing is about countless failures. Things go wrong. The thing is in the wrong position, the thing is the wrong color. The thing is not what you wanted anyway.

There is no IDE for elegance. There is no designer console that says: ‘This action sequence is a fail.’ The only way faster to the other side is to get through more fails – to get through more fails per hour. Find tools that help you fail faster.

Lesson Learned #1: Don’t even think about compiling

Modern compilers can be very fast. But if you have to make something like a hundred revisions to get something right, then adding even a few seconds to the process can turn an interesting chase into a mind-numbing checkout process.

JavaScript rules because it provide near instantaneous results. Press Enter. See result. Repeat. (Often many times.)

Lesson Learned #2: Perl, PHP and Python are Pathetic

For years the only ways to do 3D was on the server with one of the Ps (or with Java3D, hahaha). WebGL has changed all of that. Now any web app can access the incredible GPU that’s inside the computer you’re reading this text on. There’s a gotcha. And it’s a biggie. Ya gotta use JavaScript to get to this graphic goodness.

Lesson Learned #3: JavaScript is your Friend

JavaScript sucks. Everybody knows that. Even me. Google ‘JavaScript sucks’ and see if you also get 12 million hits.

But then I am not a programmer (etc). Not only does JavaScript access the GPU, and do whatever when you press enter, it also has access to the DOM. TL;DR: OMG, ponies and kittens. Access to the DOM for me as a designer is as good as giving me an intelligence pill. I can do stuff now that I can’t believe I’m doing.

Real programmers will look at what I write and think ‘cowboy.’ Is it because I am using their tools incorrectly or using them in unintended ways? No inventor knows how their creations will be used – let the results be your yardstick.

Lesson Learned #4: Do Whatever Mr.doob Does

Do you know how to lerp? Can you multiply matrices? Can you discern an Euler from a quaternion? Neither can I. 3D is really complicated. The mathematics and computational requirements for 3D is a dimension apart from your normal flatland thought processes.

Fortunately, there’s a library for that (JavaScript has ‘libraries’ for everything). [It’s hard to believe that all the difficult issues are being solved by librarians. So it goes. ;-] And Mr.doob is a ‘librarian’ with a very big white hat. His Three.js library can give you the Manhattan length of a vector or trisect a parsnip (not really) while all the time festooning the space with 3D geegaws. This means that all I need to know is that quaternions exist and I can dump any of them into one of Mr.doob’s black boxes and, bingo, problem solved.

But that’s only part of it. What is impressive are his – and the many others involved – responses to the four thousand issues raised in GitHub and to perhaps more in StackOverflow. Smart people saying smart things.

Mr.doob is not alone. For example, what Lindsay Kay is doing with Scene.js or Daniel Haehn with Xtk are also evidence of quite fabulous thinking. We kneel on the shoulders of giants.

Lesson Learned #5: GitHub is Your Content Delivery Network (CDN)

GitHub is wonderful and you and 4.5 million users already knew that. But do you know how wonderful it actually is? Neither did I. It’s just that many people seem to miss that GitHub is not only a wonderful pace for source code but also a wonderful place to host a HTML/CSS/JavaScript web site. The GitHub pages feature – using the gh-pages branch – is overlooked by too many users. But GitHub may be the best content delivery network there is. This may be debatable when you compare GitHub to fee charging CDNs but I can’t think of another no-charge operation that works as nicely.

Of course, I learned about all of this by looking at Mr.doob’s examples. And where would you keep your documentation? And where would you keep your two free editors? Follow Mr.doob and keep them all on GitHub.

Lesson Learned #6: Keep a Boilerplate File

Every pilot follows a pre-flight checklist. The priest always reads the communion service from the Bible – every time. Every McDonald’s chef builds the burger by the book. Humans are quite fallible. The written word helps compensate. Thus people who code a lot tend to keep what is often termed boilerplate code.

You can see my current boilerplate for Three.js + Leap Motion in action at the top of this post, or see it here. Items of interest include the following. (Each link below opens the source on GitHub with the referenced lines highlighted.)

Lines 1-12

Note that all the text is lower case and there are no quotation marks. And it all works. A header that does not shout and just feels airy.

Lines 7-178. The file is just about 100% JavaScript. There’s no CSS file. In between the body tags, the only tags are script tags.

This may not be the most effective way of doing things or the fastest – I’ve never performance-tested this style of coding. But it is teaching me how to use JavaScript to manipulate the DOM.

And most importantly, I don’t have to keep switching between CSS and JavaScript and whatever I’m designing.

Lines 8-11

The next thing you will notice is that I load all the scripts from the originators’ repositories rather than from my repositories. There are many good reasons for this, but I guess the main ones are that updating and caching are taken care of automatically.

Lines 13-14

Next you will find a bunch of global variables unprotected by any sort of name space. ‘How odd!’ you say. And you have reason. The code in the Three.js library is fully name spaced and prototyped and objectified and whatever. But the code the end-user sees is late-nineties beginner-style code.

I’ve had many discussions with myself on this topic. But the crux of the issue lies in the first sentence of the Three.js readme:

The aim of the project is to create a lightweight 3D library with a very low level of complexity — in other words, for dummies.

Global variables are what dummies do and functions that are declared as functions are what dummies do and so on.

And I have come to espouse this dummy style of writing for several reasons. Remember, I am not a programmer. I am a designer. An idea has just popped up. I need to see something now. Now is not the time to consider elegance. Just do it. And this is dummy code. It’s the fastest way to get that ‘hello world’ running. Almost else anything is better. But if there were a really slick boilerplate, then fear that new stuff was not as elegant would creep in.

Actually this is all getting rather complicated. Can I summarize briefly?

When I am designing, I am looking at what appears on the screen. If the code is complicated, then I end up looking at the code and not the screen. So code for dummies is really good code when you are designing.

In Detail

Lines 19-111. Most everything is usually in this init.

Lines 22-32

All JavaScript that deals with the CSS and HTML. I’m having fun learning how to do this. And I’d rather be good at JavaScript than be good at CSS.

Lines 35-77

// Three.js basics
scene = new THREE.Scene();renderer = new THREE.WebGLRenderer( { antialias: true } );
renderer.setSize( window.innerWidth, window.innerHeight );
renderer.shadowMapEnabled = true;
document.body.appendChild( renderer.domElement );camera = new THREE.PerspectiveCamera( 40, window.innerWidth / window.innerHeight, 1, 3000 );
camera.position.set( 0, 250, 600 );
controls = new THREE.TrackballControls( camera, renderer.domElement );
controls.target.set( 0, 100, 0 );stats = new Stats();
stats.domElement.style.cssText = 'position: absolute; top: 0px; zIndex: 100; ';
document.body.appendChild( stats.domElement );// Lights
light = new THREE.AmbientLight( 0x333333);
light.color.setHSL( 0.1, 0.5, 0.3 );
scene.add( light );light = new THREE.DirectionalLight( 0xffffff, 1 );
light.position.set( 1, 1, 1 ).normalize();
scene.add( light );

light = new THREE.DirectionalLight( 0xffffff, 1 );
light.position.set( -1, -1, -1 ).normalize();

light = new THREE.DirectionalLight( 0xffffff, 1 );
light.position.set( 0, 500, 0 );
light.castShadow = true;
light.shadowMapWidth = 2048;
light.shadowMapHeight = 2048;
var d = 200;
light.shadowCameraLeft = -d;
light.shadowCameraRight = d;
light.shadowCameraTop = d * 2;
light.shadowCameraBottom = -d * 2;

light.shadowCameraNear = 100;
light.shadowCameraFar = 600;
// light.shadowCameraVisible = true;

Let’s get in the basic things we need to get going: scene, camera, controllers, lights, and a statistics widget. Standard Three.js stuff.

Adding this one single line gives you a hugely powerful camera that allows you to zoom, pan, and rotate this display with your pointing device:

At the other end of the spectrum, the lights are a black hole sucking in all available energy. One can spend days and days on lighting. One day, I will come up with a simple and effective lighting scheme. I am just waiting for the light bulb to turn on…

Lines 79-110

// axes
scene.add( new THREE.ArrowHelper( v(1, 0, 0), v(0, 0, 0), 30, 0xcc0000) );
scene.add( new THREE.ArrowHelper( v(0, 1, 0), v(0, 0, 0), 30, 0x00cc00) );
scene.add( new THREE.ArrowHelper( v(0, 0, 1), v(0, 0, 0), 30, 0x0000cc) );// ground plane
material = new THREE.MeshBasicMaterial( {color: 0xaaaaaa } );
geometry = new THREE.CubeGeometry( 600, 10, 300 );
mesh = new THREE.Mesh( geometry, material );
mesh.position.set( 0, -10, 0 );
mesh.castShadow = true;
mesh.receiveShadow = true;
scene.add( mesh );// palm
geometry = new THREE.CubeGeometry( 100, 20, 80 );
material = new THREE.MeshNormalMaterial();
palm = new THREE.Mesh( geometry, material );
palm.castShadow = true;
palm.receiveShadow = true;
scene.add( palm );// fingers
geometry = new THREE.CubeGeometry( 16, 12, 1 );
for (var i = 0; i < 5; i++) {
mesh = new THREE.Mesh( geometry, material );
mesh.castShadow = true;
mesh.receiveShadow = true;
scene.add( mesh );
fingers.push( mesh );
}
}

Time to draw some geometry: axes, a ground plane, a palm, and five fingers. Give the objects this weird Three.js normal color. Whenever you see this color scheme – THREE.MeshNormalMaterial() – it tends to be a giveaway that Three.js is involved.

Lines 113-125

This info thing is a fairly new addition to the boilerplate. Clicking on the i inside a circle causes the div to display. The code is really simple. For dummies really. But more importantly, this is a an easy method to link to source and to credit Mr.doob and Three.js, as well as Leap Motion. When you get into open source, it’s really good to give credit to your sources. This little pop-up <div> seems an effective way of doing saying ‘Thank you!’.

Lines 127-165 & Lines 169-174. Two loops. Or more accurately, two loops repeatedly call window.requestAnimationFrame.

Leap.loop( function( frame ) {
var hand, direction, len;
if ( frame.hands.length > 0) {
hand = frame.hands[0];
palm.position.set( hand.stabilizedPalmPosition[0],
hand.stabilizedPalmPosition[1], hand.stabilizedPalmPosition[2] );
direction = v( hand.direction[0], hand.direction[1], hand.direction[2] ); // best so far
palm.lookAt( direction.add( palm.position ) );
palm.rotation.z = -hand.roll() ;
//palm.rotation.set( hand.pitch(), -hand.yaw(), hand.roll() );
palm.visible = true;data.innerHTML = 'Hand X:' + hand.stabilizedPalmPosition[0].toFixed(0) +
' Y:' + hand.stabilizedPalmPosition[1].toFixed(0) + ' Z:' +
hand.stabilizedPalmPosition[2].toFixed(0);} else {
palm.visible = false;
}len = frame.pointables.length
if ( len > 0) {
var pointable;
palm.hasFingers = true;
for (var i = 0; i < 5; i++) {
finger = fingers[i];
if ( i < len ) {
pointable = frame.pointables[i];
finger.position.set( pointable.stabilizedTipPosition[0],
pointable.stabilizedTipPosition[1], pointable.stabilizedTipPosition[2] );
direction = v( pointable.direction[0], pointable.direction[1], pointable.direction[2]);
finger.lookAt( direction.add( finger.position ) );
finger.scale.z = pointable.length;
finger.visible = true;
} else {
finger.visible = false;
}
}
} else if ( palm.hasFingers ) {
for (var i = 0; i < 5; i++) {
fingers[i].visible = false;
}
palm.hasFingers = false;function animate() {
requestAnimationFrame( animate );
controls.update();
renderer.render( scene, camera );
stats.update();
}

I investigated trying to combine the two into a single loop and it started getting complicated. (Hello, this is a boilerplate for dummies designers.) The animate function is standard Three.js, and the Leap.loop is worth checking out because it handles all the updating of the 3D geometry.

Lines 132-137

direction = v( hand.direction[0], hand.direction[1], hand.direction[2] ); // best so far
palm.lookAt( direction.add( palm.position ) );
palm.rotation.z = -hand.roll() ;
//palm.rotation.set( hand.pitch(), -hand.yaw(), hand.roll() );
palm.visible = true;

Is it better to use the brand new (as of a week ago) palm.pitch(), palm.roll(), palm.yaw() functions or it is better to use a Three.js lookAt( a position )? There’s a forum thread on this topic; the boilerplate opts for the simpler method. But if you have experience with this issue, kindly do add your thoughts to the comments.

The Three.js lookAt function is quite amazing. The problems revolve – literally – around gimbal lock. As objects approach the poles (90 degrees vertical), they have difficulty telling where to turn so they start spinning. The lookAt function deals with this issue effectively.

Line 138

data.innerHTML = 'Hand X:' + hand.stabilizedPalmPosition[0].toFixed(0) + ' Y:'
+ hand.stabilizedPalmPosition[1].toFixed(0) + ' Z:'
+ hand.stabilizedPalmPosition[2].toFixed(0);

This line sends numbers to the screen. It’s not for end users, it’s for debugging. If you use console.log, you get this fast-scrolling console that’s too fast to read, until – because of the surfeit of data – it freezes the console. Much better to use a line like this.

And note that the ‘data’ variable has not been declared as a variable anywhere. It’s just the name of the ID given to a div tag in line 32. The DOM knows everything and takes care of stuff like this for you for free.

Lines 151-165

pointable = frame.pointables[i];
finger.position.set( pointable.stabilizedTipPosition[0], pointable.stabilizedTipPosition[1],
pointable.stabilizedTipPosition[2] );
direction = v( pointable.direction[0], pointable.direction[1], pointable.direction[2]);
finger.lookAt( direction.add( finger.position ) );
finger.scale.z = pointable.length;
finger.visible = true;
} else {
finger.visible = false;
}
}
} else if ( palm.hasFingers ) {
for (var i = 0; i < 5; i++) {
fingers[i].visible = false;
}
palm.hasFingers = false;

There’s not a lot of data coming from the Leap Motion pointables, so we must use lookAt. Occasionally the fingers will go for a spin, but they mostly look OK.

More Thoughts

You see nothing about the Leap Controller object in the boilerplate code. Although the API documentation devotes a good amount of space to the Controller object, I have not found a great use for it in everyday coding. This is substantiated by the Leap.js source code lines 1772-1773 which state:

* Leap.loop() sets up the Leap controller and WebSocket connection for you.
* You do not need to create your own controller when using this method.

Also, I don’t always run the Leap Motion Control Panel. As long as the Leap Motion software has been installed, the device is plugged in when I turn on the computer and Leap.js is loaded, then Leap.loop will run in any JavaScript app.

Even if you have five apps loaded. Yup. Each of the Leap.loop instances will happily access Leap Motion device data simultaneously. Quite fun to watch.

Must We Complicate?

I just want to open my browser and an editor and go. So you say: ‘But you can’t do that anymore because of all the cross-origin security issues. So you should install Python and run its server feature thingy and yada yada…’

And then I reply, yes, that is what a good systems administrator would do. But I am not a systems administrator, I am a designer. So I just open the Chrome browser with the following additional command line string “—allow-file-access-from-files”. The total command line is saved in a shortcut I use for coding and giving demos, and thus is reasonably secure, as it cannot be accessed by any apps that call for the browser. And it means I can open my laptop and give a demo in seconds.

Conclusions

You may have noticed that there’s not been a lot in this post about the Leap Motion device itself. And that is the wonderful thing. It just works. Type in the right parameters and the data appears. It has been perhaps the easiest brand new physical device to communicate with that I have ever experienced.

But, please do note, I did not say that the device is perfect. On the contrary, a huge issue is that a lot of the device data sucks. Fingers go missing. Hands start to wobble. And much more. But these are all release onesy-twosy issues. These are all issues that can be and will be fixed given time and experience and ambition.

In the meantime, here’s a question for you. For many years, I’ve lived without a left hand. So I delight in making (hee hee) off-handed remarks. Thus you can imagine how I describe the Leap Motion device. I say things like this: “Oh, it’s so handy!’ or “Allow me to point out the benefits…” And, of course, my current really big project is designing and building the first Internet second hand store.

But I’m sure there’s more. So as I wave my hand goodbye, I have to ask – any suggestions for similarly disarming bons mots?

Theo Armour is a designer and erstwhile script kiddie who lives in San Francisco and Paris. You should check out his Leap-oriented Gestification project on GitHub, read his posts on Jaanga and follow him on Twitter.