M1 Interactive is a design and development firm that focuses on the creation of immersive digital environments with platforms like Unity3D, Flash, Processing, and Arduino. They recently partnered with Leap Motion to create a freely available starter kit for all developers who want to build the next generation of games on the Unity platform. This post is the first in a three-part series.

Note: The Starter Kit has been deprecated. Be sure to download our Unity Core Assets (or Core Assets: VR Edition) from developer.leapmotion.com/downloads/unity.

As developers and gamers, we’ve become used to designing around, and playing with, older controllers and ways of interacting with computers. Unfortunately, this also means that we’ve been limited by them. For example, popular apps like Surgeon Simulator 2013 and Octodad: Dadliest Catch offer players the ability to precisely control character movements. This ability is something that gamers want, but current keyboard and mouse setups make these kinds of controls clunky. These games and their controls usually end up being comical in nature – like children’s marionettes.

Trying to control a character’s arms or legs, in real time and in 3D space, is difficult with a mouse or joystick, which are designed to work primarily in 2D space. With the appearance of newer technologies, these control schemes are becoming less whimsical and more of a common practical tool for interactivity. Nonetheless, interactive controls are still in the infancy stage and most current interactive controllers – while fun for casual games – are not viable options for most applications due to their lack of accuracy.

However, the Leap Motion Controller offers some exciting new possibilities for gamers and developers alike. Launched earlier this year, it lets us virtually get our hands into the game and directly interact with virtual objects with natural, fluid movements. While the technology is now on store shelves, it’s time for the software to catch up with the hardware.

That’s why we created our Leap Motion Starter Kit for the Unity3D platform. Rather than use the device to define movements directly, as is necessary with most controllers, our goal was to do something new and innovative – and instead allow the virtual hands to interact with objects, which would in turn define the controls in the game. By partnering with Leap Motion to make the kit free for everyone, we also hope that our work will benefit the Unity platform as a whole.

In this post, the first in a three-part series, we’ll show you some highlights from the starter kit, including why we designed certain scenes the way that we did.

Building natural control interactions

With a mouse or joystick, we are limited to interacting in 2D space only. But with the Leap Motion Controller, gamers are able to interact with the world in 3D space. This makes defining parameters, such as position or rotation of an object in space, much easier. As developers, this means that we can create physical extensions of the user’s body within a game space. Players don’t have to fumble with a mouse or joystick to line up the game’s hand to pick up an object – they can just reach in and grab it.

But as we mentioned earlier, developers can also provide players with in-game tools and control mechanisms that mimic real-world interactions. We created the Leap Motion Starter Kit to allow gamers to interact with game objects using just the data from their hands and fingers – such as palm/finger position, rotation, and direction. There’s a lot you can do with that info: throw objects, swing a sword, shoot a gun, drive a vehicle, or pick up a bowling ball.

Why these interactions in particular? When we started brainstorming our starter kit, we decided to create scenes for games we would want to play with the Leap Motion Controller. As typical gamers, we knew what we wanted – swords, guns, grenades, flying, driving, etc. – and we figured that other game developers would probably want the same. Our goal was to create a flexible, easy-to-use tool that could be implemented into any Unity project and easy enough for newer developers to quickly start making games using the Leap Motion Controller.

With our steering wheel driving controls, instead of making the controls based on the horizontal or vertical position of the palms, the user must grab onto the steering wheel and rotate it the way you would a real steering wheel. In other words, it’s not the movements of the hands above the Leap Motion device that control the car, but instead the steering wheel and lever, which are controlled by the virtual hands in the game. In this way, we’re breaking down the layer between the controller and the virtual world, as well as mimicking reality and creating an infrastructure for future control implementations.

In the driving and ATV scenes, when the player grabs hold of the steering system, we record that grab position and lock a virtual representation of the hand to that position – so that whenever the player moves their hand, we calculate the amount of rotation, as well as rotation direction, to correctly rotate the steering system. As a result, we are able to have a visual representation of the hand and rotate the steering system respectively based on the hand’s position above the Leap Motion Controller.

The player can throw a grenade in 3D by moving and opening the hand. Apart from direction, the speed of your hand defines the linear velocity. The rotation of your hand defines how much angular velocity is applied to the grenade, resulting in an easy and precise simulation of throwing a grenade.

That’s all for now! Our next post will take a look at four Unity-built games that are already available for the Leap Motion platform, and what insights their set designs and control schemes have to offer developers. Finally, we’ll bring everything together by highlighting some design challenges that we were able to overcome, and which you’ll want to avoid when designing your own games.