Hi everyone,
I’d like to take a moment to talk about a series of developments we’ve been working on specifically for virtual reality. The first set involves our existing peripheral device and new things developers can do with it starting today, while the second is a look at some of our next-generation hardware and software efforts that we’re currently building from the ground up for this exciting and emerging space.
If virtual reality is to be anything like actual reality, we believe that fast, accurate, and robust hand tracking will be absolutely essential. We believe in the concept of other specialized controllers as well, but our hands themselves are the fundamental and universal human input device.
One of the most exciting things to us about virtual reality is that our technology can be more than just your hands – it can be your eyes as well. This builds off the release of a new API which opens up raw infrared imagery straight from our sensors. When mounted directly onto a head-worn display, these images become stereoscopic windows into the world around you. What it sees, you see.
This expands the tracking space to be in any direction you’re facing. You can reach forward, turn around, look up and down, and the tracking follows you wherever you go. Because our device’s field of view exceeds that of existing VR displays, you’ll find it can start to track your hands before you even see them.
To help our community explore this paradigm with us, today we’re releasing a VR Developer Mount. Available from our online store for $19.99, the mount allows developers to easily and consistently attach and remove the Leap Motion Controller from a VR headset.
We’re also releasing a software update for our beta SDK, which includes a massively improved ‘top-down tracking’ mode, as well as Unity and C++ examples. These show how to use both the image overlays and the tracking data from a head-mounted position, then give further examples of more sophisticated 3D interactions. Thanks to major software advancements since our V2 tracking developer beta was launched in May, this is all possible with the current generation peripheral device.
At the same time, I’d like to also give a hint of what we’re working on for the future. One prototype sensor that we’re beginning to show today (and will be giving out more information on in the future) is codenamed “Dragonfly.” Designed to be embedded by VR OEMs, Dragonfly possesses greater-than-HD image resolution, color and infrared imagery, and a significantly larger field of view.
With next-generation “mega-sensors” like this, a Leap Motion device can literally become your eyes into the digital and physical realms – allowing you to seamlessly mix and mash, fade and blend, between virtual environments and the sharpness of the real world.
While motion control and virtual reality are both very new platforms, we’ve always been in awe of people’s deep and abiding passion for the dream of a digital and physical convergence. It’s this energy which drives us to work tirelessly until it is a reality.
Video unavailable in my country. How fascinating!
Sorry about that, Da_James! Fixed the issue.
Thanks! A real innovation, i’m happy to see the Leapmotion in HP Notebooks for example (instead of crappy touchscreens, who wants that!). Keep up the good work 😉
Da_James..you need to sign up for a global VPN tunneling services. Then you can watch ANY video and you get privacy since you are hiding your own public IP. 🙂
Does the Leap sensor mess at all with Rift’s new positional tracking camera?
That’s what I’m interested in, too. Is there interference?
The mount by itself doesn’t cover any LEDs; when the Leap is in place, it covers the 3 central LEDs you see in this image: https://d3nevzfk7ii3be.cloudfront.net/igi/ZlJ5ShKOVUOtImyt.large
This doesn’t interfere with positional tracking, as the DK2 is designed to have way more LEDs than strictly necessary (allowing for these types of input experiments). We’ve used positional tracking with the mount on DK2 successfully without encountering any issues.
Yeah. Awesome 😀
Nice, is that an Oculus?
Indeed, a DK2 at that!
Sweet! Very nice, … the possibilities!!!
Does this setup interfere with the positional tracking camera (which is also IR) of the DK2? Or more specifically: Can you still use positional tracking with this kit & the DK2?
You can definitely still use positional tracking with a mounted Leap on the DK2.
Oh, man. The mount looks fantastic, but when can I get my hands on the Dragonfly?! What a tease!
The Dragonfly is designed to be embedded by VR OEMs; no news or ETA yet, but stay tuned for updates 😉
hello, do you plan to get your SDK with c# libraries ?
The SDK includes a C# API that’s frequently used for Unity… what kinds of libraries would you like to see?
We are used to build kiosk or business interactive with c# and wpf. We are actually have new Kinect sensor from its preview phasse and are always looking on new interaction possibilities but not necessary in game area as we are not in That business.
Dors your techlogy could be use with wpf ?
Certainly! While it uses the (older) v1 SDK, I encourage you to check out this WPF+Leap tutorial: http://pluralsight.com/training/Courses/Description/developing-leap-motion-csharp. Also feel free to post about your project (along with any questions) on https://community.leapmotion.com 😀
Will this be able to work with DIY VR kits, like Google Cardboard?
There are basically two requirements for a VR setup with the mount: enough space to put the mount, and a platform that can run the Leap Motion Controller (right now, Mac, Windows, or Linux). While we’ve been working on an Android SDK, it’s not yet available.
Man, great news. Exciting stuff! You know, I’d be fairly happy if ‘Dragonfly’ found its way into Rift CV1 or Samsung’s Gear VR – but either way, I’m excited to see where you guys go with this. It’s a good move.
On to something maybe you can comment about – any easy options for us Unreal Engine 4 guys? I know Epic’s “code plugins” are still in flux, and that’s making life a bit difficult, but I’d love to see what I could do with this + DK2 + UE4. As of 4.1.1, much of the UE4 DK2 issues have been squared away.
What’s the latency on the dragonfly? That seems to be a big issue the VR folks struggle with.
Since it’s still (very much) a prototype, we don’t have additional details about Dragonfly yet — stay tuned!
I love this. I wrote concepts like this about a decade ago and now it’s here! 🙂
Why isn’t there a small cable in the package for direct connection to the DK2, or wouldn’t it work with the USB port on the DK2 (which is also powered)?
You can use the shorter cable originally included with the Leap Motion Controller to connect with the DK2’s built-in port.
right, didn’t think of that as it’s still in the box.. on thing though, it’s a white cable, so I guess you’re all Apple users 😉
Another thing though is why using the weird shaped USB3 connector on the LM end though, it’s much harder to find cables with that connector..
Fair question; we used the USB3 connector for possible future scalability 😀
An update: after hearing reports of tracking issues, we found that the latest Oculus firmware release limits the USB speed available through the DK2 port. For the time being, we’re recommending that you use the free cable extender (bundled with your mount) to connect the Leap directly to your computer; we’ll post more updates as they become available.
IIRC, the usb3 connector works with usb2 cables
Good luck finding an USB2 cable that has the USB3 connector (micro-B) which needs to go into the Leap Motion, you won’t find it… But there are those USB3 cables with micro-B connector, but they are much harder to find, for instance I didn’t find any in our local media/electronic store which has a lot of cables.. But online you can find them…
As my LEAP motion is collecting dust in a box, this might finally be the application I’ve been waiting for…
Are you going to actually develop something which works in tandem with the Rift DK1/DK2 to show this off ?
We’ve built several demos and examples which are available from our examples gallery: https://developer.leapmotion.com/gallery/tags/vr
Great stuff!
Welcome to the Johnny Mnemonic era of internet = http://www.youtube.com/watch?v=8p0jmewhXeU
Whoa.
[…] with standard desktop mode, Collider also includes an Oculus Rift mode using our experimental VR tracking, which is rapidly evolving in developer beta. This means you can always see your hands in front of […]
… [Trackback]
[…] There you will find 30540 more Infos: blog.leapmotion.com/leap-motion-sets-a-course-for-vr/ […]
[…] technology in everything from automotive and medicine to robotics and AR/VR. While our head-mounted VR tracking beta is just one small waypoint along the road to truly immersive virtual reality, our device already […]
[…] features specialized 720p high-framerate image sensors and 16 high-powered infrared LEDs. Like its cousin Dragonfly, Meadowhawk is designed to be embedded by […]
[…] Leap motion sets a course for vr […]
[…] Kickstarter, with an initial aim toward desktop applications. In August, 2014, the company ‘Set Course‘ for virtual reality, releasing a ‘VR Mount’ which conveniently attaches the […]
[…] Kickstarter, with an initial aim toward desktop applications. In August, 2014, the company ‘Set Course‘ for virtual reality, releasing a ‘VR Mount’ which conveniently attaches the […]
[…] Source: Leap Motion […]
[…] Source: Leap Motion […]
[…] Source: Leap Motion […]
[…] Source: Leap Motion […]
[…] Source: Leap Motion […]
[…] Source: Leap Motion […]
[…] Source: Leap Motion […]
[…] Source: Leap Motion […]
[…] Source: Leap Motion […]
[…] Source: Leap Motion […]
[…] announcing the VR Developer Mount, we’ve heard lots of great questions from developers who are experimenting with virtual and […]
Imagine if you could daisy chain these things and have them work together
HELP!….When will the android SDK will be available. I want to integrate leap motion with Google Cardboard. Any timeline?
No timeline for public beta as of yet; this thread has the latest: https://community.leapmotion.com/t/android-sdk-hardware-requirements/2845
You can definitely apply for the alpha! https://docs.google.com/forms/d/1R2sqQ_cdiMPC7u-RA5bP98hlvdJmNFdVb73M313J1Ss/viewform