What if controlling a robotic arm felt as natural as moving your own arm? Over the last 22 years, Mirror Training President Liz Alessi and her team have been working in software research and development for the military to create reliable, easy-to-use products that can save time and workload. At the LEAP.AXLR8R, they’re using Leap Motion technology to create the Anthropomorphic Augmented Reality Controller (AARC) – a natural user interface that makes it easy to control robotic arms and save lives in the field.
When improvised explosive devices (IEDs) are discovered in the field, the military often uses robotic arms to neutralize the devices. With the current joysticks and buttons interface in the field, it can take hours to find and destroy a single device:
Watching the military robotic controllers struggle under time pressure and combat conditions inspired me to find a better way. We’re using Leap Motion as our 3D tracking tech, as it’s the best for our robotic arm and gripper controller software. Specifically, the spatial and temporal resolution of Leap Motion in absolute coordinates enables our robotic arm and gripper controls to work accurately, as well as enabling our augmented-reality projections of the user’s arm and hand into the robot’s camera view, giving users natural control and immersion in what we call “wearing your robot.”
Mirror Training’s software is designed to have plug-and-play functionality, so that it can be added onto existing and future robotic arm generations. Right now, they’re using off-the-shelf LynxMotion and Dagu devices, alongside robotic arms from RS JPO and TARDEC, to fully test the software. Their next big challenge is further testing the setup to prove its effectiveness in real-world situations, and a second scientific study is in the works for June or July.
Along the way, they’ve taken inspiration from other teams building with Leap Motion technology:
My time at the LEAP.AXLR8R has been exciting. If nothing else, the people (from the mentors to the other inventors) have enriched my life. We came into the AXLR8R with functional prototypes using Kinect, and had begun working with other 3D tracking technologies. We’ve since selected Leap Motion as our “best of breed” 3D tracking solution for our AARC software.
Beyond our primary defense and security applications, we also envision uses with first responders – beginning with HAZMAT and search and rescue – then military medical, hospitals, clinics, and ultimately civilian medical and consumer-level home care.
Arvind Gupta is the founder of the LEAP.AXLR8R and partner at SOSventures. This post originally appeared on the LEAP.AXLR8R blog.