By learning how to play an instrument, musicians have the power to channel beauty and emotion through their hands. This makes music theory a ripe playground for 3D motion control experiments. If learning to play a physical instrument is a matter of learning how that object works and building muscle memory, why can’t learning chord progressions happen the same way – but in the air?

Joining Volcano Salvation for the CHI 2014 Student Games Competition, Team Chorlody arrived at the conference with a simple goal – to gamify music education. The result was a rhythm action game that helps aspiring musicians wrap their heads around chords in a natural, intuitive way. Team leader and developer Yang Liu and head designer Ni Yan both play piano and hail from the University of Michigan School of Information. Their third collaborator, developer and audio producer Dili Hu, is a graduate from Michigan’s Computer Engineering Department.

Music has always been my passion. I played the piano for more than 10 years, so I came across the idea of making a music learning game. Music is often difficult for new learners. They don’t have access to instruments, or resources to take lessons for music theory, so they have to invest time and energy to experience the virtuosos of the world for themselves.

We had the idea to make a game to help the novice learners to help them experience the combination of different chords through playing a game. It makes it much easier for them to just move their fingers to control the chords and music piece.

—Ni Yan

The team built Chorlody in OpenGL, using our C++ API and documentation along the way. At first, Ni visualized a core game mechanic wherein chords flew toward you, and you had to grab and connect them in the air. After hammering out a few prototypes, however, the team encountered roadblocks when attempting to tackle the uncanny valley of a reliable Z-axis in the context of a traditional desktop environment.

After a few more iterations, Ni landed on the idea of visualizing the game within a chromatic scale – a circle of notes where each chord is signified by the three points of a triangle.

The team aims to build on the addictive sonic mechanics that took legions of gamers by storm in Guitar Hero and Dance Dance Revolution. Rather than placing players on a rail of a particular track lifted from Top 40 charts, however, the Chorlody experience empowers the player to change the accompaniment.

“The players are introduced to the concept of a root note, and the accompanying notes in the chord, early in the game,” Dili explains, as the idea is to make a scalable experience.“We intend to design both for beginners and for advanced learners and musicians. For beginners, they can play the easy levels – they can just touch one root note to play the whole chord. But for more advanced musicians, we want to design more difficult levels where you’ll need to nail each note in the chord quickly and correctly in rapid succession.”

Want to give Chorlody a whirl with your Leap Motion Controller at home? Download the build for PC and let the team know what you think of their work!