Intel® Perceptual Computing Showcase


JOY-Gesture Dynamic Phrase Synthesizer

Lin Yunfan

Along with the development of Music industry, people are expecting simultaneous interpretation of music both visual and auditory, controlled by the body language, to maximize the fun of performance by integrated sensorial experience. We’ve come out of an exciting concept of “Perceptual Music & Visual Playing”(PMVP), together with the first ever music instrument- JOY! JOY is the first ever Gesture Dynamic Phrase Synthesizer that is designed through the adapting the PMVP(Perceptual Music & Visual Playing) concept. The performer will be able to take control of different music mix, solo, effect & pitch of different music loop and visual effects by changing the gestures, distance, depth and altitude of his /hers hands in front of the Intel Perceptual Camera. It is an instant yet simultaneous audio & visual experience, connecting changes and visual perception in music through simple control of the performer.

App Specifications

  • The apps is developed base on Intel perceptual SDK with Unity 3D engine
  • No Middleware is needed.
  • 3D animation was built by MAYA and Unity3D engine while sound tracks was edited base on Protools HD and Logic Pro. The UI was developed by NGUI plug-in with Unity 3D engine
  • All music material were from Library music with licensed
  • There are buttons from UI for "song selection" "About" "Reset" "Introduction" and "Exit".