Important comment on the the Pioneer award and my app

Important comment on the the Pioneer award and my app

Robert Oschler's picture

I tried to add a shorter version of this comment to the announcement of the Pioneer award winners.  But I kept getting a message about my post triggering the spam filter so I'm posting it here as a new message.  It's quite a bit longer than the original because this is really bothering me.

I was fine with not winning the top tier prize awards.  I recognize that my app was not "pretty" and I'm sure that was one of the minimum threshold requirements for the top tier prizes.  However that changed with the announcement of the Pioneer awards, especially since the declared criteria for the the Pioneer award is as follows:

"The Checkpoint Award recognizes developer profiles which showed Clarity of Idea, an overall grasp of Perceptual Computing and a clear understanding of Intel's SDK."

My app plays Rock, Paper, Scissors with you while a real robotic head talks to you as you play.  In addition there was a live chat mode where you could talk with the head while it responded to you with funny GLADoS quips in response to your voice queries.  (GLADoS is the hostile AI presence in the game Portal).  To accomplish this the following technologies were integrated and implemented in the app:

  • Nuance Text To Speech that spoke to you while training you how to play the game
  • The open source Festival Text To Speech engine that was used for the voice of GLADoS
  • Nuance speech recognition to recognize your speech during live chat mode
  • PercSDK machine vision to detect when you shook your head NO or nodded your head YES
  • Gesture recognition to detect the Rock, Paper, Scissors, and Palm gestures.  This included custom pattern recognition work because those gestures are not detected by the PercSDK.
  • Chatbot technology used during live chat mode

In addition, the integration of the animatronic robot head was a unique and advanced technology implementation in itself.  The app is capable of being "fed" any block of text, and the app chooses appropriate facial expressions, eye movements, and jaw animations automatically to animate the head while the Festival TTS engine waveform plays.  The reason I used the Festival TTS engine for the robot head was the detailed timing files the Festival engine produces.  These timing files contain the time offsets for each phoneme into the synthesized waveform, thus allowing my app to tightly animate the jaw and sync it to the waveform.  This is why the Nuance TTS engine was relegated to the training mode since that engine does not produce those timing files, however it was indeed used in the app.  It was a lucky coincidence that the Festival TTS also does a fairly good and funny "impression" of the GLADoS AI voice.

I feel insulted that my app doesn't meet the criteria indicated for the Pioneer award.  How does an app that uses the PercSDK speech recognition, text to speech, machine vision, custom gesture recognition (depth camera usage and finger geometry usage), fail to show that I have clear understanding of the Intel PercSDK or that I don't understand Perceptual Computing?  What PercSDK core tech did I leave out?  And how does an app strictly focused on providing an advanced robotic head that talks to you while playing Rock, Paper, Scissors fail to be perceived as a clear idea?  On top of all that is a custom robotic rig with 3 cooperating modules that independently control eye movement, face motors, and jaw movements to intelligently synthesize lip-sync'ed facial expressions to dynamically synthesized speech waveforms, with all servos independently coordinated by the app in real time.  How does this not meet the criteria for a Pioneer?  As the old joke says, I certainly feel like one now because they "get all the arrows in the back".

So my reward for months of effort on my admittedly "ugly" app is Intel telling me that I don't know their SDK and my app didn't show any innovation or clarity for the ideas it implemented.  I guess it's time to download the Kinect SDK.

-- roschler

 

4 posts / 0 new
Last post
For more complete information about compiler optimizations, see our Optimization Notice.
Robert Oschler's picture

Almost forgot, I also added code that calculates the length of a TTS utterance and turns down the mic volume automatically using the Windows API while the TTS waveform is playing to avoid unwanted triggering of the Nuance speech recognition engine, turning the mic back up when it's done.  Also, the machine vision tech to detect headshakes as mentioned above used the face detection SDK as its base.  Headshake detection is not part of the PercSDK and therefore is another example of an advance technique in my app that shows expertise with the PercSDK.   As a final note, I did a brief scan of some of the other winning apps and saw at least one that used only a single PercSDK core technology.  Now I get to spend my holiday bewildered and astonished.

-- roschler

Omar Barlas's picture

Robert, I totally agree with you, there were couple of Apps in the contest which were much better than what were selected as winners, and I also do believe that couple of apps which must have met all eligibility criteria , but they did not.

Same is the case with my App, Illuminator App http://youtu.be/Wuh8NqtG0D4  I was also not expecting a failure to meet their minimum criteria and hoping that Intel will share their scoring and ranking for each of the app in the contest.

Robert Oschler's picture

Omar,

"Robert, I totally agree with you, there were couple of Apps in the contest which were much better than what were selected as winners, and I also do believe that couple of apps which must have met all eligibility criteria , but they did not."

I would like to make an important distinction for the sake of this thread.  I specifically did not broach the topic of the selection of the main contest winners (top tier prizes) because one of the most difficult aspects of participating in a contest is the judging.  Despite a contest's published criteria for judging in the end it's always a highly subjective experience.  But the Pioneer award changed all this because of the clearly delineated criteria of giving the award to apps that showed the author's command of the PercSDK, Perceptual Computing, and whether or not the idea was clear.

I watched the video for your app.  Whether or not I liked the app is irrelevant (I did like it, but it's still irrelevant).  But I did see the successful implementation of at least 3 of the core PercSDK technologies: Text To Speech, Speech Recognition, and Gesture Recognition.  It was also immediately clear it was a "personal assistant" style of app.  By definition that puts you and your app in the Pioneer class.

The Pioneer award by Intel's very own words was created to reward developers' that put in the effort  to understand the PercSDK and successfully implement an idea.  By doing this they created a clear message both to those who won and those who didn't win that award.  To those who didn't, it's a very bad message as I stated in my original post.  Worse in our case it's a false one.

To recapitulate, the top tier awards were highly subjective and that's always a difficult process for both participants and judges.  But the Pioneer award was clearly a pass/fail grade, an objective yardstick of a developer's skill with the PercSDK, and I believe they have left people out of that award class in great error.  I wanted to develop a consulting path for myself to develop PercSDK apps for clients.  What happens when I tell a prospective client about my advanced perceptual robotics app that uses virtually the whole SDK, and they ask how I did in the widely publicized PercSDK contest?  Right now I have to tell them, if they don't find out by themselves, that Intel felt I didn't understand or know how to successfully implement the SDK and that my app idea lacked clarity.  Obviously I've lost credibility completely as a perceptual computing developer because the creator of the PercSDK, Intel itself, doesn't believe my app warrants the Pioneer award.

-- roschler

 

Login to leave a comment.