How Can a Spacecraft Use AI?

Photo Courtesy of NASA

Pose Detection (Part 3 of 3)

In our previous post, we implemented the HoG method to detect people in frames. HoG works fine when people are upright, however it does not detect people well in clusters or in unusual positions. In this tutorial, you will learn how to implement a deep neural network designed to estimate the posture of multiple people in an image using the Intel® AI DevCloud. 

HoG and SVM are a pretty powerful combination, but we can improve our detection with a pretrained neural network. We will use an open source pose estimator developed by researchers at Carnegie Mellon University and implemented for TensorFlow* and CPU by Ildoo Kim. This software enables real-time pose estimation on CPU devices thanks to Kim's selection of MobileNet, a neural network designed for edge and mobile computing. Read more about the original implementation on GitHub.


AI DevCloud for Post Estimation
For this tutorial, we will use the Intel AI DevCloud, but feel free to use any cloud service like AWS or Google Cloud for the challenge. The Intel AI DevCloud allows us to generate data much faster than with only a personal computer. 

For users new to the Intel AI DevCloud and submitting jobs to computer clusters, take a look at the "Welcome" notebook after signing up: Browse to the following URL, replacing <userID> with your userID: https://hub.colfaxresearch.com/user/<userID>/notebooks/Welcome.ipynb.

The main techniques to master here are:

  • Submitting a job with `qsub`
  • Checking the job status with `qstat`
  • Outputting the results with iPython magic commands

Open the notebook on the DevCloud by opening a cell at the bottom of the notebook and cloning the repository with the magic command `!git clone --depth=1 --branch=demo https://github.com/JustinShenk/tf-pose-estimation:

 

Jupyter Notebook on the DevCloud

and browse to the notebook at 

https://hub.colfaxresearch.com/user/<userID>/notebooks/tf-pose-estimation/pose-estimation.ipynb

The first cell uses the magic command "%%writefile" to save the cell contents as "pose_job.txt". Important: Remember to add a trailing newline to the cell or else the job won't run.

write file

We can call system commands in the Jupyter* Notebook by preceding a line with an exclamation mark. `!ls` for example calls the `ls` system command. And we can return the output to a variable. This allows us to check the status of the job submitted with `qsub`.

q-sub

Since we want to be notified when the job is finished (a fraction of a second per image), we can use a helper function `wait_for`:

wait for

Calling `wait_for(job_id)` prints a period every 2 seconds until the job is finished:

job files

We can then examine the output of our job (stderr, followed by stdout) and see that an image was saved at `output/test.png`:

Github example

We can plot the contents of `output` with `plot_dir` and see our image:

Pose detection

To estimate the poses for several images in a directory, replace the python src/run.py command in pose_job.txt with python src/run_directory.py --folder=<folder>.
Note: Algorithms like deep neural networks can be optimized for many CPUs, FPGAs, IPUs, and VPUs (ie, Intel® Movidius™ Neural Compute Stick) on a variety of platforms by using the new Intel® OpenVINO™ toolkit.
 
Challenges
  • Cue lights or music in real-time with hand position (Basic)
  • Cue lights or music in real-time with gestures (Intermediate)
  • Analyze body posture over time to suggest exercise (Advanced)

​Further Challenges

  • Control music in the room by calculating optical flow
  • Optimize the real-time posture detection models to run on the Intel® Movidius™ Neural Compute Stick  or with an Up Squared* board
  • Track the number of people in each frame of a webcam capture and use it to activate LED lights

Read the previous two blogs in this series: 

Computer Vision Introduction and Face Detection (Part One)

Body Detection (Part Two)


Justin ShenkAbout the Author

Justin Shenk

Master student and research assistant, University of Osnabrueck, Germany

Justin is an AI Master Thesis worker at Peltarion researching deep learning model introspection. He develops AI software as an Intel Software Innovator and demos his projects at Intel’s booths at NIPS, ICML, and CVPR.. He previously worked as a neuroscientist in the US.

 

For more complete information about compiler optimizations, see our Optimization Notice.