Pose Detection (Part 3 of 3)
In our previous post, we implemented the HoG method to detect people in frames. HoG works fine when people are upright, however it does not detect people well in clusters or in unusual positions. In this tutorial, you will learn how to implement a deep neural network designed to estimate the posture of multiple people in an image using the Intel® AI DevCloud.
HoG and SVM are a pretty powerful combination, but we can improve our detection with a pretrained neural network. We will use an open source pose estimator developed by researchers at Carnegie Mellon University and implemented for TensorFlow* and CPU by Ildoo Kim. This software enables real-time pose estimation on CPU devices thanks to Kim's selection of MobileNet, a neural network designed for edge and mobile computing. Read more about the original implementation on GitHub.
AI DevCloud for Post Estimation
For this tutorial, we will use the Intel AI DevCloud, but feel free to use any cloud service like AWS or Google Cloud for the challenge. The Intel AI DevCloud allows us to generate data much faster than with only a personal computer.
For users new to the Intel AI DevCloud and submitting jobs to computer clusters, take a look at the "Welcome" notebook after signing up: Browse to the following URL, replacing <userID> with your userID: https://hub.colfaxresearch.com/user/<userID>/notebooks/Welcome.ipynb.
The main techniques to master here are:
- Submitting a job with `qsub`
- Checking the job status with `qstat`
- Outputting the results with iPython magic commands
Open the notebook on the DevCloud by opening a cell at the bottom of the notebook and cloning the repository with the magic command `!git clone --depth=1 --branch=demo https://github.com/JustinShenk/tf-pose-estimation:
and browse to the notebook at
The first cell uses the magic command "%%writefile" to save the cell contents as "pose_job.txt". Important: Remember to add a trailing newline to the cell or else the job won't run.
We can call system commands in the Jupyter* Notebook by preceding a line with an exclamation mark. `!ls` for example calls the `ls` system command. And we can return the output to a variable. This allows us to check the status of the job submitted with `qsub`.
Since we want to be notified when the job is finished (a fraction of a second per image), we can use a helper function `wait_for`:
Calling `wait_for(job_id)` prints a period every 2 seconds until the job is finished:
We can then examine the output of our job (stderr, followed by stdout) and see that an image was saved at `output/test.png`:
We can plot the contents of `output` with `plot_dir` and see our image:
- Cue lights or music in real-time with hand position (Basic)
- Cue lights or music in real-time with gestures (Intermediate)
- Analyze body posture over time to suggest exercise (Advanced)
- Control music in the room by calculating optical flow
- Optimize the real-time posture detection models to run on the Intel® Movidius™ Neural Compute Stick or with an Up Squared* board
- Track the number of people in each frame of a webcam capture and use it to activate LED lights
Read the previous two blogs in this series:
About the Author
Master student and research assistant, University of Osnabrueck, Germany
Justin is an AI Master Thesis worker at Peltarion researching deep learning model introspection. He develops AI software as an Intel Software Innovator and demos his projects at Intel’s booths at NIPS, ICML, and CVPR.. He previously worked as a neuroscientist in the US.