Intel® Perceptual Computing Showcase

LEGO Self-Driving Car

Martin Wojtczyk
United States

This application combines the Perceptual Computing SDK and the depth camera with a LEGO Mindstorms set to create an autonomous mobile vehicle that explores its environment, avoids obstacles and creates a map. The goal of this project is to engage kids and teenagers in science and technology by using LEGO and Computers tools they know and love. When a person is detected in front of the camera by the PerC SDK, the robotic Lego car awakes, opens it eyes, and greets the person with a welcome message. The robot drives around the room, explores the area, avoids tables, chairs, and stairs and simultaneously generates a map, which will be depicted on the screen. This is just a starting point to get kids and teenagers involved in robotics, image processing, speech recognition, speech synthesis and Human-Robot-Interfaces.

Tools and Technology

  • The code was implemented in C++ with the MS Visual Studio 2010 IDE. The PerC SDK and the depth camera provide speech recognition in Command & Control mode, a face detector and face recognition as well as speech synthesis for user engagement. The depth stream allows for obstacle avoidance. OpenCV was utilized for several image manipulation and conversion steps and to analyze the visible scene through the depth sensor. We use the programming environments NBX/NXC to interface with the Lego EV3 controller. Many C++ classes of the Qt Toolkit have been used to implement the Human-Robot-Interface, the user interface, file access, thread to thread communication and animations. Qt Designer was utilized to create the application's form layouts. MS Paint was used to draw the graphical assets. The GNU image manipulation program (GIMP) was utilized to fine-tune and edit most pictures and graphical items. CMake is used to find the necessary dependencies (Qt, OpenCV, PerC SDK) and to create an initial MS Visual Studio 2010 project file. Find- and configuration scripts have been implemented for CMake where necessary to find the Intel PerC SDK headers and libraries and to set the proper compiler flags.

Development Tools

  • OpenCV
  • Qt framework
  • C++
  • MS Visual Studio
  • CMake.