Archived - How to Build the Intel® RealSense™ Spatial Awareness Wearable

The Intel® RealSense™ SDK has been discontinued. No ongoing support or updates will be available.

We’re going to assume that you are looking to build the system exactly as we built the first version of the Intel® RealSense™ Spatial Awareness Wearable (SAW). This is by no means a requirement. We provide all the pieces so that developers and researchers can explore their own implementations using our system as a reference. This tutorial will walk you through how to realize the system exactly as we did for our first fully functional version.

Step 1: Gathering the Required Pieces

The first step is to gather all the components necessary to build the system. We have a list of these components on the Getting Started page with links to the parts that we used. You can choose your own brands or varieties of these components, but these are what we used for our prototype and what we know works in this configuration.

Hardware

You will need:

  • 1 - Laptop or convertible PC, powered by Intel® processors, running Windows* 8.1 with at least one USB 3 port (We strongly recommend a fanless system.)
  • 8 - Particle Core* or Particle Photon* microcontrollers
  • 8 - Adafruit* Micro Lipo with MicroUSB Jack - USB Li-Ion/LiPoly chargers
  • 8 - Sparkfun* Polymer Lithium Ion batteries - 850 mAh
  • 8 - Vibration motors
  • 1 - Latching switch–push button (only for one actuator)
  • 1 - Portable, battery-powered wireless router
  • 1 - Intel RealSense Depth Camera (R200)

Step 2: Setting Up Your Development Environment

Software

In this section, we’ll walk you through setting up your development environment. The software list includes the tools that we used to build as well as libraries to which we link. The requirements are as follows:

  • Windows 8.1
  • Visual Studio* 2012
  • Cinder 0.8.6 for Windows and Visual Studio 2012
  • OpenCV 2.4.9
  • Intel RealSense Depth Camera Manager (DCM) for R200 camera
  • Intel RealSense SDK (v 4.0 or above)

We’ll assume that you already have an Intel powered machine running Windows 8.1 with Visual Studio 2012 installed. Let’s set-up the rest of your development environment.

OpenCV

We use the fantastic toolbox of OpenCV for some image processing tools. OpenCV (Open Source Computer Vision Library) is an open-source computer vision and machine-learning software library. OpenCV was built to provide a common infrastructure for computer vision applications and to accelerate the use of machine perception in the commercial products. Being a BSD-licensed product, OpenCV makes it easy for businesses to utilize and modify the code. We use OpenCV version 2.4.9 for Windows. You can download it from here.

After downloading the package, open the self-extracting file, and then choose your installation destination. Next, you’ll set-up your environment variables for OpenCV. Sarang walks you through that process in this video.

Cinder

Our project uses Cinder, a community-developed, free and open source library for professional-quality creative coding in C++. Cinder gives us a framework that works very well for setting up our system processing the data from the camera in a frame by frame way. You can download Cinder from here. We’re using version 0.8.6 for Windows and Visual Studio 2012.

After downloading the zip file, unzip it to your target directory (where you want Cinder to live on your system). Next, set up your environment variables for where to find the Cinder libs. This is outlined in the following video.

Intel® RealSense™ Depth Camera Manager (DCM) and SDK for R200

Now we’ll set-up the Intel RealSense components. To do this, you’ll need an R200 camera plugged into a USB3 port on your development machine. If you have any previous versions of the Intel® RealSense™ SDK and DCM installed, you’ll need to uninstall those before beginning this process.

Installing the DCM

Make sure that your R200 camera is plugged into a USB3 port on your development machine. Download the Intel RealSense DCM (R200) from the Intel website. You can get that here.

Attention: There are two different DCMs available through the website. Be sure to download the DCM for the R200 camera.

After you’ve downloaded the installer for the DCM, run the installer. If the installer fails to detect your camera, try unplugging and re-plugging the R200 camera into a USB3 port, then re-run the installer.

Installing the Intel® RealSense™ SDK

Once this is complete, you’re ready to install the Intel RealSense SDK, which is available here. Run this installer and you should have everything that you need to start building the main software application. You will have the option of installing the entire SDK or adding or subtracting components. Watch the video below for a walk through of this process including selecting components.

Step 3: Building the Actuator Feedback Boxes

actuator feedback box

The Actuator Feedback Boxes are the 3D printed boxes containing the Particle Core or Particle Photon boards with attached vibration motors and batteries. We are going to provide two different versions of the actuator boxes, one for Particle Core (our original) and one for Particle Photons (a later version).

For each, there is a standard box that can be built for seven of the motors, and there is a plan for an actuator box with a button that can be used to toggle all the motors on and off without accessing the machine driving the system directly.

The STL and SolidWorks files for the 3D printed boxes are available on the Getting Started page in the Wearable Components Designs zip file. Download the STL files and print the boxes and lids that will house the actuators.

Now, you’re ready to build the feedback motor components. This video will walk you through that process.

Building the Particle* Core [Original] Version

Following are instructions for building the actuators using Particle Core. Refer to the diagram below for locations on the board.

vibration actuator

  1. Desolder all the pins on the Particle Cores and take them out
  2. Connect the battery charger to the Particle Core by connecting the Bat pin to the VIN pin and the GND pin to the GND pin on the Particle Core. Do not connect the battery to the battery charger while soldering or short circuits may occur
  3. Solder the 500mA slot onto the battery charger. This will speed up the charging as it allow more current for charging the battery
  4. Connect the red wire (positive) of the vibration motor to the pin A0 of the Particle Core and the blue wire (Ground) to the GND pin of the Particle Core
  5. If you are making the actuator with the button, solder the middle pin of the latching switch to the D6 pin of the Particle Core. For the other two pins of the latching switch, one can be connected to the 3V3 pin and other can be connected to the GND pin of the Particle Core. Reversing these pin connections flips the On/Off positions for the latching switch. The idea is that when the switch is pressed to the “ON” position, it sends a 3V signal to the D6 pin
  6. Once everything is soldered, you can connect the battery to the battery charger and see if the Particle Core powers on. This can be confirmed by looking at the light on the Particle Core
  7. If you haven’t used a Particle Core before, follow the instructions to learn how to flash the Particle Core

Important tip: Once you have soldered the Particle Core, only power the Particle Core using the battery charger. If the battery has no charge, you can charge the battery or power the core by any USB power source by connecting the USB cable to the micro USB port of the battery charger. If you haven’t soldered the Particle Core, then you can use the micro USB connector on the Particle Core itself to power the core for flashing new code on it

  1. You can find the code for the Particle Core in the package folder. You can use the Cloud API of Particle Core to flash the Cores. Once the Core is flashed, we are disconnecting it from the cloud, so to reflash the Core with any updated code, you have to factory reset the Core to update the code. Once you do a factory reset, you can update the Core as mentioned above. To factory reset the Particle Core, follow the video Spark Core Factory Reset . It is highly recommended to play with the Particle Core before using it for this project so you get an idea of how it works
  2. Once the Core is flashed and the Core is running without any problems, you can 3D print the boxes and the lid from the files given in the Wearable Components Designs zip file provided in the project files on the Getting Started page. Put the circuit into that box. It should be assembled as shown in the picture showing the completed actuator box above. In case of the actuator with the button, you can put the button in the case as well in the slot provided
  3. Put the battery in the box and cover with the lid

Building the Particle Photon [Newer] Version

This is the instruction order for building the actuators using Particle Photon. Refer to the diagram below for locations on the board.

Attention: This version is untested on the original version outlined here. The only expected challenge is that the IP assignment for these devices might differ from the behavior of the Particle Core microcontrollers. We will be releasing a future code version that incorporates an IP assignment fix that renders this a non-issue with following releases.

particle photon

  1. Desolder all the pins on the Particle Photons and take them out
  2. Connect the battery charger to the Particle Photon by connecting the Bat pin to the VIN pin and the GND pin to the GND pin on the Particle Photon. Do not connect the battery to the battery charger while soldering as short circuits might happen
  3. Solder the 500mA slot on the battery charger. This will speed up the charging as it allow more current for charging the battery
  4. Connect the red wire (positive) of the vibration motor to the pin A4 of the Particle Photon and the blue wire (Ground) to the GND pin of the Particle Photon
  5. If you are making the actuator with the button, solder the middle pin of the latching switch to the D6 pin of the Particle Photon. For the other two pins of the latching switch, one can be connected to the 3V3 pin and other can be connected to the GND pin of the Particle Photon. Reversing these pin connections flips the On/Off positions for the latching switch. The idea is that when the switch is pressed to the “ON” position, it sends a 3V signal to the D6 pin
  6. Once everything is soldered, you can connect the battery to the battery charger and see if the Particle Photon powers on. This can be confirmed by looking at the light on the Particle Photon
  7. If you haven’t used a Particle Photon before, follow the instructions on Welcome to Particle Docs to learn how to flash the Particle Photon

Important tip: Once you have soldered the Particle Photon, only power the Particle Photon using the battery charger. If the battery has no charge you can charge the battery or power the Photon by any USB power source by connecting the USB cable to the Micro USB port of the battery charger. If you haven’t soldered the Particle Photon, then you can use the micro USB connector on the Particle Photon itself to power the Photon for flashing new code on it.

  1. You can find the code for the Particle Photon in the package folder. You can use the Cloud API of Particle to flash the Photons. Once the photon is flashed, we are disconnecting it from the cloud, so to reflash the photon with any updated code, you have to factory reset the Photon to update the code on the Particle Photon. Once you do a factory reset, you can update the Photon as mentioned above. To factory reset the Particle Photon, follow the instructions on the website Welcome to Particle Docs It is highly recommended to play with the Particle Photon before using it for this project so you get an idea of how it works
  1. Once the Photon is flashed and the Photon is running without any problems, you can 3D print the boxes and the lid from the files given in the Wearable Components Designs zip file provided in the project files on the Getting Started page. Put the circuit into that box. It should be assembled as shown in the picture showing the completed actuator box above. The image is of a Particle Core but it will be same otherwise. In case of the actuator with the button, you can put the button in the case as well in the slot provided
  2. Put the battery in the box and cover with the lid

Step 4: Setting Up the Portable Router

Every device connecting to the portable router should be assigned a fixed IP address in the router and reserve that IP address on the router for each specific device. This is done because the main camera application running on the laptop runs a TCP server that looks for clients with specific IP addresses for specific vibration actuators.

Similarly, the vibration actuators run a TCP client application that connects to a TCP server with a specific IP address. You will need to go into the router settings and make sure every single vibration actuator and the laptop that runs the main application have reserved IP addresses in the router.

In the sample code for the Particle Core (the WiFi-enabled microcontroller), the Particle Cores connect to a server with a specific address and port number, (IP Address: 198.162.0.102, Port No.: 5555). The laptop needs to be assigned this IP address in the router. This IP address can be changed, but you have to make sure that the IP address in the Particle Core code matches the reserved IP address for the laptop in the router. So if one changes, the other has to change. Similarly, you have to make sure you have fixed IP addresses assigned to all the vibration actuators in the router.

In the main application code, you can see that the IP addresses of the vibration actuators are hard coded.

//Fixed IP's of the clients
//Here you specify the assigned fixed IP's of the Vibration Actuators which you stored in the router
TLIP = "192.168.0.101";  //IP Address for Top Left Vibration Actuator
TCIP = "192.168.0.103";  //IP Address for Top Center Vibration Actuator  
TRIP = "192.168.0.111";  //IP Address for Top Right Vibration Actuator
MLIP = "192.168.0.105";  //IP Address for Middle Left Vibration Actuator
MCIP = "192.168.0.106";  //IP Address for Middle Center Vibration Actuator
MRIP = "192.168.0.107";  //IP Address for Middle Right Vibration Actuator
BLIP = "192.168.0.108";  //IP Address for Bottom Left Vibration Actuator
BRIP = "192.168.0.109";  //IP Address for Bottom Right Vibration Actuator

You should make sure that the IP addresses in the main application code and the reserved ones assigned to the vibration actuators in the router match. To do that, you can either use the reserved IP addresses already assigned to the vibration actuators in the router, and then change it in the main code, or make sure you assign the same IP address in the router as you reserve the IP addresses in the router.

It is very important that the IP addresses for all the devices should be reserved in the router so that every device will get assigned the same IP address when it reconnects to the router each time. If this is not the case, behavior can be unpredictable and motors receive the wrong data for other areas in the field of view.

Step 5: Running the Application

Now we are ready to prepare to build and run the application. First, you’ll need to download the project from the Getting Started page. Once that is finished, unzip the contained folder to your preferred development workspace.

When you open the project, if you followed our instructions in previous videos for setting up the environment variables for Cinder, OpenCV, and Intel RealSense SDK, everything should be configured properly and ready to run. If you did not, you’ll need to make sure that you adjust the include libraries and linker properties appropriately to your own installation.

Open the IRSAW directory and open the project solution file (IRSAW.sln) in the vs2012 directory. Choose whether you want to build in debug or release mode and build the program. Make sure that the Intel RealSense camera (R200) is plugged into a USB3 port on the machine. Run the program and everything should work. The following video illustrates the process.

Last Step: Put it All Together

Now that you’ve constructed the system’s various components, it’s time to put it all together.

Make sure that the Intel RealSense camera (R200) is connected to your dev machine, that the router is on and configured, and that the laptop is connected to the portable Wi-Fi for the system (assuming the configuration is already done for the router, which we covered above).

Follow the steps in this video and you should be up and running!

Для получения подробной информации о возможностях оптимизации компилятора обратитесь к нашему Уведомлению об оптимизации.
Возможность комментирования русскоязычного контента была отключена. Узнать подробнее.