Driver Management Reference Implementation

Overview

The Driver Management Reference Implementation is designed to detect and track driver behavior and driver actions. Driver behavior, such as drowsiness, distraction, eyes off the road, yawns and blinks, is tracked to keep metrics that make sure the driver is in prime driving condition. Driver actions help track metrics such as talking on a phone, texting, drinking/eating, etc. Using the metrics from driver action and behavior can prevent dangerous situations.

Table 1
Time to Complete approximately 30 - 40 minutes
Programming Language C++, Python
Software Intel® Distribution of OpenVINO™ toolkit 2020 Release

 

Target System Requirements

  • Ubuntu* 18.04.3 LTS
  • 6th to 10th Generation Intel® core™ processors with Iris® Pro graphics or Intel® HD Graphics
  • USB webcam

Recommended Development Kits

How It Works

The application uses the inference engine included in the Intel® Distribution of OpenVINO™ toolkit and is designed to detect and track driver behavior and driver actions to ensure safe driving. Using deep learning models, camera video streams, and analytics running on In-Vehicle-Computers, it detects driver’s drowsiness states and distraction behaviors, and provides real time alerts to driver, and analytics per driver, truck and route over time.

Figure 1: Architecture Diagram

 

Get Started

Step 1: Install the Reference Implementation

Follow the steps below to install the Reference Implementation.

NOTE: If the host system already has Docker images and containers, you might encounter errors while building the reference implementation packages. If you do encounter errors, refer to the Troubleshooting section at the end of this document before starting the reference implementation installation.

1. Open a new terminal, go to the downloaded folder and unzip the downloaded RI package.

unzip driver_management.zip

 

​2. Go theto driver_management/ directory.

cd driver_management /

 

​3. Change permission of the executable edgesoftware file.

chmod 755 edgesoftware

 

4. Run the command below to install the Reference Implementation.

./edgesoftware install

 

5. During the installation, you will be prompted for the Product Key. The Product Key is contained in the email you received from Intel confirming your download.

6. When the installation is complete, you see the message “Installation of package complete” and the installation status for each module.

 

NOTE: If you encounter any issues, please refer to the Troubleshooting section at the end of this document. Installation failure logs will be available at path - /var/log/esb-cli/Driver_Management_<version>/<Component_name>/install.log

7. Go to the working directory:

cd Driver_Management_<version>/Driver_Management/OpenVino-Driver-Management

 

NOTE: <version> is the Intel® Distribution of OpenVINO™ toolkit version downloaded.

Step 2: Install the Application

1. Build the Docker images for whole application:

sudo make docker-build

2. Run the Driver management container:

sudo make docker-run

NOTE: If you encounter any issues, please refer to the Troubleshooting section at the end of this document.

Run the Application

Start the Docker

sudo make docker-start

NOTE: If network calls fail during image build on a corporate network, e.g,, ‘apt-get update’ error, please refer to the Troubleshooting section at the end of this document.

Download Models

bash /app/DriverBehavior/scripts/dlposemodel.sh

Run the Application

make run

 

NOTE: If you encounter any issues, please refer to the Troubleshooting section at the end of this document.

 

Open the Web UI

Open 127.0.0.1:5000 on your web browser

 

 

Load AWS* Certificates

1. Select Configuration from the sidebar.

2. Click Browse and select the following certificates: certificate.pem.crt, private.pem.key and AmazonRootCA1.pem.

3. When the certificates have been added, click on Upload Certificates.

Access Driver’s Behavior Cloud Dashboard

1. Go to sidebar and select Dashboard and click on log in.

2. Sign in at AWS services.

  • Username: intel
  • Password: Intel2019_

3. In “Visualize and Explore Data,” select Dashboard.

 

4. Select DriverManagementDashboard.

5. You will see the dashboard below:

 

Add a New Driver in the Web UI

1. Select Add New Driver from the sidebar.
2. Add the Driver's Name.
3. Upload the Driver's Picture according to the suggested image format (Optional).
4. Click on Create New Driver.

Run Driver Management


1. Select Run driver Management from the sidebar.
2. Turn off camera input and add input videos for driver behavior and driver action (sample input videos can be found in UI/videos directory).
3. Select Send Data to AWS.
4. Click on Run Driver Management.

5. Output for driver behavior and action recognition will appear as below:


 

6. You will see the Kibana dashboard:

 

 

 

Summary and Next Steps

This application successfully implements Intel® Distribution of OpenVINO™ toolkit plugins for detecting and tracking driver behavior and driver actions. It can be extended further to provide support for feed from network stream (RTSP camera), and the algorithm can be optimized for better performance.

As a next step, try one of the following:

  • Use deep learning models and driver facing camera video streams to detect driver’s drowsiness, distraction state and behaviors to provide real time alerts to driver. This reference implementation uses Intel® Distribution of OpenVINO™ toolkit Open Model Zoo pre-trained models and 3rd party models, but you can extend it to use your own models.

  • Driver Authorization uses deep learning models and pre-registered photos to detect if the driver is an authorized person. Extend this to include other functions such as alerting if an unknown driver is in the cabin.

  • Driver Management Dashboard uses edge-to-cloud data connection to show the driver manager the current status of the driver. AWS IoT Core* is the cloud connector used in this implementation. Choose your own cloud service providers and implement your own dashboard to consume the edge data generated from the In-Vehicle Computer.

Learn More

To continue your learning, see the following guides and software resources:

 

Known issues

  • Driver action and Kibana dashboard work only with the open network.

 

Troubleshooting

Installation Failure

If host system already has Docker images and its containers running, you will have issues during the RI installation. You must stop/force stop existing containers and Images.

  • To remove all stopped containers, dangling images, and unused networks:

sudo docker system prune --volumes

  • To stop Docker containers:

sudo docker stop $(sudo docker ps -aq)

  • To remove Docker containers:

sudo docker rm $(sudo docker ps -aq)

 

  • To remove all Docker images:

sudo docker rmi -f $(sudo docker images -aq)

Docker Image Build Failure

If Docker image build on corporate network fails, follow the steps below.

1. Get DNS server using the command:

nmcli dev show | grep 'IP4.DNS'

2. Configure Docker to use the server. Paste the line below in   /etc/docker/daemon.json file:

{
    "dns": ["<dns-server-from-above-command>"]
}

3. Restart Docker:

sudo systemctl daemon-reload && sudo systemctl restart docker

Product and Performance Information

1

Performance varies by use, configuration and other factors. Learn more at www.Intel.com/PerformanceIndex.

Give Feedback