Multi-Camera Detection of Social Distancing Reference Implementation

Overview

Social distancing and face masks are one of the most effective nonpharmaceutical way to prevent the spread of disease. This reference implementation gives a solution to prevent the spread of disease by using computer vision inference in the Intel® Distribution of OpenVINO™ toolkit to measure distance between people and store data to InfluxDB. This data can be visualized on a Grafana dashboard.

The reference implementation and the software listed in table 1 are installed when selected as part of the Edge Insights for Vision package. If you have not installed that package yet, you can download it here and then follow the installation instructions Edge Insights for Vision.

Table 1

Time to Complete 50 - 70 minutes
Programming Language Python* 3.6
Software Intel® Distribution of OpenVINO™ toolkit 2021 Release

Target System Requirements

  • Ubuntu* 18.04.3 LTS/CentOS 7.x
  • 6th to 10th Generation Intel® Core™ processors with Intel® Iris® Plus Graphics or Intel® HD Graphics
  • USB webcam

Recommended Development Kits

How It Works

A multi-camera surveillance solution demonstrates an end-to-end analytics pipeline to detect people and calculates social distance between people from multiple input feeds. Frames are transformed, scaled and normalized into BGR images which can be fed to the Intel® Distribution of OpenVINO™ toolkit for inference. The steps below are performed for the inference.

  • Apply Intel's person detection model, i.e., person-detection-retail-0013 to detect people from all the video streams.

  • Compute Euclidean distance between all the people from the above step.

  • Based on above measurements, check whether any people are violating N pixels apart.

  • Store total violations count of social distancing data in InfluxDB.

  • Visualize the stored data of InfluxDB on Grafana dashboard.

Figure 1: Architecture Diagram

Get Started

Step 1: Install the Reference Implementation

NOTE: If you have installed Edge Insights for Vision, the Reference Implementation was installed with the package and it is already available in the target system at the below where <version> is the Edge Insights version downloaded. (It can be found in the readme file.)

<install_directory>/edge_insights_vision/Edge_Insights_for_Vision_<version>/Reference_Implementation__MultiCamera_Detection_of_Social_Distancing/mcss-covid19/

In this case, you can skip the following installation steps and proceed to Step 2: Test with Input Video

Follow the steps below to install the Reference Implementation: 

  1. Open a new terminal, go to the downloaded folder and unzip the downloaded RI package:  
    unzip multi_camera_detection_of_social_distancing.zip

     

  2.  Go to multi_camera_detection_of_social_distancing/ directory:
    cd multi_camera_detection_of_social_distancing
      
  3. Change permission of the executable edgesoftware file: 
    chmod 755 edgesoftware
     
  4. Run the command below to install the Reference Implementation: 
    ./edgesoftware install

     

  5. During the installation, you will be prompted for the Product Key. The Product Key is contained in the email you received from Intel confirming your download. 
    Figure 2: Product Key
  6. When the installation is complete, you see the message “Installation of package complete” and the installation status for each module. 
     
    Figure 3: Installation
  7. Go to the working directory: 
    cd  MultiCamera_Detection_of_Social_Distancing_Reference_Implementation_<version>/MultiCamera_Detection_of_Social_Distancing/mcss-covid19/

     

NOTE:<version> is OpenVINO version 

Step 2: Test with Input Video

The application works better with input feed in which cameras are placed at eye level angle.

Please download sample video at 1280x720 resolution and place it in the “resources” directory. (Data set subject to this license. The terms and conditions of the dataset license apply. Intel® does not grant any rights to the data files.)

To use any other video, specify the path in run.sh file inside the application directory. For example, if two of the test video files are in resources folder, then make changes to the following variables INPUT1 and MIN_SOCIAL_DIST1 for the test video.

INPUT1="${PWD}/../resources/<name_of_video_file>.mp4"
MIN_SOCIAL_DIST1=<appropriate_minimun_social_distance_for_input1>

Step 2: Initialize Environment Variables

Run the command below to initialize Intel® Distribution of OpenVINO™ toolkit environmental variables:

source /opt/intel/openvino_2021/bin/setupvars.sh

Step 3: Test with Custom Input Video (Optional)

To use any other video, specify the path in run.sh file inside application directory. For example, if two of the test video files are in resources folder, then make changes to the following variables INPUT1 and MIN_SOCIAL_DIST1 for the test video. 

INPUT1="${PWD}/../resources/<name_of_video_file>.mp4"
MIN_SOCIAL_DIST1=<appropriate_minimun_social_distance_for_input1>

Step 4: Test with USB Camera (Optional)

To test with a USB camera, specify the camera index in the run.sh file. On Ubuntu*, to list all available video devices, run the following command:

ls /dev/video*

For example, if the output of the command is /dev/video0, then make changes to the following variables such as INPUT1 and MIN_SOCIAL_DIST1 in the run.sh file inside application folder.

INPUT1=/dev/video0
MIN_DIST1=<appropriate_minimun_social_distance_for_input1>

 

Run the Application

Run the application on CPU

  1. Go to application directory:
    cd application

     

  2. Inside the run.sh file, change the following parameters:
    PERSON_DETECTOR=”${PWD}/../intel/person-detection-retail-0013/FP16/person-detection-retail-0013.xml”
    DEVICE1="CPU"

     

  3. Run the script:
    chmod +x run.sh
    ./run.sh
    

    NOTE: Application parameters can be changed as per the requirements in run.sh.

Run the application on GPU

  1. Go to application directory:
    cd application 

     

  2. Inside the run.sh file, change the following parameters:
    PERSON_DETECTOR=”${PWD}/../intel/person-detection-retail-0013/FP16/person-detection-retail-0013.xml”
    DEVICE1="GPU"      

     

  3. Change the permission of run.sh file and run the script:
    chmod +x run.sh
    ./run.sh

     

Run the application on HDDL

  1. Go to application directory:
    cd application

     

  2. Inside the run.sh file, change the following parameters:
    PERSON_DETECTOR=”${PWD}/../intel/person-detection-retail-0013/FP16/person-detection-retail-0013.xml”
    DEVICE1="HDDL" 

     

  3. Change the permission of run.sh file and run the script:
    chmod +x run.sh
    ./run.sh

     

Figure 4: Application running on 4 channels

Data Visualization on Grafana

1. Navigate to localhost:3000 on your browser.

NOTE: If browser shows Unable to connect, then make sure Grafana service status is active – ‘sudo service grafana-server status’. If service is not active, then start the service by running the command ‘sudo service grafana-server start’ in terminal.

2. Login with user as admin and password as admin.

3. Go to Configuration and select Data Sources.

4. Click + Add data source, select InfluxDB, and provide the following details:

  • Name: Mcss Covid
  • URL: http://localhost:8086
  • Auth: Choose skip TLS Auth
  • InfluxDB details:
    • Database: McssCovid
    • HTTPMethod: GET

5. Click Save and Test.

Figure 5: Data Source Creation

 

6. Click + icon on the left side of the window, then select Import.

7. Choose Upload.json File and import "mcss-covid19/resources/multi_cam.json" file.

8. Click on import.

Figure 6: Import Dashboard

 

9. Click on Multi Camera Covid-19 Solution dashboard to view real-time violation data.

Figure 7: Final Dashboard

 

Summary and Next Steps

This application successfully leverages Intel® Distribution of OpenVINO™ toolkit plugins for detecting and measuring distance between the people and storing data to InfluxDB. It can be extended further to provide support for feed from network stream (RTSP camera) and the algorithm can be optimized for better performance.

Learn More

To continue learning, see the following guides and software resources:

Product and Performance Information

1

Performance varies by use, configuration and other factors. Learn more at www.Intel.com/PerformanceIndex.