Intel® Edge Software Device Qualification (Intel® ESDQ) for Vision

ID 659375
Updated 12/27/2023
Version 2.1
Public

author-image

By

Overview

Intel® Edge Software Device Qualification (Intel® ESDQ) for Vision package provides customers with the capability to run an Intel provided Vision test suite at the target system, with the goal of enabling partners to test their platform for Intel® Edge Insights for Vision package.  

The information below is specific to the Intel® ESDQ for Vision package. For documentation on the Intel® ESDQ CLI binary, refer to Intel® Edge Software Device Qualification (Intel® ESDQ) Overview. 

Select Configure & Download to download Intel® ESDQ for Vision and refer to Get Started for installation steps.  

Configure & Download

Target System Requirements

  • One of the following processors:
    • 12th or 13th generation Intel® Core™ processors
    • Intel® Processor N-series
    • Intel® Core™ i3 Processor N-series
  • Graphics card:
    • Intel® Arc™ A-series graphics  
  • Operating system:
    • Ubuntu* 20.04 (includes Ubuntu for IoT)
    • Ubuntu* 22.04 (includes Ubuntu for IoT)
  • At least 64 GB of disk space  
  • At least 8 GB of memory
  • Direct Internet access.

Ensure you have sudo access to the system and a stable Internet connection.

 

How It Works

The Intel® Edge Insights for Vision Test Module in the Intel® ESDQ for Vision package interacts with the Intel® ESDQ CLI through a common test module interface (TMI) layer, which is part of the Intel® ESDQ binary.

The selected components from the download options will be validated via the automated test suite.

Test results will be stored in the output folder. Intel® ESDQ generates a complete test report in HTML format and detailed logs packaged as one ZIP file, which you can email to the Intel® ESDQ support team at edge.software.device.qualification@intel.com.

Intel® Edge Insights for Vision Test Module

Intel® Edge Insights for Vision Test Module is a validation framework for  Intel® Distribution of OpenVINO™ toolkit 2023.2 in a Container and Intel® Distribution of OpenVINO™ toolkit 2023.1 Runtime. 

The Intel® ESDQ framework provides Intel partners (which include Original Design Manufacturers, Original Equipment Manufacturers, and System Integrators) a base to compare the performance of their custom designs with Intel’s reference designs. Furthermore, this framework allows Intel partners to validate the Intel® Edge Insights for Vision package and generate a hardware health status report.  

The test module serves the following purpose: 

  • Health Check: It runs a health check to make sure all the components that are part of the vision package listed below are properly installed and configured correctly. 
    • Intel® Distribution of OpenVINO™ toolkit 2023.1 in a Container 
    • Docker* Community Edition (CE)
  • Component Level Functionally Tests: For each of the OpenVINOTM toolkit components below, the test suite runs a series of sample applications such as decode, transcode, encode, and object detection on CPU, integrated GPU, and discrete GPU (if present), and then validates the output of the sample applications to make sure that all the components work as expected. 
    • classification_sample_async with CPU
    • hello_reshape_ssd with CPU
    • hello_query_device
    • hello_classification_sample with CPU
    • hello_nv12_input_classification with CPU
    • run_tc_benchmark_app with FP32_googlenet-v1_CPU
    • Image Classification Async Python sample with CPU
    • hello_query_device Python sample
    • hello_classification Python sample with CPU
    • hello_reshape_ssd Python sample with CPU
    • classification_sample_async with GPU
    • hello_reshape_ssd with GPU
    • hello_classification_sample with GPU
    • hello_nv12_input_classification with GPU
    • run_tc_benchmark_app with FP32_googlenet-v1_GPU
    • Image Classification Async Python sample with GPU
    • hello_classification Python sample with GPU
    • hello_reshape_ssd Python sample with GPU

 

Get Started 

The following are included in the installation of Intel® ESDQ for Vision package:

  • Intel® ESDQ CLI version 11.0.0
  • Intel® Edge Insights for Vision version 2.1  

 

NOTE: The screenshots may show a package version number that is different from the current release. See the Release Notes for information on the current release.

Download and Install Intel® ESDQ for Vision

  1. Select Configure & Download to download Intel® ESDQ for Vision, and then follow the steps below to install it. 

    Configure & Download

 

  1. Select Version or Tag, Target System, and Distribution options based on your system environment. Then, click the Download button to download Intel® ESDQ for Vision. In the next screen, accept the license agreement to proceed.
Configure and download screen. It says "Select options below to download," with options being version or tag, target system, distribution, and download.
Figure 1: Configure and Download

 

  1. Copy the Product Key. You will also receive an email from Intel with the product key.
Installer showing 3 steps to get started: Prepare the target system; copy the zip file to your target system; and extract and install the software.
Figure 2: Product Key in Download Page

 

  1. Transfer the downloaded package to the target Ubuntu system and unzip. 
    unzip edge_software_device_qualification.zip
  2. Go to the edge_software_device_qualification directory and view the edgesoftware_configuration.xml to see the Intel® ESDQ ingredient and test module for Intel® ESDQ for Vision package.  
    <project path="installation/src" id="63e376ff1af736ccf4e7f55b" version="2022.3" label="esdq/Vision_Test_Module"/>
<project path="installation/src" id="6351d2c0fb01e1e7576d8c63" version="10.0" label="esdq" esb_install="true"/>
  3. Provide executable permission to the edgesoftware file.  
    chmod +x ./edgesoftware
  4. Install Intel® ESDQ CLI and Intel® Edge Insights for Vision test module.  
    ./edgesoftware install
  1. When prompted, enter the Product Key. The Product Key is in the email you received from Intel confirming your download (or from the saved Product Key you copied earlier). 
  2. When the installation is complete, you see the message “Installation of package complete” and the installation status for each module.
Screenshot of terminal with text that shows different modules are being installed, and then it says “Installation of package complete. Recommended to reboot system after installation.” A table lists all the modules, their IDs, and their status as “success.”
Figure 3: Installation Status for Each Module

 

  1. Reboot the system. 
    sudo reboot

Run the Application

For documentation on the Intel® ESDQ CLI binary, refer to Intel® Edge Software Device Qualification Overview. The test modules are already available in the target system.

 

  1. Go to the Intel® ESDQ directory: 
    cd ~/esdq
  1. Run Intel® ESDQ for Vision test and generate a report:  
    esdq run

The following is an example of the report:

Test report showing tool suite information and test modules.

Test report showing hardware information.

Test report showing test module information, test execution summary showing all three test cases passed, and hardware information.

Test report showing software information and the performance of the three test cases showing expected and recorded values.

 

Release Notes 

Intel® Edge Insights for Vision Test Module 

Current Version: 2.1

  • Supports OpenVINO™ toolkit 2023.1
  • Supports Intel® Edge Insights for Vision 2.1
  • Test module architecture simplified to support discrete GPU (dGPU) and integrated GPU (iGPU)
  • Supports Ubuntu* 22.04 (includes Ubuntu for IoT) and Ubuntu* 22.04 (includes Ubuntu for IoT)

Version: 2022.1   

  • Intel® Edge Insights for Vision Test Module is upgraded to version 2022.1 to support OpenVINO™ toolkit 2022.1 and will not provide backward compatibility with older OpenVINO™ toolkit versions.    
  • This current release supports both Ubuntu* 20.04.    
  • Container OpenVINO™ tests are added. 
  • Removed Intel® VATS test module.

Version: 2021.4.2    

  • Intel® Edge Insights for Vision Test Module is upgraded to version 2021.4.2 to support OpenVINO™ toolkit 2021.4.2 and will not provide backward compatibility with older OpenVINO™ toolkit versions.
  • This current release supports both Ubuntu* 18.04 and 20.04.  
  • For 9th - 11th Generation Intel® Core™ processors, Kernel 5.4 is needed. For 12th Generation Intel® Core™ processors, Intel Atom® x6000E Series, Intel® Pentium® processors, and Intel® Celeron® N and J Series processors, Kernel 5.11 is needed to get the GPU working. There are known issues with Media SDK test. See the Known Issues section for more details.
New in this Release    
  • Container OpenVINO™ “super _resolution_demo”  tests are replaced with “image processing demo” tests for the device options of CPU and GPU.
  • Fixed Media sdk tests to support 12th Gen Intel® Core™ processors.  
  • AWS Greengrass is removed from the Vision Package.
Known Issues    
  • To get the GPU working on latest Gen Intel Atom® x6000E processor series and Pentium® processor and Intel® Celeron® processor N and J Series, Kernel 5.11 is needed along with the following:
    • Install intel-compute-runtime  
    • Add ID to grub (if the driver is not yet enabled in i915)
    • On platforms that do not have HEVC VDEnc enabled, H265 encode is not supported. In this case, “mediasdk_sample_encode” test case will fail.

 

Troubleshooting

Intel® Edge Insights for Vision Test Module 

Network Requirements

If faced with one of the following errors, check your firewall or proxy settings. You may update apt source to your local mirror site or run Intel® ESDQ in VPN network:

  • Failed to install python requirements
  • Failed to clone the GIT REPO” for model downloads
  • Request timed out after 3 minutes Unable to connect to github

      
openvino GPU tests failed 

[ ERROR ] Failed to create plugin /opt/intel/openvino_2021/deployment_tools/inference_engine/lib/intel64/libclDNNPlugin.so for device GPU 

Please, check your environment 

[CLDNN ERROR]. clGetPlatformIDs error -1001 

Possible Solution 

To access GPU capabilities, you need to have the correct permissions. The following will list the group assigned ownership of the render nodes, and list the groups you are a member of: 

stat -c "%G" /dev/dri/render*
groups ${USER}

If a group is listed for the render node, which isn’t listed for you, you will need to add yourself to the group using gpasswd. In the following, you will be added to the ‘render’ group and a new shell spawned with that group active: 

sudo gpasswd -a ${USER} render 
newgrp render

 

Intel® OpenVINO™ Toolkit Prerequisites

GPU Operations

To get GPU working on 12th or 13th generation Intel® Core™ processors, Intel® Processor N-series, and Intel® Core™ i3 Processor N-series, the following steps need to be followed:

Note: Requires Kernel 5.11 or above.  

 

  1. Install intel-compute-runtime driver:
sudo add-apt-repository ppa:intel-opencl/intel-opencl
sudo apt update
sudo apt install -y libze-loader libze-intel-gpu intel-opencl-icd
  1. Add PCI device ID to grub (if the driver is not yet enabled in i915):    
sudo nano /etc/default/grubfind GRUB_CMDLINE_LINUX_DEFAULTGRUB_CMDLINE_LINUX_DEFAULT="quiet splash i915.force_probe=4571"###
sudo update-grub

  

Media sdk sample encode test failed  

Media sdk sample encode test failed with the message: 

“ERROR], sts=MFX_ERR_UNSUPPORTED(-3),…"  

Follow the steps below as a possible solution: 

  1. sudo apt install --reinstall linux-firmware 
  2. Add i915.enable_guc=2  in /etc/default/grub: GRUB_CMDLINE_LINUX_DEFAULT="quiet splash i915.force_probe=* i915.enable_guc=2" 
  3. sudo update-initramfs –u 
  4. sudo update-grub 
  5. Reboot the system.
  6. After reboot, execute the command:  dmesg | grep guc
  7. Verify the logs for guc information. If output doesn't contain guc details, then get the latest guc/huc firmware. Copy the firmware to /lib/firmware/i915/, and reboot the system.

 

Support Forum

If you're unable to resolve your issues, contact the Support Forum. 

Product and Performance Information

1

Performance varies by use, configuration and other factors. Learn more at www.Intel.com/PerformanceIndex.