Real-Time Sensor Fusion for Loss Detection

Deploy sensor fusion technology for loss detection at self-checkout and enable a more seamless experience. Use machine learning to connect different sensors such as point-of-sale systems, weight scale sensors, cameras, and RFIDs to accurately detect checkout items.

Target Operating System Ubuntu* 16.04 LTS or 18.04 LTS
Time to Complete Approximately 7.5 hours
Software Used EdgeX Foundry*
Docker*
Portainer
Intel® Video Analytics API (Intel® VA API)
Intel® RFID Sensor Platform SW Toolkit (Intel® RSP SW Toolkit)

GitHub* 


What You Will Learn

Learn how different sensor devices can use the common open-middleware framework, EdgeX Foundry, to optimize retail operations, and detect loss at checkout. The sensor fusion is implemented using a modular approach, combining point-of-sale systems, computer vision, RFID, and scale as microservices.

Gain insight into:

  • EdgeX Foundry and how data flows through it
  • Microservices architecture
  • Benefits of edge compute in IoT
  • Benefits of sensor fusion
  • Loss detection
  • Item verification
  • Improving customer experiences

Learn to build and run an application with these capabilities:

Recognize and trigger an event on the EdgeX bus when a product has entered and exited the field of vision within a checkout area.


Use the EdgeX extensible framework for add-on services and sensors.


Use multiple edge sensors to accurately recognize items, detect discrepancies, and record a real-time transaction log (RTTL).


How It Works

Sensor fusion, also called basket reconciliation, is the root of the detection system.

Different sensor devices trigger events, such as detecting an RFID tag or specific food type.


The EdgeX framework publishes these events to the main application as an event message, and through the microservices assembles the information to correlate events to the detected products.


The data is then reconciled to make sure that it matches what is actually being purchased.

 

Any objects that cannot be reconciled could mean the purchase is suspicious. A point-of-sale system integrates with either the EdgeX REST or MQTT device services to send the events. The Scaled Devices Service provided is specifically for a CAS USB scale and is a good starting point for integrating other USB scales. Alternatively, the scale events can be sent to the EdgeX REST or MQTT device service. The object detection model (based on the Intel® Distribution of OpenVINO™ toolkit) used by the Intel® Video Analytics API is an example for object detection. For a robust production-ready solution, you must provide an improved object detection model.

 

flow chart graphic showing how the smart retail analytics solution works