Archived - “When the Shoe Fits…” Using 3D Imaging to Improve Shoe Sizing

The Intel® RealSense™ SDK has been discontinued. No ongoing support or updates will be available.

Abstract

This case study shows how Volumental used Intel® RealSense™ technology with its cutting-edge 3D modeling and virtualization software to scan a customer’s feet in real time so consumers could select an appropriately sized pair of shoes. With Volumental's software and the Intel® RealSenseTM F200 camera and SDK, capturing three angles in 3D is sufficient to size feet within 2mm (1/12th of an inch). Now that the Intel Real Sense camera(s) will be available in  Ultrabook™, 2 in 1, and all-in-one systems, Volumental sees a world with custom fitting shoes, clothes and even medical devices will be within reach of the mass market. 

Introduction

If you’ve ever shopped for children’s shoes, you know that buying the right-sized pair of shoes is challenging. Not only is your child growing fast, but you have to get your child to sit still to try on shoes. Then you have to wonder about the accuracy of his or her response about the fit of the shoes, given the shoes are associated with a favorite cartoon character.

What if your child’s feet could be quickly and accurately measured before trying on any shoes so that you could limit the selection to shoes in that size?

Volumental and Intel® RealSense™ Technology

Volumental (https://www.volumental.com/) has a plan to realize that concept using Intel RealSense technology. And 3D scanning for accurate measurement of the body won’t just be for measuring feet and buying shoes. Caroline Walerud, CEO says: “At Volumental we believe that everything you wear should be made to fit you, not the other way around.” Volumental believes that 3D scanning, most notably of the body and body parts, will open up the field of product customization, not just better fitting clothes and shoes but medical devices as well. And now with Intel RealSense technology, this reality will be within the reach of the mass market.  

Volumental is a 3D body scanning startup, a spinoff from a doctoral research project in Computer Vision at the Royal Institute of Technology in Stockholm. When Intel announced Intel® RealSenseTM hardware and software, Volumental took notice and immediately saw the technology as a vehicle for their cutting-edge 3D modeling and visualization software. But moving from a broad concept to an actual project took some innovation.

Intel led with the user-facing Intel® RealSenseTM F200 camera, integrated into Ultrabook™, 2 in 1, and all-in-one systems, while Volumental wrote the code to measure and create 3D models of feet, first for children (the more difficult) and then adults. 

Volumental did user testing to see how consumers
would “naturally” try to measure their feet using a laptop or tablet.;
Intuitively, users thought they would move a device
around their foot to perform a scan (Figure 1),
rather than place a device on the floor.
But once shown a visual with the device on the floor, users agreed that made more sense. It became obvious that a 2 in 1 PC in tent mode (or stand mode) provided the the best experience (Figure 2).  It was important that users should be able to scan bare or socked feet, in the interest of creating a product that wouldn’t feel awkward to use, and that was easy and intuitive.  By adding user instructions for the position for each of the 3 poses,  it’s possible to get good results in a home setting.
For retail stores, where shoe departments want to allow self-service but not risk customers walking off with the scanning device, a foot scanning kiosk, like the one shown in Figure 3, was the right solution. Diagrams on the bottom pad show the user how and where to stand. 


Using the Intel RealSense depth and color cameras, Volumental has created an application that takes 3D images of the feet, either bare or in socks. The application scans the feet using three views, detecting each foot and the direction it is facing (Figure 4). 

The Intel RealSense 3D camera provides both a color image and corresponding depth information for each pixel.

The Volumental software analyzes the data from the three snapshots and provides the length, width, and girth around the instep, creating a useful 3D model of the customer’s feet.  The resulting model captures thousands of measurements and can be used for both size recommendation and the making of bespoke shoes. 

Figure 5 shows an example of a customer’s 3D model, and shoe stores can provide a catalog of footwear available in that size. 

How it works

Using pre-beta release hardware and the Intel RealSense SDK 2014, Volumental, Intel, and PLAE, a San Francisco-based children’s shoe manufacturer, showed a demo of the possibilities of measuring children’s feet at the Intel Developer Forum in San Francisco in 2014. And early in 2015, Volumental and Intel showed off their latest version—a freestanding kiosk computer with the Intel RealSense F200 camera at foot level, which will be placed in some retail shoe departments in 2015.

The ShoeSizer software combines computer vision and machine learning to create accurate 3D models of feet in a few seconds. It’s ok if the feet are moving (slightly) as Volumental uses algorithms that keep track of the 3D trajectory and make sure the result is accurate.

The software’s algorithms are based on machine learning, and the algorithms are trained on large databases of feet to ensure the creation of accurate models even when there is image noise or missing data. While the algorithms and actual processing are Volumental’s intellectual property, Intel was able to help improve performance by focusing on multithreading using OpenMP*. The resulting models contain more information related to the foot’s size than just a conversion to shoe size.

Other challenges

Touchscreen systems requiring the use of the hands aren’t helpful in this usage that focuses on feet. However, the additional capabilities of Intel RealSense technology including the capability to use voice commands quickly became valuable. The kiosk accepts user’s voice commands to activate the scanning process.

But one has to consider that there’s a vast variety of socks, all with different imaging factors. It took extensive testing to find out that white cotton works the best but other colors, patterns and textures can be worn – although black, heavy texture was originally a challenge. Volumental’s code had to adjust to accept a wide range. And, believe it or not, the floor color and texture are also important since the plane of the floor is a factor in measurement. This is another benefit of measuring at a kiosk with a base for the foot scan.
 

Beyond the shoe

Volumental CEO Caroline Walerud comments “This is only the beginning for Volumental, we are looking at productifying and commercializing this proof of concept…and are very excited about the opportunity to bring our first solutions to the market. We see easy foot data capture as a step towards an entirely new way of manufacturing shoes in a more localized and customized manner. In the future manufacturing will change dramatically; it will involve individual customers to a much greater extent than today.”

While inviting for retail shoe shopping, there is also value to the orthotics and medical device industries and to forward-thinking footwear companies to offer product customization. Volumental is currently working on a number of customer projects with partners in orthotics and bespoke shoemaking in addition to major shoe retailers. With new cameras coming from Intel you’ll soon be seeing 3D scanning for consumers wherever you shop.

About the Author

Colleen Culbertson is a Platform Application Engineer with the Developer Relations Division, who authors articles and a regular blog on the Intel® Developer Zone.


Intel, the Intel logo, and Intel RealSense Ultrabook are trademarks of Intel Corporation in the U.S. and/or other countries.
Copyright © 2015 Intel Corporation. All rights reserved.
*Other names and brands may be claimed as the property of others.

For more complete information about compiler optimizations, see our Optimization Notice.