Co-authors: Lu Wang (Office Depot), Jing Nicole Kong (Office Depot)
Real-time recommender systems are critical for the success of the e-commerce industry. Training deep neural networks (DNNs) of recommender systems at scale is challenging at Office Depot*. This project shares the experience and lessons learned from Office Depot and the Intel team in building an efficient, real-time DNN-based recommender system using Analytics Zoo1 and Apache Spark* on Amazon Web Services (AWS)*, all with the goal of increasing revenue per visit to the website.
Office Depot, Inc. (NASDAQ:ODP) is a leading business-to-business integrated distribution company providing business services and supplies, products, and technology solutions through its fully integrated omni-channel platform of approximately 1,350 stores, online presence, and dedicated sales professionals and technicians to small, medium, and enterprise businesses. Through its banner brands, Office Depot, OfficeMax*, CompuCom*, and Grand & Toy, the company offers its customers the tools and resources they need to focus on their passion of starting, growing, and running their business.
Office Depot was looking to move from a conventional rule-based vendor recommendation solution to a more intelligent in-house deep learning recommendation solution that can make personalized recommendation decisions based on a customer’s real-time session behavior. For this effort we faced the challenge of training DNNs using the massive amount of clickstream data at scale. Over 300 thousand active products are selling on the Office Depot digital business platform, which produces over 100 million distinct sessions monthly. Currently, Office Depot adopts the Spark ecosystem as the big data platform and uses Sparse MLlib to develop machine learning models; however, we struggled to leverage deep learning models at scale on our existing Spark ecosystem.
Newly developed DNNs have continuously proven successful as recommender systems by capturing the non-linear relationships in the user-item datasets in many industries. Empirical evidence shows that using deep layers of DNNs offers better recommendation performance.2,3,4 Therefore, it is critical for e-commerce industries like Office Depot to apply DNNs of recommender systems on the existing big data platform and pipelines.
In this article, we explain how to build an end-to-end flow of recommender system on AWS at scale, using distributed Keras on Analytics Zoo and Spark.
Analytics Zoo Solution
Analytics Zoo, open source by Intel, provides unified analytics and an AI platform that seamlessly unites Spark, TensorFlow*, Keras and BigDL5 programs into an integrated pipeline; the entire pipeline can then transparently scale out to a large Apache Hadoop*/Spark cluster for distributed training or inference, without extra graphics processing unit (GPU) infrastructure.1
The end-to-end flow of the recommender system runs on AWS, and includes a training flow and an inference flow (Figure 1).
First, the system processes the clickstream data using Spark on AWS. It indexes all the products and users employing Spark native StringIndexer and PipeLine, and further saves the pipeline model using MLeap.
val userIndexer = new StringIndexer().setInputCol("strUserId").setOutputCol("userId") val itemIndexer = new StringIndexer().setInputCol("strItemId").setOutputCol("itemId") val pipelineEstimator: Pipeline = new Pipeline() .setStages(Array(userIndexer, itemIndexer)) val plModel: PipelineModel = pipelineEstimator.fit(df)
Then, the system further assembles features into RDD of samples for recommender models of Analytics Zoo. For the session recommender, features of each session are prepadded with a length of 10.
Different DNNs are trained at scale using Analytics Zoo, including a neural collaborative filtering (NCF) network, a Wide and Deep (WND) network, and a session recommender based on recurrent neural networks (RNN, Figure 2), and saves the model on Amazon S3*. Currently, Analytics Zoo provides three built-in recommender models for different use cases.
Figure 2. Session recommender based on RNN
val model = SessionRecommender[Float](itemCount, itemEmbed, maxLength) model.compile(loss="mse", optimizer= "adam") model.fit(data, nbEpoch=20) model.saveModel(modelPath)
Finally, the system loads the bundle files of the pipeline model and transforms products into indices. Then, it loads the recommender model and serves it for inference on the web service using a local predictor of BigDL.5 Recommendation of products for each customer are based on ranking of the probability of adding a specific product into his cart. The inference flow scales out efficiently by leveraging play framework, and substantially speeds up response latency by removing Spark dependencies.
val model = LocalPredictor(Module.loadModule[Float](bigDLModelPath)) model.recommenderForUsers(data, 5)
Results and Discussion
Models were evaluated from both offline performance comparison and online A/B test.
Table: Offline performance comparison of different recommender systems
|Model||ALS||NCF||Wide and Deep||Session Recommender|
|Top 5 Accuracy||16.2%||46.7%||45.2%||52.3%|
The offline performance comparison (Table 1) shows that, by adopting the end-to-end flow of Analytics Zoo solution-based DNNs, we saw a significant improvement of accuracy compared to traditional recommendation algorithms like ALS. NCF improved the top five accuracy from 16.2% to 46.7% compared to ALS by capturing the non-linear relationship between the users and products.2 The WND model achieved a competitive top five accuracy compared with NCF. On top of a DNN to learn non-linear relationships between users and products, we added a wide model to memorize user profiles including loyalty indicator, Office Depot card user indicator, and so on. Our user profiles added noise to WND.
According to Wu,4 each user session in an e-commerce system could be modeled as a sequence of web pages, and a deep RNN could track how users browse the website using multiple hidden layers (Figure 2). By building a session recommender based on two layers of GRU using Analytics Zoo, we boosted the top five accuracy 6% more on top of NCF and WND.
We deployed session recommenders on our website and ran an online A/B test over two weeks against the current rule-based recommender system. The session recommender showed a lift of 1% for sales and 1.6% for average order values, respectively, compared to the current recommender.
This article briefly introduced the challenges that Office Depot faces to build a DNN-based recommender system to increase online sales. We presented an end-to-end deep learning pipeline using Analytics Zoo running on AWS to model the sequences of products using DNN-based recommenders. We showed online and offline evaluation results, and found a session recommender based on RNN that boosted both sales and average order value. A similar session recommender will likely play a key role in other use cases of e-commerce; more examples and APIs are in the Analytics Zoo Model Recommendation.
1. Analytics Zoo
2. X He, L Liao, H Zhang, etc., 2015. Neural Collaborative Filtering
3. H-T Cheng, 2016. Wide & Deep Learning: Better Together with TensorFlow
4. S Wu, W Ren, C Yu, etc., 2016. Personal Recommendation Using Deep Recurrent Neural Networks in NetEase
6. J Dai, Y Wang, X Qiu, etc., 2018. BigDL: A Distributed Deep Learning Framework for Big Data