This article is a cross-post from a post originally published on the AWS website in September 2023.
overview
Fetch, a consumer engagement and rewards company, offers an app that lets users earn rewards on purchases by scanning their receipts. The company also analyzes these receipts to generate insights into consumer behavior and provides those insights to its brand partners. As weekly scans rapidly increased, Fetch needed to improve its speed and accuracy.
On Amazon Web Services (AWS), Fetch optimized its machine learning (ML) pipeline using Hugging Face and Amazon SageMaker, a service for building, training, and deploying ML models using fully managed infrastructure, tools, and workflows. Now, the Fetch app can process scans faster and with much higher accuracy.
Opportunity | Accelerate ML Pipeline Fetching in 12 Months with Amazon SageMaker
The Fetch app allows customers to scan their receipts, receive points, and redeem those points for gift cards. To instantly reward users for scanning receipts, Fetch needed to be able to capture text from receipts, extract relevant data, and structure it for the rest of the system to process and analyze. With over 80 million receipts processed per week, and hundreds of receipts per second during peak traffic periods, the process needed to be fast, accurate, and at scale.
In 2021, Fetch began optimizing the app’s scanning capabilities. Fetch is an AWS native company, and its ML operations team was already using Amazon SageMaker for many of its models. This made our decision to migrate our models to Amazon SageMaker to power our ML pipeline easy.
Throughout the project, Fetch had weekly calls with the AWS team and received support from subject matter experts that AWS partnered with Fetch. The company built, trained, and deployed over five ML models using Amazon SageMaker over a 12-month period. In late 2022, Fetch rolled out an updated mobile app and new ML pipeline.
“Amazon SageMaker is a game changer for Fetch. We use almost every feature extensively, and when new features come out, they quickly add value. It’s hard to imagine doing this project without the capabilities of Amazon SageMaker.”
Sam Corzine, Machine Learning Engineer, Fetch
Solution | Reduce latency by 50% using ML and face-hugging on Amazon SageMaker GPU instances
“By leveraging the flexibility of Hugging Face AWS Deep Learning Container, we were able to improve the quality of our models, and Hugging Face’s partnership with AWS makes it easy to deploy these models.”
Sam Corzine, Machine Learning Engineer, Fetch
Fetch’s ML pipelines are powered by several Amazon SageMaker features, specifically Amazon SageMaker Model Training, which reduces the time and cost of training and tuning large ML models, and Amazon SageMaker Processing, a simplified managed experience for running data processing workloads. The company uses multi-GPU instances to run custom ML models for fast performance. “Amazon SageMaker GPU instances are easy to use,” says Ellen Light, backend engineer at Fetch. Fetch trains these models to identify and extract important information about receipts that businesses can use to generate valuable insights and reward users. Amazon SageMaker also makes Fetch’s custom ML system seamlessly scalable. “Amazon SageMaker makes it easy to scale up our systems, especially for inference and runtime,” said Sam Corzine, ML Engineer at Fetch. On the other hand, the deployment of standardized models reduces manual work.
Fetch relied heavily on Amazon SageMaker’s ML training capabilities, specifically training jobs, to refine and iterate on its models. Fetch can also train ML models in parallel, speeding development and deployment. “There is little resistance to deploying models,” says Alec Stashevsky, applied scientist at Fetch. “Basically, we don’t have to think about it.” This has increased confidence and increased productivity throughout the company. In one example, a new intern was able to deploy a model on his own by his third day on the job.
Since adopting Amazon SageMaker for ML tuning, training, and retraining, Fetch has increased the accuracy of its document understanding models by 200%. We continue to fine-tune the model for further improvements. “Amazon SageMaker is a critical tool for building these great models,” says Quency Yu, ML Engineer at Fetch. To optimize the tuning process, Fetch leverages Amazon SageMaker Inference Recommender. This is a feature of Amazon SageMaker that reduces the time required to get ML models into production by automating load testing and model tuning.
In addition to custom ML models, Fetch uses AWS Deep Learning Containers (AWS DL Containers). Enterprises can use it to quickly deploy deep learning environments using optimized, pre-packaged container images. This simplifies the process of using libraries from Hugging Face Inc. (Hugging Face), an artificial intelligence technology company and AWS partner. Specifically, Fetch uses the Amazon SageMaker Hugging Face Inference Toolkit, an open source library for providing transformer models, and the Hugging Face AWS Deep Learning Container for training and inference. “By leveraging the flexibility of the Hugging Face AWS Deep Learning Container, we were able to improve the quality of our models,” says Corzine. “And Hugging Face’s partnership with AWS makes it easy to deploy these models.”
Since adopting Amazon SageMaker, we’ve seen performance improvements for all metrics measured by Fetch. The company reduced the latency of its slowest scans by 50%. “Improved accuracy creates trust in our data among our partners,” says Corzine. Partners can confidently increase their use of Fetch’s solutions. “Being able to meaningfully improve the accuracy of literally every data point using Amazon SageMaker is a huge benefit that extends across our business,” says Corzine.
Fetch can now extract more types of data from receipts, giving us the flexibility to build the resulting insights based on the specific needs of our brand partners. “By leveraging ML, we can now extract exactly what our partners want from their receipts,” says Corzine. “Thanks to our investments in ML, our partners can create new types of offers, which is a huge additional benefit for them.”
Users are also enjoying the updates. Since the release of the new version, Fetch has grown from 10 million monthly active users to 18 million. “Amazon SageMaker is a game changer for Fetch,” says Corzine. “We use almost every feature extensively, and as new features emerge, they quickly add value. It’s hard to imagine doing this project without the capabilities of Amazon SageMaker.” For example, Fetch moved from a custom shadow testing pipeline to Amazon SageMaker shadow testing. This validates the performance of your new ML model against your production model to prevent outages. Shadow testing is more direct because Fetch allows you to directly compare performance and production traffic.
Results | Extend ML to new use cases
Fetch’s ML team is continually working on new models and iterating on existing models to improve performance. “Another thing we like is that new features in Amazon SageMaker allow us to keep our technology stack up to date,” says Chris Lee, ML developer at Fetch. The company will continue to expand its use of AWS for a variety of ML use cases, including fraud prevention across multiple teams.
Already one of the largest consumer engagement software companies, Fetch aims to continue to grow. “AWS is a key part of how we plan how we scale, and we will continue to leverage the power of Amazon SageMaker to improve accuracy,” says Corzine.
About fetch
Fetch is a consumer engagement company that provides brand partners with insights into consumer purchases. We also offer a mobile rewards app that allows users to earn rewards on purchases through the receipt scanning feature.
If you need help using SageMaker’s Hugging Face in your company, please contact us here. Our team will contact you to discuss your requirements.
