Web scraping for online cosmetics marketplace

Learn how GroupBWT helped an FMCG company optimize its procurement processes by extracting competitors assortment information

single cases background

The Client Story

Our client is a major FMCG beauty and cosmetics retailer. Dealing with numerous SKUs predetermined risks relating to assortment planning, he found that some products were sold out before meeting demand, while others remained unsold for months. This was causing significant costs and affected his profits. The company approached us looking for a data-driven solution to their stocking issues, and eliminated assortment inconsistencies

Industry: Retail and e-commerce
Cooperation: Since 2018
Location: USA

Improved assortment planning system with the possibility to track real-time data on the market

Achieved 85% accuracy in predicting demand, and doubled revenues within 4 months

Introduction

Retail industry challenges related to the products’ variety

The retail industry has a myriad of input data and prerequisites that should be constantly tracked and analyzed to maintain a profitable business model.

Old-fashioned ways of doing business have now been overtaken by new, faster and cheaper methods, such as web data scraping. Web scraping is mainly used by e-commerce businesses to monitor competitors prices, reviews, and availability. In this case we will highlight another less obvious application of web data scraping, Product Assortment Planning and Optimization.

single cases content
The Solution

Custom scrapers for 3 different purposes were built to organize the data collection

By performing this action, scraper was able to receive information on the current inventory of the product. That left a number of products still in supply. By executing the same algorithms on a daily basis, we created a stable data pipeline. Additionally, we built an administration panel that could be turned into an interactive dashboard.

Our client could now track all changes in real time, and all the data we collected could be exported for further use. The final step was to test its performance and to eliminate any possible technical inaccuracies. Following detailed efficiency checks, the whole process was scaled up nationwide. We checked all stocked products and their prices every day. These checks were done in two ways, firstly by writing directly on the site the quantity in the stock. Secondly, we added a random amount to the cart, for example 1.000.000, this prompted the site to show “error”, giving us an insight into the actual number available.

In order not to overload the competitors’ sites we made sufficient intervals between requests and stretched each scraping session over several hours. We were careful to ensure that there were no critical loadings put on the competitors’ sites which may adversely affect his business. We also introduced a proxy to prevent us from being locked.

single cases content

In order not to overload the competitors' sites we made sufficient intervals between requests and stretched each scraping session over several hours. We were careful to ensure that there was no critical loading put on the competitors' sites which may adversely affect his business. We also introduced a proxy to prevent us from being locked

avatar
Vladimir Kalashnik
Developer
The Results

Our client's assortment process was improved, and led to increased profits of 47% over the following 4 months, with 85% accuracy in demand predicting

We tackled the problem by applying web scraping mechanisms, and building a customized scraper that follows the given algorithms. Scraper uses proxy and time delay between queries to avoid a ban.

Our task was to develop a streamlined procurement system based on the real-time market infrastructure and this was achieved with great success. As a result, our client now has a much better oversight in a highly competitive infrastructure, and improved his financial indicators. He then reorganized the supply division by integrating our automated solution. A greater level of certainty and flexibility was seen after only two months. Now the demand for his products can be predicted with 85% accuracy

single cases content

Ready to discuss your idea?

Our team of experts will find and implement the best eCommerce solution for your business. Drop us a line, and we will be back to you within 12 hours.

Contact Us