Client Story
A UK-based property management platform serving independent hotels, B&Bs, and vacation rentals across Europe was building an AI pricing module to replace manual rate management with automated, market-based recommendations. The platform needed a daily structured feed of competitor prices from Airbnb, Booking.com, and Hotels.com — platforms that don’t share data.
| Industry: | Hospitality |
|---|---|
| Year: | 2025 |
| Location: | EU |
Read summarized version with
"We need to get those prices in, and we need to have a daily feed of those prices so that the AI engine that we're developing can look at all the movements every day and then come up with a recommendation as to what the best price should be." — Director, CEO's Office, EU Hospitality SaaS Platform
"The real challenge was finding the right prices. By mapping the competitive set across 18 different platforms first, the AI isn't just guessing, but finally making decisions based on the same inventory our customers are fighting against every day." - Head of Data, EU Hospitality
Daily Pricing From Platforms That Don't Give It Away
Airbnb and Booking.com don’t provide pricing APIs. Turning browser-visible rates into a structured, property-level AI feed — daily at scale, across two distinct property segments — required two separate pipelines, not one. Equally urgent: identifying which properties across 18 competitor platforms were actually in the market. Those platforms publish no client directories.
The AI engine doesn’t compare a London hotel against all prices on the market — it compares it against a defined set of similar properties in the same location, segment, and size. Without that comp set defined first, the recommendations are meaningless
Two Pipelines and a Competitor Audit
The audit was the prerequisite — the AI engine cannot recommend a price without a defined competitive set.
Competitor Platform Audit. GroupBWT crawled 18 platforms — Amenitiz, Cloudbeds, SiteMinder, Lodgify, Guesty, Mews, and thirteen others — surfacing ~62,000 candidate sites in aggregate before deduplication and availability filtering. The estimated live competitive set after filtering was 10,000–50,000 active properties. Unstructured pages went through Claude Haiku 3.5 to extract contact data that static selectors couldn’t reach.
The audit output — property-level identifiers — was then matched to corresponding Booking.com and Airbnb listings, producing the URL universe that the daily scrapers consume. Each client receives a curated comp set drawn from this universe, tuned to their geography and property type.
Vacation Rentals Pipeline. A dedicated scraper pulls from Airbnb and Booking.com daily. The 90-day window is the AI’s near-term inference horizon — the booking window where competitor prices shift most dynamically. The 365-day monthly sweep is the seasonality training range. Both single-guest and max-occupancy price points are tracked per listing so the model can calibrate recommendations across property sizes. Requests geo-route through Oxylabs residential proxies matched to the client’s markets: Booking.com returns different prices by request country, and wrong-geography data would skew every UK and European recommendation.
Hotels and B&Bs Pipeline. The hotel pipeline mirrors the cadence on Booking.com exclusively — Hotels.com was retired in October 2025 after bot protection proved insurmountable, with coverage maintained by expanding Booking.com listings 8x. Three rate criteria are collected per property per day: any/cheapest, non-refundable, and breakfast-included. The AI recommends in a policy context — without this split, it compares prices that aren’t comparable.
Both pipelines entered full-scale production in October, 2025, orchestrated on Kubernetes via ArgoCD. Structured data is delivered to S3 daily in a fixed CSV schema — consistent field names, normalized dates, stable property IDs — so the client’s ML pipeline picks it up without any additional parsing or reformatting.
"Separating vacation rental and hotel scraping into two pipelines isn't about convenience. The competitive set is different, the booking platforms are different, the data model is different. One scraper built for both will fail to serve either." — Lead Data Engineer, GroupBWT
The Platform's AI Engine Has the Market Data It Needs
- Uninterrupted daily feed since October 2025 — the AI pricing module receives live competitor data every morning without any preparation work from the client’s team.
- Competitive set defined across 18 platforms — for the first time, the client has a structured universe of competitors per property type and geography that the AI model can train and recommend against.
- Two segment-matched pipelines in production — vacation rentals and hotels run separately, with data models matched to each segment’s booking dynamics.
As of April 2026, the client’s in-house AI team is using the feed as the training set for their pricing model — the production recommendation layer is on their roadmap.
Tech stack: Python 3.12+, RabbitMQ, MySQL, Docker, Helm, ArgoCD, Kubernetes, Oxylabs Residential, Claude Haiku 3.5, BuiltWith, PublicWWW, Google Search API
Need Live Competitor Data to Power an AI Pricing Feature?
We design and deploy scraping pipelines for product teams that need market pricing at a cadence and volume no commercial API will provide.
You have an idea?
We handle all the rest.
How can we help you?
