This is a guest post by Klaus Schaefers, Senior Software Engineer at Booking.com and Basak Eskili, Machine Learning Engineer at Booking.com, in partnership with AWS.
As a global leader in the online travel industry, Booking.com continuously works to improve the travel experience for its users. Latency is a key factor in achieving this—nobody likes waiting for their search results to be returned.
Booking.com generates several million real-time predictions per minute, supporting a variety of algorithms, including ranking and fraud detection. Common to most algorithms is the need for ultra-low end to end latencies, often less than 100 milliseconds. Higher latencies not only result in a worse user experience, but also ultimately mean a loss in conversion rate. To meet these demands, Booking.com developed an ultra-low latency feature platform, capable of serving ML features with a p99.9 latency below 25 milliseconds at a scale of 200,000 requests per second.
…
