How Zomato Manages Its Logs
Zomato is the largest food-delivery app in India, with over 75 million active users and operations in more than 3000 cities.
The app handles thousands of food orders every minute during peak times and festivals. To ensure everything runs smoothly monitoring the system performance is very crucial. One way to do this is through logs monitoring.
Logs are records of events that happen within the app, for example when an order is placed or delivered.
Old System: Elasticsearch
- What it was: Initially, Zomato stored all its logs in a system called Elasticsearch.
- Problems: Managing Elasticsearch was challenging and costly. The team faced high toil in maintaining the Elasticsearch cluster, and the rising costs made it unsustainable.
Solutions Explored AWS S3 and Trino
- What they tried: During ElasticSearch, Zomato experimented with storing logs in AWS S3 and using Trino to query them.
- Problems: This approach resulted in high latency when reading the logs, which was not suitable for their needs.
Switching to ClickHouse
- New solution: Zomato decided to switch to ClickHouse, an open-source system developed by Yandex.
- Benefits: ClickHouse is designed for high-speed data writing and reading. It uses advanced compression techniques to save storage space and supports a SQL-like query language, making it accessible even to non-developers.
Understand How ClickHouse Works
- Log flow: Logs are first sent to Kafka, a system that handles data streams efficiently.
- Batching: The logs are then grouped into batches, with each batch containing up to 20,000 log entries.
- Insertion: These batches are inserted into ClickHouse. This process is quick, taking less than 5 seconds from the generation of a log message to its insertion into ClickHouse.
By adopting ClickHouse, Zomato has significantly improved its log management system.
This change ensures that the app remains fast for its users, even during high traffic times. The new system allows Zomato to handle large volumes of data efficiently, providing better performance and reducing costs.
