How to scale a logging infrastructure to accept a billion messages a day
Over the past year, OpenTable have been re-architecting their system from a monolithic architecture to move more towards microservices and small applications. As the infrastructure has changed, so too the logging infrastructure has had to change. In this session, recorded at the DevOps Conference, Paul Stack explains how they did it.
We then adopted the ELK stack. This allowed us to be able to scale more. As the company moved into the cloud, we had to be able to scale even more. We decided that we would move to Apache Kafka.
This talk is all about the steps we went through and how we approached the work as well as the lessons learned. We are now in a position where we are able to elastically scale our logging infrastructure easily.