Prometheus and Kubernetes – a match made in open-source heaven
As many startups of the last decade, SoundCloud’s architecture started as a Ruby on Rails monolith, which later had to be broken into microservices to cope with the growing size and complexity of the site. The microservices initially ran on an in-house container management and deployment platform. Recently, the company has started to migrate to Kubernetes. In their talk at the DevOpsCon, Fabian Reinartz and Björn Rabenstein demonstrated the current Prometheus setup at SoundCloud, monitoring a large-scale Kubernetes cluster.
With the introduction of microservices, the existing conventional monitoring setup failed both conceptually and in terms of scalability. Thus, starting in 2012, SoundCloud invested heavily into the development of the open-source monitoring system Prometheus, which was designed for large-scale highly dynamic service-oriented architectures. Migrating to Kubernetes, it became apparent that Prometheus and Kubernetes are a match made in open-source heaven.
Fabian Reinartz is a software engineer at CoreOS and one of the Prometheus core developers.Previously, he was a production engineer at SoundCloud and worked on information retrieval during his academic education.
Björn Rabenstein is the team lead of Production Engineering at SoundCloud and one of the main Prometheus developers. Previously, he was a Site Reliability Engineer at Google and a number cruncher for science.
