Prometheus and Kubernetes – a match made in open-source heaven
As many startups of the last decade, SoundCloud’s architecture started as a Ruby on Rails monolith, which later had to be broken into microservices to cope with the growing size and complexity of the site. The microservices initially ran on an in-house container management and deployment platform. Recently, the company has started to migrate to Kubernetes. In their talk at the DevOpsCon, Fabian Reinartz and Björn Rabenstein demonstrated the current Prometheus setup at SoundCloud, monitoring a large-scale Kubernetes cluster.
With the introduction of microservices, the existing conventional monitoring setup failed both conceptually and in terms of scalability. Thus, starting in 2012, SoundCloud invested heavily into the development of the open-source monitoring system Prometheus, which was designed for large-scale highly dynamic service-oriented architectures. Migrating to Kubernetes, it became apparent that Prometheus and Kubernetes are a match made in open-source heaven.