All systems are Go!

Parse picks Google’s Go over C#, Java, C++ and JRuby

Natali Vlatko
Like image via Shutterstock

When you’re looking at the prospect of a total rewrite, its important to review all the options. That’s exactly what Facebook’s Parse team did when choosing Google’s Go as their next step, and their hard work was worth the effort.

Facebook’s Parse team has recently published a blog post detailing their switch from Ruby on Rails to Google’s Go, with the move described by Engineer Charity Majors as having “saved our sanity”.

Parse’s initial versions had begun as a Ruby on Rails project in 2011. Majors describes their beginning on Rails as a period of fast functionality additions and small-team iterations made possible by a deep bench of library support, gems, deploy tooling, and Ruby on Rails best practices.

Combined with Capistrano to deploy their code, RVM to manage their environment, Chef to deal with their infrastructure and a host of open-source tools, Parse was deemed to work perfectly fine “for a while”.

After getting to a point where it took 20 minutes to do a full deploy or rollback, not to mention the complicated load balancer shuffling they were doing, the team realised that their growth had also resulted in a growing-out of their Ruby on Rails foundations. Their problems consisted of, but were not limited to, the following:

  • Fast API traffic and app growth resulted in the need to rapidly spin up more database machines to handle new request traffic
  • Increased traffic and growth together caused Rails’ “one process per request” model to break down, with the Rails worker pool easily filling up when a large amount of slow requests came in. Auto-scaling groups were not fast enough to react and worker-status was often declared wasteful as they waited for another service
  • Adding more databases and workers to keep up with the growth also added more points of failure and “more ways for performance to get degraded”

Becoming aware of the fact that Rails wouldn’t scale, Majors states that they “had to move to an async model that was fundamentally different from the Rails way”. The team acknowledged that an entire rewrite was going to be difficult, but it was essential in order to allow for further growth.

Making the new language choice

When looking at their options, the Parse team considered the following alternatives:

  • JRuby and Java: While JRuby seemed like the obvious choice, the issue of asynchronous library support also had the team thinking about another possible rewrite in Java in the long run. The JVM can handle massive concurrency, but this wasn’t enough to convince the team
  • C++: Majors says that the C++ route didn’t seem like the right choice thanks to their already existing C++ code being harder to debug and maintain, despite having “a lot of experienced C++ developers on our team”. Lack of library support and HTTP request handling sealed the deal for the team in opposing this option
  • C#: C#’s concurrency model was given a nod as the best of the lot, however the unavailability of libraries that inter-operate with many common open-source tools on Linux meant that Parse’s own toolbox would have to change dramatically on top of the rewrite
  • Go: Complex interaction with MongoDB is said by Majors to be core to Parse. Go’s MongoDB driver, asynchronous operation at a low level, coupled with the team’s excitement about working with Google’s open-source programming language cemented their choice to go with Go

Before embarking on the mammoth task of a whole rewrite, the team tested Go by rewriting their EventMachine push backend from Ruby and were impressed with the results:

After rewriting the EventMachine push backend to Go we went from 250k connections per node to 1.5 million connections per node without even touching things like kernel tuning. Plus it seemed really fun. So, Go it was.

After all the effort, the end result meant that full API server deployment became a 3 minute task, the full integration test suite was done in 2 minutes and overall reliability of Parse “improved by an order of magnitude”. Most importantly, their API no longer spun out as more databases and backing services were added.

To sum up, Parse loves Go. “We’ve found it really fast to deploy, really easy to instrument, really lightweight and inexpensive in terms of resources. It’s taken a while to get here, but the journey was more than worth it”.

Natali Vlatko
An Australian who calls Berlin home, via a two year love affair with Singapore. Natali was an Editorial Assistant for (S&S Media Group).

comments powered by Disqus