How YouTube are using Go to augment MySQL
Cutting-edge meets old school in YouTube with Vitess, which allows MySQL to be run at Google-size scale.
much talk of the exciting potential of NoSQL databases, it’s easy
to forget that MySQL is
still the dominant player.
YouTube love MySQL so much that they have augmented it to keep
up with the video site’s enormous traffic in an open-source project
The first product to emerge is Vtocc, which is designed to
efficiently multiplex large numbers of incoming connections. At
conference this week, Sugu Sougoumarane and Mike Solomon from
spoke about the project and their experiences of working with
MySQL, the unfashionable open-source relational database, is
preferred at YouTube for its reliability and simply being a known
quantity. However, its major drawback is scaling to a necessary
size. After all, it takes some serious engineering talent to serve
Gangnam Style 970 million times.
“The major problem with MySQL is that once you get to a certain
point, you spend a lot of time managing hardware and how many
instances you have,” said Solomon. “We want to automate that chunk.
We want to take every action that is complicated and error prone
and make it heal itself.”
Unwilling to mess with MySQL itself, however, they opted to write a
new product, Vitess, that runs in conjunction with MySQL. Its first
branch, Vtocc, has “been very steady, and it has all the necessary
tools for you to use in a production environment,” said
Sougoumarane. It allows thousands of SQL queries to be consolidated
into batches for efficiency, caching results to deal with duplicate
With YouTube sticking doggedly to old hand MySQL, it’s perhaps
surprising that they chose to use Google’s own server-side
language, which only just
celebrated its third birthday, to build Vitess. Though now
offered on Google App Engine, it still has yet to find mainstream
adoption – except among Google.
Sougoumarane described the language’s features as
“well-thought-out”, adding: “They help you compose things in a much
more elegant way than traditional languages.” Go’s thread
management features are particularly useful, they said, though
admitting that its error handling, scheduling and garbage
collection is currently left wanting.