Going up

How YouTube are using Go to augment MySQL

Elliot Bentley
go-mysql1

Cutting-edge meets old school in YouTube with Vitess, which allows MySQL to be run at Google-size scale.

With so
much talk of the exciting potential of NoSQL databases, it’s easy
to forget that MySQL is
still the dominant player
.

YouTube love MySQL so much that they have augmented it to keep
up with the video site’s enormous traffic in an open-source project
called Vitess.

The first product to emerge is Vtocc, which is designed to
efficiently multiplex large numbers of incoming connections. At
Usenix LISA
conference
this week, Sugu Sougoumarane and Mike Solomon from
YouTube
spoke about the project
and their experiences of working with
Go.

MySQL, the unfashionable open-source relational database, is
preferred at YouTube for its reliability and simply being a known
quantity. However, its major drawback is scaling to a necessary
size. After all, it takes some serious engineering talent to serve

Gangnam Style
970 million times.

“The major problem with MySQL is that once you get to a certain
point, you spend a lot of time managing hardware and how many
instances you have,” said Solomon. “We want to automate that chunk.
We want to take every action that is complicated and error prone
and make it heal itself.”

Unwilling to mess with MySQL itself, however, they opted to write a
new product, Vitess, that runs in conjunction with MySQL. Its first
branch, Vtocc, has “been very steady, and it has all the necessary
tools for you to use in a production environment,” said
Sougoumarane. It allows thousands of SQL queries to be consolidated
into batches for efficiency, caching results to deal with duplicate
requests.

With YouTube sticking doggedly to old hand MySQL, it’s perhaps
surprising that they chose to use Google’s own server-side
language, which only just
celebrated its third birthday
, to build Vitess. Though now
offered on Google App Engine, it still has yet to find mainstream
adoption – except among Google.

Sougoumarane described the language’s features as
“well-thought-out”, adding: “They help you compose things in a much
more elegant way than traditional languages.” Go’s thread
management features are particularly useful, they said, though
admitting that its error handling, scheduling and garbage
collection is currently left wanting.

Author
Comments
comments powered by Disqus