Deploy at scale with XARs: Facebook’s self-contained executable archives
The Facebook incubator team is at it again. This time, they’ve hatched a new kind of way to distribute self-contained executables with a new archive. Never worry about distributing at scale with XARs!
With over 2 billion users, deploying new code for Facebook is a kind of like a high-wire balancing act above a canyon where the slightest misstep ends in fire, chaos, and sharks. So how do Facebook’s developers deal with the kinds of operational challenges that come with distributing large pieces of software to thousands of machines? XARs, or eXecutable ARchivese, are a system for distributing self-contained executables for both code and data dependencies.
What are XARs?
Pronounced like “czar”, Facebook is giving XAR room to grow. The X is a placeholder for other languages, as it was designed to replace language-specific archives like PAR, JSAR, and LAR. Essentially, XAR lets developers package lots of files into one self-contained, executable file. Distribution and installation is a breeze.
XARs can be run from anywhere without needing a virtual environment or pre-installed modules on the receiving system. There’s no need to sacrifice speed, either; XARs are just as fast as natively installed apps. Importantly, dependencies are isolated from the machine the XAR is running on. This makes sure that there’s a compressed executable with no chance of getting tangled in whatever is already on the machine.
So, when should developers use a XAR? Well, the first is just to collect a large number of files for automatic mounting. A XAR file compresses the data, shrinking its overall size. Facebook engineers claim that some programs can be compressed to just 20% to save space and reduce IO bloat
Second, by making a XAR executable with the xarexec, it is a self-contained package of executable code. Kind of tautological, but in practice this means that XARs could replace virtualenvs and PEX files. There’s less overhead, more compatibility, and improved speed. Downside: you will need a setuid for mounting, but that’s a small price to pay.