Twitter announces an experimental Scala compiler focused on compilation speed
Twitter is home to some of the biggest Scala codebases on the planet. So it’s no surprise that a group of Twitter engineers have come up with an experimental new Scala compiler aimed solely at improving compiler speed.
Twitter has a lot of things going on behind the curtain. For one, it takes ones of the largest Scala codebases in the world to keep the Twitterverse running and tweeting. Improved compilation time is one of the most common requests from engineers there.
So, Eugene Burmako and a group of developers at Twitter have come up with the Reasonable Scala Compiler as a solution to Twitter’s specific problems. The Reasonable Scala Compiler (or rsc) is openly inspired by the work of Martin Odersky and Grzegorz Kossakowski.
Okay, so how does this Scala compiler work?
Burmako and his team believe it’s possible to dramatically improve the compilation speed time for typical Scala codebases.
According to Burmako:
“Our typechecker is based on the insights of the Kentucky Mule project developed by Grzegorz Kossakowski who has convincingly demonstrated that typechecking Scala can be embarrassingly parallel.
Our compilation pipeline consists of only 4 passes over syntax trees in comparison with 20+ passes in Lightbend/Typelevel Scala and 15+ passes in Dotty.
Furthermore, we believe that restricting our compiler to support just a subset of Scala will enable further compilation speedups. We call this subset “Reasonable Scala” and will dedicate focused effort to identifying it.”
Basically, the rsc is aimed at fostering innovation in compilation performance and openly prototyping performance-focused designs. The RSC findings will be made available to the Scala community at large; Burmako and his team have already stated that they plan on open sourcing the rsc in the future. (As of now, the compiler is on GitHub.)
The rsc is aimed to have a certain level of cross-compilabiltiy with other Scala codebases. At the very minimum, everything in rsc will be compatible with Scala. After all, Twitter runs on Scala and has a large codebase already. As Burmako said, “we take compatibility extremely seriously.”
- Dramatically improve Scala compilation performance
- Study compilation time overhead of various Scala features
- Identify a subset of Scala that can be compiled with reasonable speed
- Facilitate knowledge transfer to other Scala compilers
- Full backward compatibility (consider Lightbend Scala instead)
- New language features (consider Dotty and Typelevel Scala instead)
- Improvements to the type system (consider Dotty and Typelevel Scala instead)
- Runtime performance (will be addressed independently)
This is an exciting new project aimed at improving compilation speed. Obviously, it’s still in the beginning stages. If you’re interested in checking it out, head on over to the Reasonable Scala Compiler GitHub page.