Experiences of working with Scala
The Joy of Scala
Picture a sunny morning in 2004: a light and airy office in a village in rural England where a group of people are seeking the ideal development language. An innate and intuitive distrust of proprietary options; a less visceral - but nonetheless real - dislike of PHP; a positive feeling for some aspects of
Java but a sense it could be cleaner, less cumbersome and altogether "better"; an evaluation of the lesser-known and more exotic languages; and an evolving admiration for, and growing recognition that there was one: expressive, fast, safe, maintainable, scalable. It had to be Scala!
Looking back this was an ambitious but risky step. In those days Scala was not the stable, tried and tested solution to our problems it is now. The compiler was buggy. There were few libraries, limited documentation and even less sample code. The community was very small but punched above its weight to support other members when they encountered problems. But we all knew that we had found something good and, despite these early teething problems, it was worth sticking with. With greater knowledge came intellectual satisfaction rather than contempt.
As time goes by ....
Over the last five odd years we have completed a variety of projects, for many clients in many different environments and domains. Each has brought its own unique challenges, but from each assignment we have learned: where to start; how to integrate Scala into a multi-language, multi-paradigm environment; how to build Scala web applications that scale and are easy to replicate; design patterns and "best practice"; about Scala libraries and what makes them more or less general and flexible. And also of course how to construct the development environment and team structure that makes us most effective and efficient.
What follows in an attempt to pass on some of the things we've learnt. We're not saying all these will be right for you in your environment, that they're the best ways of doing things or that they will work for everyone. This is just what we've picked up along the way.
Why we still use Scala? The benefits.
The benefits we originally sought to obtain from Scala have, largely, been realised but as we've been through full development cycles we've found that some of the delights of using Scala are both more subtle and more profound than we realised.
Scala is hugely capable at giving the programmer more power to express code in the way he wants to think about it. Be it through case classes, custom extractors for pattern matching, user-defined operators or just utilising existing Java libraries in more eloquent code, the language has charisma!
From expressiveness comes conciseness. Or at least, the option to be concise: it's for the programmer to decide whether to directly mirror the verbosity of Java, or to imitate Haskell's terseness. Unsurprisingly, most users will pick a happy medium!
Scala code is generally a boilerplate-free zone. There is often a tendency amongst certain developers to get a misplaced sense of satisfaction from writing boilerplate code: writing hundreds of lines of code without thinking about them seems like productivity, but it doesn't take a genius to work out that it's basically wasted time and completely contrary to the DRY (Don't Repeat Yourself) principle, maybe the single most important consideration in software engineering.
But why? If you find perpetual repetition in your code, it's most likely a deficiency of the language you're using. A few years ago, Jean-Paul Nerrière of IBM identified 1500 of the most useful English words and combined with a simplified English grammar he created a language called "Globish". It's
great because it makes it very easy for non-English speakers to communicate. But just imagine how frustrating it is when you want to talk about your cousin but have to refer to her as your father's sister's daughter every time you mention her. Like Globish, Java simply doesn't have the vocabulary or constructs to avoid this repetition.
What might take several lines of code in Java, or the use of a common design pattern, might be expressed in a single line of functional Scala code. This not only makes it quicker to write but also makes it easier to pick up someone else's code and understand it, not to mention old, long-forgotten (and of course undocumented) code of your own...
Let's not overlook the other edge of this sword: that with conciseness can come incomprehensibility: anyone who has dabbled with Code Golf will know the sorts of symbolic soup your code can resemble if conciseness is your only master. Scala is all about striking that balance.
Many beginners find static typing a pain: You're trying to get something working quickly, but the compiler is just whinging about the sloppiness of your coding. Sure, the code might have worked, but you wouldn't have known without actually running it. But "oh!," I almost hear you crow, "I wrote a unit test first to check that..."
I'm going to be blunt about this: can you seriously tell me you would rather write a unit test than have the compiler tell you your program is never going to work, so there's no point even trying (and by the way, your errors are here, here, here and here)?
After a while static typing will become the biggest timesaver in your development cycle. It's the crux of what gives Scala developers so much confidence to develop. Instead of developing in tiny iterative steps, Scala mitigates the risk of making tremendous refactorings across multiple crosscutting concerns. If you've been using Scala for long enough, you won't think twice before embarking upon a significant rewrite; knowing that the Scala typechecker is there as an immensely powerful safety-net gives you the confidence to naively shuffle code around, safely assured that – although the compiler might give you a long 'to do' list of errors – once you've fixed them, your code will just work.
Speed and Robustness
Scala's combination of expressivity and static typing means in practice that it takes far less time to develop working applications, from tiny toy code, to heavyweight applications.
From a business perspective, this means you have a choice of doing more with the same resource, to develop applications more quickly, or do the same with less.
The greatest efficiency and effectiveness gains are made with the addition of frameworks, customised libraries to your domain, a well set up development environment, a well structured and appropriately-skilled development team and robust architecture.
Native Scala libraries have been a while coming. This is possibly because existing Java libraries have been adequate for the purpose, but more likely because building good, reusable and effective frameworks is hard.
It's a common criticism of Scala that it's too complex. And it's true that Scala's feature list is not small. Most Scala developers probably use less than half of the features available to them. But unless you're building libraries or doing academic research, this is all that's necessary: all the productivity benefits stem from Scala's everyday features, not from the more esoteric details. If you don't need them, don't use them (and you won't even know they're there).
The Scala development team, ably lead by Martin Odersky, are very conscious of Scala's position at the intersection between cutting-edge academic research, and commercial pragmatism. Much thought has gone into the design of the language to make it coherent and practical across the length of the learning curve, by means of appropriate documentation and meaningful error messages.
In case you had made it this far without realising, Scala and Java are compatible. A fundamental premise of Scala's development has been full compatibility with Java. That is to say, you can use Java libraries in your Scala code, and you can use Scala libraries in your Java code.
But how is this possible, given Scala's vastly superior capabilities? Some ingenious ideas have gone into making Java's many warts evaporate when referencing Java from Scala. There's no need to cloud your mind wondering about special cases for primitives or boxing; Scala makes it an implementation detail.
But what about in the other direction? Well, unfortunately there's no water-into-wine miracle to suddenly turn Java into Scala just because you're using Scala libraries, so if you're calling Scala code from Java, you have to do it in the usual Java way, and you may need to know some implementation details. But let's not forget that compiled Scala is plain old Java bytecode, and the Java compiler can't tell the difference.
This should not be overlooked. Scala is simply a far more satisfying language to work with. On top of the added confidence you get as a programmer, there's never the sense that you're fighting the language to be productive; the code just flows.
The job of programming is fundamentally that of translating: programmers are given a human description of a problem which needs to be solved, and they convert that into a form the computer understands. Humans aren't getting any better at framing their problems in code, but Scala is making the computer better at understanding human language. This reduces the work involved in the translation; the steps necessary to encapsulate the problem as code, so the programmer can spend less time solving the syntax of the problem and more time solving the semantics.
Morale and productivity tend to be higher in Scala teams and the community is stronger because it's simply a more satisfying way to code, and seeing development happen more quickly makes the productivity more tangible to the development team.
We believe that all of the above virtuously conspire to give us an edge (as Scala application developers and consultants), and an edge to our customers (reliant on speed of delivery, speed to change, quality and cost effectiveness).
How do I get started?
Clearly there are a number of different starting points and each will have its requirements: a startup will have different needs from an established organisation's Java development team. However there are a number of common questions that occur: What do I do first?
Analyse and prioritise
Many organisations will have a backlog of development needs: new features, new applications, new releases, major and minor changes along with a pipeline of projects that are desirable but have not passed business-case hurdles. Whilst it is our core belief that in the long term most of these would be better performed in Scala, the key issue is one of prioritisation: where is the greatest benefit to be attained soonest, and with the least risk?
Sadly there is no easy answer to this question – we have developed a process, enabled by software (developed in Scala, of course) to facilitate a development team (senior technical managers, line-of-business owners, and developers) identify the best projects to start with. The key considerations are:
• Business criticality and impact
• Performance requirements
• Scala expertise
• Team development
Analysis of these factors allows us to produce a portfolio of candidate projects to migrate to Scala.
Assess and develop your organisation
You might have a great team of developers, but which of them have the motivation and propensity to become great Scala developers? Your prioritisation above will give you a good idea about which projects should use Scala first, and some idea of the resources you will need but what team will be optimal to deliver now and in the future?
Do you already have the Scala resources or are you re-training, hiring or seeking to bring in outside expertise to help? You may find that a lot of your Java developers are secret Scala devotees in their spare time.
Many organisations may choose to bring in Scala experts at the start to work alongside their Java developers, to help set things up, get things going and to act as mentors once the team is running on their own.
Create your development environment and process
Despite being relatively new, there are already a great many open-source frameworks, methodologies and tools on offer from the Scala community to help you develop your first Scala application. Each has its own merits and we don't believe there is a one-size-fits-all answer. You could ask one of your developers to research and recommend an answer, or you could seek advice from a Scala specialist to recommend the optimal toolset, environment and process based around your existing setup, requirements and future aspirations.
Position Scala within your environment
Most organisations have a heterogeneous environment involving a range of technologies which have been well integrated to fulfil their needs. These include databases, application frameworks, business applications, security infrastructure, web interfaces and multiple layers of messaging and communications services and protocols.
You will need to understand where and how Scala needs to integrate; how to achieve the greatest benefit from it; how to insulate your application from the future, and all the uncertainty that brings.
Operational management and support
Having reached the milestone of having initial applications developed and running, they need to be supported and managed. Scala fits comfortably into a Java environment, though there are still specific support considerations. Scala applications can be considerably more robust than other languages, though – like any system – require support and maintenance. This could be developed through an internal operations team, while second and third level support may be provided by the development team or a third party.
Quality assurance and audit
Any new Scala development, especially by an inexperienced team should be quality-assured and audited. Scala's flexibility offers many possible ways of implementing the same project, but each will have different performance characteristics, and some will be easier to manage than others. Scala goes a long way towards facilitating best practice in software development, but there's more than one way to skin a cat. It's not always clear which architectural approach is the most appropriate from a perspective taking into account long-term considerations like maintainability and future releases.
The success of many projects relies on the expertise and experience of external resources. Bringing in vital skills can be important at critical times on all projects. The Scala market is on the cusp of great things, but it's not yet as mature as the Java marketplace.
More mature Scala organisations
Many organisations have been using Scala for some time. Perhaps not for all their development projects but with some elements of Scala mixed with the major Java applications. Very few have the experience of evaluating how effective their Scala implementations actually are, how they are performing compared to how they could perform, how their teams are working and how efficient their development environment is.
We believe that developing in Scala offers significant benefits to most organisations, and for organisations already using Java, Scala offers low-hanging fruit as a boost to productivity.
These benefits are obtainable almost immediately but will be even more applicable in the long term as well. We believe that maximising the benefits of Scala implementation relies on the skills and experience of the team involved.
Scala Technology exists to help organisations maximise the benefit of their use of Scala. Working closely with Martin Odersky and the Scala team in Switzerland, Scala Technology promotes the use of best practice in implementing Scala in industry. Led by the team which launched the first commercial applications in Scala in 2004, our team has the tools, processes and experience to develop teams and applications that can truly take advantage of everything Scala has to offer.