“Machine learning is taking us in a new direction”
The world of software development is ever-changing but even though new trends, frameworks and platforms appear almost on a weekly basis, some things never change. We invited Kevlin Henney, a UK-based independent consultant, trainer, reviewer and writer and speaker at Software Architecture Summit to talk about the “New New” and to weigh in on the growing interest in software architecture.
Kevlin Henney: William Gibson said that “The future is already here — it’s just not very evenly distributed.” We have a similar case in software development. Collectively we have the knowledge, we know to develop effective software effectively, but we don’t all know that we know and we don’t all know what everyone else knows. Knowledge is both our work and our bottleneck.
If software development were a more settled domain, the consolidation of this knowledge would be easier. But software development is turbulent and driven by forces external to software development, which makes it harder for things to settle. Rediscovery and reinvention of old ideas in both new and existing contexts is, therefore, more likely.
JAXenter: Do you think IT can someday be like a „natural science“ where mankind constantly accumulates wisdom without losing it in the next decade?
Kevlin Henney: I think that knowledge retention and propagation in the world of software is in one sense improving, although I would not necessarily compare it to the natural sciences.
Papers, blogs, source code, talks, books, conversations, classes, etc. are shared and available online. In software development, we have an economy of ideas that is based on knowledge and sharing. When I think back 25 years or so, it’s easier to see we are getting better at it.
Knowledge is both our work and our bottleneck.
More things are open and available, and knowledge is no longer as elite or insular as it was. That said, the challenge of knowledge propagation and its cultural retention relies on more than just mechanism: people and technology are also part of the equation.
Most of the world’s software developers are new. This has pretty much always been the case, but the numbers involved are larger than ever before. This means that experience is always very much in the minority compared to inexperience, no matter how available data, information, knowledge and wisdom have become!
Practical software development is also dependent on technology. Architectural options that simply did not exist in the past, or would have been impossible or difficult to bring about, are now commonplace, and many technical decisions that were common have fallen into retirement. In other words, the window of what makes practical sense is dependent on markets, hardware, other software, etc., and these are all in a state of motion. If you change bandwidth, performance, memory or any technical parameter by an order of magnitude, you are looking at potentially significant shifts in the software architectures that make sense. Systems whose functionality still makes sense, but whose architecture no longer make sense, tend to be called legacy.
And it’s always worth keeping in mind that software development, unlike the natural sciences, is a utilitarian endeavour: it is about creating things for use. What people, as consumers want is also a moving target, driven by familiarity and their own creativity, as well as the changing hardware landscape.
Yet underneath all this change and instability, we find an increasing number of ideas have a surprising stability and longevity.
JAXenter: The general interest in software architecture is very high – especially in Germany. Isn’t this a sign that we now have a more mature attitude towards software development practices and ethics, compared to past times?
Kevlin Henney: I would agree. There is a growth of interest on many fronts, from recognising software architecture as a first class concept to acknowledging that the process of software creation is not like the process of creating physical artefacts. Progress on all of these fronts seems to be driven, in part, by making better use of knowledge we already have for making the process of development more economic and less painful.
JAXenter: The microservices approach is highly debated nowadays. Is this an example of an old approach which has become the “New New”?
Kevlin Henney: It is not that microservices are an old approach that has become new again, but that microservices is the combination of a number of old approaches that have come together to create a new opportunity. All of the concerns and considerations they address have been considered in one paradigm or another over the past few decades.
Using OS processes as the principal unit of system composition, execution and protection dates back to the 1970s. I attended my first talk on service orientation twenty years ago, and its view of service granularity and design had more in common with modern microservices than what became popularised as SOA. The development questions of encapsulation, loose coupling, cohesion, heterogeneity, etc. have recurred across paradigms and technologies for a long time. Using distribution as a means of scaling and composition is also old. And so on.
Systems whose functionality still makes sense, but whose architecture no longer make sense, tend to be called legacy.
What has changed, though, and made this particular blend of these ideas popular and applicable today is that a number of costs have gone down — from financial costs to runtime costs — and many sources of friction have disappeared — from deployment to the stack you can build on. Add to this the demands placed on many modern systems, and you have a particular architectural blend that moves from possible to probable. That is what is new.
JAXenter: Do you see a modern architectonical concept which represents something really new to you?
Kevlin Henney: I think machine learning is taking us in a new direction, one fraught with ethical and engineering difficulties, as well as one marked with benefits and possibilities.
Deep learning is often considered a module or a feature of a system, but I think that it will make its mark on our architectural thinking over time, yielding approaches that defy many of our cherished beliefs about what makes an architecture good. Such self-adapting, highly coupled, incomprehensible architectures will have internal structures that are the products of dataset learning rather than human knowledge and deliberation.
JAXenter: Thank you very much!