Does technical debt still matter in DevSecOps?
Despite all the advantages of DevSecOps, challenges like technical debt still remain. In this article, Mike Bursell explains why this isn’t such a bad thing. After all, identifying a problem is half the battle. He also goes into the reasons why technical debt still exists, how it can be useful, and explores some basic do’s and don’ts for developers.
Before we go any further, I should be clear about something: I’m a big fan of technical debt. If you’re still reading after that revelation, I’m going to assume that you’re either on the same page as me or so horrified by the idea that you want to learn more, so let me explain a little. First of all: a guilty secret. I love technical debt. My reasons aren’t exactly altruistic, but they do play into part of what it means for me to be a security person. Technical debt will always keep me in a job, will allow me to poke my nose into interesting projects, and, well, there’s lots of it. There’s a better reason than those, however, which is that once the technical debt is acknowledged, we’re at least part-way to solving it.
Because, to be clear, if your project has technical debt (and here’s a clue: they all do), and it’s hidden, then it’s useless. It’s worse than that; it actually weighs on the project, as you can never easily know what you’re missing, but could have addressed. If it’s visible, however, things can change, and you can improve matters – if not now, then at least in the future. The reason for this comes down to the definition of technical debt. For me, technical debt comes in two forms:
- Stuff that we didn’t do, but which we should have done, and
- Decisions we made that could have been better.
If we know what we didn’t do, then we can do it later, when appropriate; if we know we made suboptimal decisions, then we can change them. We’ll come back to that “when appropriate” below; it’s important.
Technical debt and DevSecOps
We should also explicitly be talking about DevSecOps, because security is so often a hidden subject of technical debt. Let’s define DevSecOps: my preferred understanding of DevSecOps is the putting of security at the center of the practice, processes, and delivery of DevOps through automation, tooling, and cultural change.
Here are some examples of security-related technical debt that I’ve seen over the years:
- Neglecting to put authentication on currently non-public APIs
- Lumping capabilities together
- Hard-coding roles to initial expectations without thought to future requirements
- Compiling cipher suite selections into the application
All of these could appear in projects using classic development methodologies or more agile ones like DevSecOps. In fact, I would argue that in many cases, projects employing classic development methodologies are actually in a better position to remedy technical debt than more agile or DevOps projects. This is because there is generally a well-defined product management role that can collect customer requirements between releases, rank them, and decide which to prioritize for the “next release”. In the DevOps world, it can be easy for difficult features – or difficult decisions – to fall to the bottom of the “board”. Unless you consolidate requests, a focus on per-sprint features can obscure debt. These problems are compounded if you have swallowed the simplistic line that “with DevSecOps, security is everybody’s responsibility”, and therefore have no security experts embedded in the team.
As with more traditional methodologies, it remains the case with DevOps that security is complex and often requires deep knowledge. Technical debt can accrue easily particularly if you are asking people who are not experts to make decisions that they are not well-equipped to make. To take an example from above: how should someone with little knowledge of security best practices know when it’s appropriate to compile cipher suites into an application and when it isn’t? Technical debt is a security issue because:
- Security concerns are often non-functional and are therefore less likely to be tracked.
- Deep knowledge of security is scarce as there are too few owners, architects, and designers who understand the impact of security.
- Security is often left till last, so it’s easy for it to drop off the bottom of the TODO list.
So, let’s delve a little deeper into why technical debt can be an asset to a project, and more specifically, to an open source project. As noted above, when technical debt is acknowledged, informed decisions can be made to remedy that debt or to leave it as is for the time being. This is the point of the “when appropriate” comment above: sometimes, and however unpopular a view this may be, security needs to come second to usability, or a particular timeline. Or maybe your team doesn’t have the requisite expertise or knowledge about your customers’ plans to make an informed decision on design or implementation at this point. This is fine, but you should record the decision and why it was made because then you can revisit it in the future. What’s more, if this decision is recorded and the project is open source, it may be that somebody else who does have the resources, expertise, or knowledge of possible deployments will remedy the debt that you cannot currently manage.
A case study
It’s time for a case study of how technical debt can aid a project. Let’s consider that encryption was not included on a particular interface. What steps can we take to remedy this? The first is to document it, because then it can become known. There are at least four places in which we should document it:
- In project documentation, e.g. “We ran out of time, existing reqs don’t include certificate management.”
- In product documentation, e.g. “This API is designed to be used in a protected environment, and should not be exposed on the public Internet.”
- In source code, e.g. in-line comments:
// 2018-05-02 (email@example.com) Planned to use TLS 1.2 (check // for newer version) probably won’t need client // authentication. Don’t use ECC cipher suites. // Certificate management and revocation currently not specced.
4. In unit tests, e.g. a test that tests to see whether data is transmitted over the interface in cleartext, though it’s expected to fail.
How do these help? Well, the project documentation allows clearer requirements to be created, the product documentation allows customers to make informed decisions about how to deploy the application safely, and the source code documentation allows for simpler future implementation. The last – the unit test – may seem strange (who wants a test that’s known to fail in their project?). However, if you really want to ensure that a feature or piece of functionality is implemented, there’s nothing like a red light on a project dashboard to keep you honest. So, now we have documented known and useful technical debt.
Basically, what we’re trying to do is avoid blame (particularly when we’re the ones in danger of being the recipients). Good documentation should benefit multiple stakeholders, whether they are architects, designers, engineers, testers, operations, product managers, project owners, sales and marketing, support or customers.
What to do and what to avoid
Finally, what can you do to encourage a culture where technical debt is exposed and documented? Here are some “dos” and some “don’ts”.
- Encourage honesty.
- Document everything.
- Record “I suddenly thought…” moments.
- Reserve time in every meeting for discussion of technical debt.
- Consider a project “debt recorder” role – which can rotate through the team.
- Allow a culture of blame, even after the fact.
- Assume that it’s better to hide technical debt from the customer.
- Put off creating unit tests just because a component hasn’t been implemented yet.