SolarWinds hack and security – What is a software bill of materials?
We spoke with Jyoti Bansal, CEO of Traceable and Harness, Brian Fox, CTO and Cofounder of Sonatype, and Jeff Hudson, CEO of Venafi about the SolarWinds hack and cybersecurity. Learn about the security behind a software bill of materials, and what developers can do to protect themselves from cyberattacks.
JAXenter: Regarding the proposed executive order in response to the SolarWinds Corp attack: What would the international cybersecurity ramifications be?
Brian Fox: Cybersecurity adversaries and attacks are a global concern. When software developers build applications with known vulnerable open source components, or once safe components are later discovered to have vulnerabilities, everyone using the application is at risk. SBOMs establish a practice to assess, record, and track open source software components in applications over time in order to minimize the risk of successful breaches. SBOMs are especially important when identifying cybersecurity risks in critical application infrastructure across industries.
Although the requirements apply only to federal contractors, their influence will actually be far-reaching. With so many non-US entities working with, or hoping to work with the US, these practices, especially creating and maintaining SBOMs, will inherently become commonplace in at least portions of organizations around the world. While companies both in the US and internationally may soon be required to create SBOMs in order to retain US Government contracts, I believe organizations will quickly recognize their importance in not only cybersecurity but general software hygiene and will become a standard practice in any company creating software.
Jyoti Bansal: The international cybersecurity ramifications will depend on the specific details of the executive order. The executive order is focused on vendors selling into the federal government and requires a software bill of materials, breach notification and preservation of digital records with the goal of boosting security of the products and protecting federal agencies. From an international perspective, this places a high value on vendor transparency about their software and services, which may be a concern for multi-national vendors
who want to protect their internal ‘secrets’ from public disclosure.
This order also attempts to address supply chain security risks with the goal of lessening U.S. reliance on semiconductors manufactured overseas – the concern being the potential of backdoors inserted in chips before they are shipped to this country. The order will undoubtedly put more pressure on U.S. semiconductor manufacturers to raise production here in the U.S.
While companies both in the US and internationally may soon be required to create SBOMs in order to retain US Government contracts, I believe organizations will quickly recognize their importance in not only cybersecurity but general software hygiene and will become a standard practice in any company creating software. – Brian Fox
Jeff Hudson: We don’t know the specific details of the proposed order will be, but we can expect two things: it will likely have a multinational impact, similar to GDPR; and it will likely have a cloud-native perspective.
GDPR forced all nations to agree on a common response to data breaches, both inside or outside their borders, and this executive order will probably have similar implications beyond our domestic borders. In addition, the order will almost certainly address cloud-first technologies because they are being adopted so rapidly. An order that does not address the rapid changes to the software development process in cloud-native environments will have minimal impact.
JAXenter: The Cyber Supply Chain Management and Transparency Act of 2014 proposed that all software purchased by government agencies must have a software bill of materials, but it didn’t pass. What has changed since then?
Brian Fox: Since 2014, we’ve seen the “Heartbleed” vulnerability that impacted the entire internet infrastructure and the Struts vulnerabilities that led to breaches at Equifax and other organizations. In 2020, we witnessed over 1,000 attacks on open source projects and in the first quarter of 2021 over 5,000 similar attacks on open source components on global businesses like Apple, Tesla, Zillow, and many others. People now recognize that software isn’t being written from scratch, but created using custom-built code, commercial ready-made code, and open source and third party components. This means you don’t always know the origins of the components you’re using. This wasn’t understood by lawmakers or many executives 7 years ago.
There’s also been a steady drumbeat in D.C., with the NTIA’s Software Component Transparency effort to get companies creating SBOMs for products sold to the government and the FDA has also been very vocal about medical device manufacturers creating and maintaining SBOMs.
That said, the biggest difference between 2014 and now, is the 2020 SolarWinds breach, which highlighted that traditional security hasn’t been set up in a way to stop the growing threat on software developers and their infrastructure. The SolarWinds attack is an example of a bad actor poisoning the software supply chain to get at someone even further downstream by subverting the engine of trust regarding the origin and reliability of software. There is no way to get a handle on these types of attacks without SBOMs, and even that is just a first step.
Jyoti Bansal: A lot has changed in the last seven years — the adoption of microservices and cloud native applications has evolved creating an explosion of new software. The concept of a software bill of materials is great on paper, but in practice, the challenge will be keeping it current and in pace with modern software development approaches. Applications today are built using the practices of continuous delivery, where changes are released frequently and continuously. The executive order should address the fact that software vendors need to maintain a real time, current view of their software configuration, components, and APIs.
Jeff Hudson: Cyber supply chains have changed dramatically since 2014. Our economy has been digitally transformed and we are much more reliant on complex cyber supply chains. This shift is driving enormous innovation, but we are more susceptible to attacks.
If information security is a constant battle between control and accessibility, it’s clear the pendulum has swung towards the latter in the last 6 years.
JAXenter: Does a software bill of materials pose any security risks?
Brian Fox: No — it enhances security. A basic tenet of any security program is to understand the risks present in your systems and applications, then prioritize risk mitigation steps. For software, this requires organizations to have full visibility to all of the code in an application. An SBOM is the ONLY way to do this. Like an ingredient label for your software, an SBOM takes inventory of what is actually ending up in your application and provides a standard list of open source software components that make up 90% of a modern application today.
While fewer than 50% of companies produce SBOMs as a standard practice in software development, it is far from a novel approach. Without an SBOM, the attack vs defense posture is asymmetric. The attackers have the time, motivation and skills to determine what is inside software by means like decompilation and other advanced analysis techniques. The same is not true for the typical consumer of software. Whatever minor risk there is around an adversary finding the SBOM, is counteracted by the knowledge it gives organizations to then be able to rid the issue immediately.
Prescriptive regulation alone is probably insufficient. The answer will be having industry leaders adopt secure development practices and make security an unambiguous priority at all levels. -Jyoti Bansal
Jyoti Bansal: No that I can see. An SBOM is like listing the ingredients in a box of cereal or can of soup. The risk of not knowing about a potential ingredient is substantial. In fact, not having a clear and transparent understanding of the software configuration, components and APIs is a leap of faith on the part of federal agencies and software customers. The more transparent the software bill of materials is, the more it can be vetted, tested, and trusted.
Jeff Hudson: Depending on how we define software bills of materials, they could pose some security risks. A bill of materials can provide some assurance to end users. For example, vulnerable or previously compromised libraries or tools will become more visible and this could force us to accelerate our mitigation strategies.
On the other hand, a software bill of materials can allow attackers to extract useful information that they can then use in attacks. This is a little like burglars knowing which alarm company your house is using and if you are using contact sensors or motion detectors.
JAXenter: If prescriptive regulation is not the answer, what is?
Brian Fox: In an ideal world, companies would self-regulate their cybersecurity hygiene. Organizations and government bodies would give recommendations and guidance, but prescriptive regulation wouldn’t be necessary. But, in our software driven world, daily breach headlines indicate that government regulations might be a necessary motivator for action.
Passing the onus onto device manufacturers and organizations developing software to ensure that it is secure from the beginning and over time, reflects similar regulations guiding safety across other industries like auto manufacturing or agriculture.
If no other manufacturing industry is permitted to ship known vulnerable or defective parts in their products, why should software manufacturers be any different? In any other industry it would be considered gross negligence.
Jyoti Bansal: Prescriptive regulation alone is probably insufficient. The answer will be having industry leaders adopt secure development practices and make security an unambiguous priority at all levels. Accountability is another part of the answer — the cost of security breaches should be sufficient to motivate vendors and IT professionals to make changes to proactively detect and prevent more vulnerabilities. Bug bounty programs and responsible disclosure is another part of the equation, and both industry and legislation should
assertively support and reward efforts by the community to find and close vulnerabilities.
Jeff Hudson: The only way governments can help protect individuals and companies from becoming victims of insecure software build processes is to incentivize the software industry to build security into their development processes. While this executive order may help, it’s unlikely to really change the asymmetrical advantage attackers have. Ultimately, financial repercussions are the only thing that will force companies to act.
Based on what we know now, the executive order will probably have the unintended consequence of slowing software companies down and giving attackers the opportunity to innovate faster.
JAXenter: The President of Microsoft said the SolarWinds hack was not “espionage as usual” and represents a new kind of cyberattack. Can you expand on this?
Brian Fox: The SolarWinds hack was a sophisticated example of a bad actor poisoning the software supply chain to get at someone even further downstream; in this case it propagated downstream to approximately 18,000 customers. The attack started with bad actors intruding through malicious code that was implanted into SolarWinds Orion instances via trojanized updates. These updates delivered a backdoor known as SUNBURST and Solorigate, which were deployed on systems running Orion platform versions. Customers automatically pulled these malicious updates, and became a sitting duck. By attacking the SolarWinds software supply chain and mingling their malicious code with the legitimate, trusted code being delivered to their clients, attackers were able to cast a much wider net downstream.
Once malicious code gets into developers’ machines and build environments, it can end up in their internal corporate networks and in the product they deliver to all their customers.
Jyoti Bansal: The attack unfortunately represents a broad and targeted espionage-based attack on confidential information held by government agencies and other important institutions. The gravity and widespread nature of the attack clearly demonstrates that the impact of attacks by nation-states has reached a new level of risk. The fact that attackers exploited the software supply chain of critical organizations shows their determination and foreshadows that this will be a burgeoning attack vector going forward.
Jeff Hudson: This attack was definitely not espionage as usual. First, it was a long-game play – it took over 12 months to execute. Second, it ‘jumped left’ to circumvent typical security protocols by attacking SolarWinds’ software build environment. Third, it was extremely stealthy; attackers removed evidence of their attack after they planted the malware in the Orion software. And last, but not least, the real target of the attack was not SolarWinds, but SolarWinds’ customers – large technology companies and government agencies.
JAXenter: Between SolarWinds and Facebook, we are inundated daily with news about security and data breaches. What can the average developer do?
Brian Fox: This is a great question, as the remedy is only partly delivered by InfoSec groups. I would argue that
software supply chain hygiene, governance, and security have more to do with Development than InfoSec teams.
Here are a few things developers and development teams can do:
The only way governments can help protect individuals and companies from becoming victims of insecure software build processes is to incentivize the software industry to build security into their development processes. – Jeff Hudson
Improve the Quality of Artifacts Flowing Through Software Supply Chains. We’ve long reported that 1 in 10 open source components being downloaded contain known security vulnerabilities that can represent breach points for adversaries. Just as quality is inspected across supply chains of physical goods, its time organizations inspect and track the quality and security of components and containers flowing through software supply chains.
Map and Secure the Toolchains That Build Your Software. Next, you need to map out your build and release platforms. What’s in your DevOps pipeline tech stack and who has access to it? Is your infrastructure publicly accessible? How do you manage updates? The machinery and the code it produces should be tamper-free – meaning, they’re not erroneously injecting malicious code somewhere along the way.
SolarWinds isn’t the only company to experience a software supply chain attack that looks to be focused on the build process and perhaps even the build tools themselves. Before SolarWinds, in May 2020 there was Octopus Scanner which was caught by GitHub for having IDEs injecting malicious code as part of the build process. Similarly, Gitpaste-12 (which should now be Gitpaste-30) leveraged trustworthy sites like GitHub and Pastebin to host itself and maliciously infect users. Finally, publicly accessible build infrastructure has been exploited for several years for several purposes, like the Jenkins Cryptomining campaign.
In short, define your entire software supply chain (where do you source open source software packages, containers, IaC, software updates, build and release pipelines). Without this step, you can’t see, let alone prevent, attacks on your supply chain.
Jyoti Bansal: Shift security left, shield right — ensure that security is implemented in the software development life cycle instead of waiting to add in security after products are deployed into production. Developers should constantly be looking for vulnerabilities in the code for both internally developed and third-party software. As data breaches and exfiltration continue to be top concerns, developers need to understand the sensitivity of the data that the applications have access to and think of how to ensure only authorized users can see it.
Developers should also: implement zero-trust, perform extensive penetration testing and thorough code review of suppliers prior to implementation and, ensure that the loop between issues found in dev and staging are fixed promptly and make it to production by using strong CI/CD practices.
Jeff Hudson: Developers first need to be aware that these cyber espionage attacks are striking the fabric of software development – software build pipelines and environments. This means that instead of relying on single measures at the end of the process, security needs to be integrated throughout the software build pipeline
Development teams own the environments and processes used to develop software, so they have a responsibility to increase security into these environments. Since they aren’t security experts, they should consult with their InfoSec peers to find solutions that reduce friction for development teams and satisfy security and compliance requirements.
For example, digital keys and certificates, which serve as machine identities, are used throughout the software build process. One significant step software developers can take is to manage and protect these machine identities as a way to better secure their build environments.