The eternal vulnerability in the Software Supply Chain
It is not news that developers have been significantly using open source code, components and libraries for the last few decades. Enterprises are using open source code to help them bring code into production on time and on budget. Open source is a time and budget saver, enables code reuse and allows the developers to focus on delivering the required functionality without having to reinvent the wheel.
A recent study conducted by Sonatype, the company that supports the world’s largest repository of open source components and a leader in Software Supply Chain, strongly suggests that we live in a world of open source and we are not going back. Open source components are being downloaded more than ever before. According to Sonatype, in 2015, 31 billion Java open source components were downloaded from the Central Repository. Considering that it was 17 billion last year we see that the trend is accelerating dramatically; an 82 percent increase year-over-year.
Enterprises today create applications that use hundreds or more different open source components, libraries, frameworks, and tools. How many exactly? According to the report, last year, the average enterprise downloaded 229,000 Java open source components and the average application consists of 106 components.
However, we cannot reap the benefits of open source software without also introducing the risk of potentially unsafe code. If you have never considered how safe it is to use open source code into our applications then the answer may surprise you. According to the research, 6.1% of the total downloaded open source components included at least one known security defect. This means 1 in 16 Java open source components is known to be vulnerable.
Think of it for a moment. If you were a professional cook would you create a meal for your customers knowing that 1 in 16 of your ingredients was rotten?
The report analyzed 25,000 applications and identified that 6.8% of their components in use had a known security defect that could lead to successful attacks. Note that a single component may contain multiple instances of that known defect. For example, it is not uncommon for a software component to contain tens or even hundreds of SQL injection vulnerabilities.
The report continues its study and reports that each day 1,000 new projects are created and 10,000 new versions are released. With that rate it is certain that the average application will include more and more open source components by the time. Therefore, this further increases the chances of adding more vulnerable components in applications.
It is important to keep in mind that the statistics deal only with known vulnerabilities. The risk is even bigger if we take into consideration the unknown and zero-day vulnerabilities.
This is not a new problem
The problem is not new as several of these vulnerabilities affected millions of devices in the last couple of years in the form of the Heartbleed, Shellshock, POODLE and the Apache Commons deserialization vulnerabilities. What Sonatype’s report suggests is that the problem is far more pervasive than most people realize and it is getting worse as more and more open source components are released and are included in applications. We now have empirical evidence how pervasive problem this is.
It is reasonable to ask why these vulnerabilities end up unnoticed into the final applications. There are several reasons for this. First, there is a common belief that open source components are secure because they have already been used and thoroughly tested by the open source community. This misconception has been debunked not only by the many incidents that have been made public in the news (such as the ones described above) but also now Sonatype officially puts an end to this myth.
Moreover, developers do not consider any potential security implications when they add new open source libraries. Their decision is driven exclusively by the provided functionality. In most cases, developers do not have a security background. Additionality, enterprises that perform security code reviews for their own proprietary code do not perform the same level of security scrutiny for any included open source code. Even if it was desired to perform the same security code review for all included open source code, in most cases it would be impractical. Think of a CISO that has several hundreds of developers individually sourcing open source components. In such cases it is very difficult to manage all the imported code and properly audit it. And if big enterprises have the resources to audit some of this code, for smaller organisations this is completely impractical.
And if you think that this is the end of the problem, think again! Most of the open source components include other sub-components. So, when developers import a particular open source component into the application their sub-dependencies are also imported. These sub-dependencies could have their own defects that need to be managed and audited.
One might believe that we might be overemphasising the problem because there is a simple solution to the problem. Just update your open source components to their latest version and apply any patch. The truth is that developers are reluctant to update their dependencies because they do not contain only security fixes but also changes in functionality and their APIs that could impact the expected application behavior. And according to Sonatype’s report, older components (age 3+ years) are three times more likely to contain vulnerabilities. And the older the dependency the greater the risk.Over 10% of components greater than five years old had never released a newer version. And not all open source projects are equal. Some provide more frequent releases and bug fixes and some others have a very slow release process. Sonatype’s report provides evidence that 59% of open source projects with known security vulnerabilities in their dependencies were never repaired. The astonishing thing to note here is that these projects were constantly releasing newer versions of their components that improved their functionality but completely ignoring the security defects that in most cases were never fixed! What about the other 41% of the open source projects? These are considered to be the best suppliers of open source components. On average, how much time did the best suppliers need to fix security defects? Sit tight because the number might surprise you: 390 days for a security fix on average, according to Sonatype.
Thus, it is evident that once a vulnerability is introduced into an application via an open source component or library, most probably it will not be fixed for a very long time, if it ever gets fixed.
But how severe are these vulnerabilities after all? Can we live with them? Sonatype’s report also provides information for us in order to properly assess the severity of the problem. So, according to Sonatype, 70% of the components with known vulnerabilities had a Common Vulnerability Scoring System (CVSS) level of 5 or greater. Since most of these open source projects are heavily used in enterprises today and in mission critical systems, it means that as we speak these systems already contain 70% of vulnerable open source components whose CVSS score is 5 and above!
On top of that, open source components are not the only ones that find their way into the application. Countless code snippets copied off the Internet, copy/paste code from forums and websites like Stack Overflow without the code going through security assessment gets introduced into applications as well.
In other words, you cannot avoid introducing vulnerable open source components into your applications. Vulnerabilities were, are and will always be part of the open source software. And this is what makes them eternal! Vulnerabilities will never stop to be introduced in application via open source components and they will only be increased in number since new open source projects and versions are released at a wild rate.
What can be done about it?
There is no single solution to this problem. There are several actions that enterprises can do to solve this problem. Let’s call it the problem of Software Security Debt in the Software Supply Chain.
First, enterprises need to ensure that they audit in a timely and continual manner all the open source code that are using. This way enterprises will have an up-to-date insight of what are the known vulnerabilities that are introduced by the open source components.
Second, enterprises could try to allocate the resources needed to fix these defects in the open source components. But this obviously has a cost.
Additionally, enterprises could try to change the way they import dependencies, reduce the number of allowed open source projects, continuously track and trace the precise location of every open source component, perform security code reviews on each imported open source library, keep track of all the identified vulnerabilities and regularly assess and manage the risk of their Software Security Debt. In other words, define and enforce a strict supply chain management process.
And even if enterprises do all the above, each time a new vulnerability in an open source library becomes known, they have to go through a race to find which applications are affected, assess the risk, locate where the exact instance of the vulnerability is and implement a fix before any adversary exploits the zero day. And of course this is an endless loop as enterprises need to go through the same “fire drill” again and again every time a new zero day is disclosed.
In short, all the above are slow, very costly, error-prone, unreliable ways to manage the problem of the Software Security Debt in the Software Supply Chain
But it does not have to be this way. Using costly and unreliable approaches should not be the only way to mitigate this problem. You might be wondering if there are any tools to automate this process. The fact is that most security tools fail to solve this problem because they lack complete visibility of the application and its inputs as well as the fact that these tools often produce large amounts of false positives. A security tool that produces false positives cannot, by definition, be deployed in production in an unconditionally blocking or remediation mode, because false positives require analysis from human security experts.
So, any tool that produces false positives cannot improve the already slow, very costly, error-prone, and unreliable existing ways of dealing with the problem of the Software Security Debt.
Thankfully, a new security technology has emerged the last few years that can nicely complement an enterprise’s strategy against these vulnerabilities. The technology is called Runtime Application Self Protection (RASP) and is a complete game changer. What makes RASP a game changer is the ability to have full visibility of what is running on the application. RASP solutions operate at run-time which gives access to a wealth of runtime information and the ability to have full control over what is actually executed on the machine. In general RASP solutions could be much more powerful than Web Application Firewalls (WAFs) and Intrusion Prevention Systems (IPSs).
This is because, if implemented properly, a RASP solution can identify all exploits with a guarantee of zero false positives!
At first this might sound unrealistic or as an attempt to generate hype and buzz about a technology that does not stand up to its promise. However, even if it is hard to believe, in Waratek we work hard to achieve such astonishing results. The Waratek RASP container protects the whole stack of Java applications completely invisibly, with zero code changes, zero false positives, with minimal performance overhead and without changing the expected behavior of the running application under normal load.
RASP, a ‘must-have technology’ according to Gartner, is transforming the way we approach software security. By deploying a RASP solution as part of their strategic security management, organizations can gain the full benefits of open source without compromising the security of their applications. Using the Waratek RASP Java container, organisations can not only protect their software infrastructure from currently known vulnerabilities but also from as yet unidentified, zero-day vulnerabilities, with minimal configuration and zero operational maintenance.