How Noise Filters Help You Fix the Vulns that Matter
Share with Your Network
Vulnerability scoring systems have been around a long time. But while they’ve evolved over time, many still lack grounding in rigorous statistics and data science and tend to suffer from the same problem: noise.
When it comes to vulnerability management, noise means too many false positives. This isn’t a “false positive” in the sense of identifying vulnerabilities incorrectly but rather prioritizing them incorrectly. For instance, when rating vulnerabilities on a scale of 1 to 10 (with 10 being most critical to fix), your risk scoring system might insist that 30% deserve a score of 10. At first, that might seem like a win. After all, having to fix three out of 10 vulns is a lot better than fixing all of them, right?
The problem is that our research shows scoring 30% of vulns as most critical may leave you running to fix way more vulns than you probably have to. (Real-world data compiled and analyzed by Kenna Security and Cyentia Institute shows that just 2% to 5% of vulnerabilities ever see exploitation in the wild.)
You can call it noise, or false positives, or whatever term you like. The truth is, it’s easy in a scoring system to bump everything up so all your criticals have high risk scores—and yes, this in some ways isn’t too bad—but wouldn’t it be more useful to filter out the noise and focus on the vulnerabilities that matter most?
Scoring risk: Comparing three approaches
The following figure compares three different risk scorings systems and how they distribute CVE scores. The three systems are: CVSS 2 (orange), a risk score from a leading scanning platform (red), and Kenna.VM (blue).
You’ll notice on the chart that CVSS 2 has many CVEs scored in the 90-100 range. Based on what we know about how many vulns pose an actual risk, that result represents a lot of noise—to the tune of about 5,000 vulns, or roughly 13% of total vulns scored, deemed the highest priority. Now look at the second set of results, the scores provided by the scanning platform. They do a somewhat better job in filtering out noise compared to CVSS, but you’re still faced with remediating nearly 4,000 vulnerabilities in the same category. So it’s a little better, but still not great.
If you’re not delighted at the prospect of remediating 4,000 vulns, you may instead want to study the distribution in the third set of risk scores. These scores reflect the reality that most vulnerabilities don’t pose a great risk, so most vulnerabilities earn a risk score of 40 or less. If you’re filtering out noise and focusing on the risk these vulns actually pose, the score distribution should be weighed not to the right as they are with CVSS 2 and the scanning platform, but to the left, as they are in the third grouping. By getting the criticals right and by almost completely eliminating the noise, you’re left with just a couple hundred vulnerabilities to patch, not thousands. This has the effect of bubbling up the most critical and risky vulnerabilities to the top.
Less noise, more value
Noise reduction benefits are substantial, primarily because they mean your remediation efforts will be that much more efficient. This increases your ROI in your vulnerability management solutions by allowing you to spend fewer resources to close open vulnerabilities and therefore efficiently drive down risk. You save time and money.
In fact, this next figure shows you how noise reduction improves efficiency.
The chart above illustrates the measurable difference that noise has on remediation coverage—or the completeness of remediation. In this real-world test, the same three approaches were used to prioritize vulnerabilities determined to be high-risk: CVSS 2 (orange), that leading vulnerability scanning platform (red), and Kenna.VM (blue). The chart shows how many high-risk vulnerabilities had to be remediated to reach 50% coverage. To reach 50% remediation coverage using CVSS 2, the organization needed to remediate 17,279 vulnerabilities. Using the scanning platform, there was a slight improvement—remediating 15,214 vulnerabilities was necessary. Using the risk-based Kenna.VM platform, however, just 627 vulnerabilities had to be patched to reach 50% remediation coverage. Clearly, focusing on the risk a vulnerability poses to you leads to better, more efficient remediation process.
Many vulnerability scoring systems are simply polluted with too much noise, leading IT and DevOps teams to remediate thousands of vulnerabilities that pose little to no risk while still missing some that do.
A proven approach to filtering out noise
Every day, we see how it is possible to filter out the noise that most risk scoring algorithms allow into their results.
Eliminating noise requires combining a large amount of sensor data, file-based malware analysis, and ground truth telemetry along with OSINT and other sources. We use this data to train and improve our models and then apply it to an environment using additional metadata about that environment. A key to reducing noise is incorporating contextual awareness—something other systems lack. All of this flows into our Exploit Prediction Model. This model takes into account all CVEs that are known to have been successfully exploited. But that’s just the beginning. Taking stock of this kind of contextual information allows you to measure the predictive accuracy of a scoring algorithm up to the current point in time (you can grade your algorithm), and measure how different variables contribute to risk—what we call their predictive capability. And it’s not just math. An example of a variable might be, “Does an exploit kit for a given CVE exist?” If so, that’s important, but it’s not everything. (This differs from common score bump methods that are too simplified and assign too much value to a single variable.) What’s needed is to ascertain how this variable contributes to risk. Has the exploit been observed in your industry? Are the assets hosting the vulnerability Internet facing? Do they host mission-critical applications?
All these questions have to be answered accurately, and using as much automation as possible. That’s how you produce useful, noise-filtered filtered fix lists of actual high-risk vulnerabilities in a timely manner.
In essence, reducing noise becomes a data science problem, because the analysis of large quantities of high-quality threat data (like the prevalence of CVEs in the wild) is key. It requires calculating the predictive capability of every variable used in prediction models. At Kenna, we use machine learning to build these models—in simple terms, we use this technology to find the best risk scoring algorithm using as much context as possible.
So it’s possible not only to filter out noise, but you can do so with accuracy. And the benefits include lower risk, lower costs, more optimal use of resources, and greater efficiency overall. After all, it’s coverage and efficiency that matter here.
Want to learn more? Review the Prioritization to Prediction series of research reports that we’ve produced with Cyentia Institute to learn how exploit prediction helps real-world organizations drive down their risk by focusing on the vulnerabilities that matter—and not the ones that don’t.