Establishing Criteria for Vulnerability Management Solutions

Over the past several months, we have engaged with many organizations that are considering automating their vulnerability management programs.  Because many already have manual processes in place to manage their vulnerability data, they’ve been able to provide us with a tremendous amount of insight as to what functionality they deem most important in migrating to an automated vulnerability management system.Through these interactions, we’ve found ourselves in a unique position –  we are able to compile a consolidated list of criteria we’ve received from a wide variety of organizations, both large and small. We thought it would be worth sharing the top data points we’ve collected to potentially help you clarify your own requirements and objectives in this blossoming sector.

We’ve organized the list into three broad categories that security folks we’ve met with called out as being relevant to any automated vulnerability management solution. While the list of questions represent a varied cross-section of the marketplace, we’d welcome additional feedback/input from other security professionals out there as to what key questions or criteria you feel may have been overlooked.

Implementation

• What vulnerability assessment (VA) tools does the product integrate “out-of-the-box”?
• Which bug tracking systems & trouble ticketing systems does it work with?
• Is there an open API? How easy or difficult is it to work with?
• Is the product/service customizable?  Again, how easy or difficult is this?
• For large companies and complex environments, how well will the solution scale?
• How does the product/service manage, suppress, or possibly eliminate, the reporting of false positives? Are there ways to identify false negatives?
• Can the product/service correlate internal and external network address space in order to prevent the tracking of the same vulnerability multiple times?
• Can the product/service categorize reporting by business units, geography, departments, applications, etc. and then correlate vulnerabilities with those assets for tracking purposes? What other ways can business context be applied to the vulnerability data?
• How will the product work and fit into your existing workflow and remediation process? In other words, does it play well with others?
• Are industry standards supported such as: Common Vulnerability Scoring System (CVSS); Web Application Security Consortium Threat Classification (WASC-TC); Common Vulnerability Enumeration (CVE); Common Weakness Enumeration (CWE); Common Platform Enumeration (CPE); others?

Reporting

• Will it provide real time and trending reports that track open/closed vulnerabilities and the ongoing status of remediation?
• Can these reports be tailored and adapted for viewing by various levels of management, internal/external auditors, security specialists, etc.?
• Are your existing reporting and metrics tracked and supported?
• How does it reconcile the scoring and prioritizing of vulnerabilities given the variety of products and tools available that use their own scoring systems?

Key Benefits

• Will the solution accelerate the remediation and testing process?
• In the end, is this going to solve the pain we are currently living through?

We recognize there’s likely a wealth of additional criteria that should be included in defining the key characteristics of an automated vulnerability management solution, but we believe this list serves as a good foundation.  The information we’ve collected from organizations over the past several months and outlined in this post should be viewed as a “living breathing” set of criteria and a work in progress.  We look forward to obtaining feedback from folks in the security community who are as passionate about this topic as we are at HoneyApps!