Denim Group, Ltd. Blog

Vulnerability Manager: Translating Security Vulnerabilities into Software Defects

By Dan Cornell

Integrating vulnerability scanners with defect tracking systems is how Vulnerability Manager got its start.  A HUGE problem we see with so many groups doing security testing (in-house or external) is that assessments/penetraiton tests/code scans are run resulting in the identification of a number of vulnerabilities.  These vulnerabilities are then stuffed into a 200+ page PDF report (complete with color graphs!) and that report is handed off to the development team.  If the secuity folks are particularly conscientious they put sticky notes on the pages with the vulnerabilities they really want to see fixed.

Is it any wonder developers react in a defensive way?

Our first solution to this problem was to make each security vulnerability a software defect.  That way we can communicate with developers “on their level”  Nearly everyone at Denim Group is a software developer by background, so, regardless of whether they are doing software development, penetration tests, or code reviews, they think in terms of software bugs (defects).

So a naïve approach would be to create a software defect for each vulnerability.  That way you could track the defects as they were addressed and make sure all of them had been resolved.  However, let’s look at an example of the 1:1 approach to translating security vulnerabilities into software defects…  If you find 100 XSS vulnerabilities in an application and translate those into 100 defects in your software defect tracking system, one of three things will happen:

1. The quality assurance (QA) manager will kill you

2. The development lead will kill you

3. The developers will simply ignore you

This highlights the challenge we ran into.  When we did application assessments we would find a ton of security vulnerabilities, but those vulnerabilities needed to be communicated to development teams as a more concise and sensible set of software defects.  Otherwise you will just aggravate them.  Sending developers 200+ page PDFs is a non-starter.  Sending them hundreds of new vulnerability/defects is also a non-starter.  However, packaging multiple vulnerabilities into a single defect seems to work for both security folks and developers.

In the Vulnerability Manager, now that we have all of our vulnerability data centralized and consolidated, we can do just that.  We support two operations with the defect tracking systems:

1. Create a new defect

2. Check the status of a defect

That way we can bundle vulnerabilities together to create a defect that appears directly in the defect tracking system.  And over time we can watch to see which defects (and hence vulnerabilities) have ostensibly been addressed by the development teams so they can be re-tested to validate the fix.

Currently the bundling is done manually – basically you pick the vulnerabilities you want to use to create the defect and check checkboxes.   This is all right, but can get cumbersome.  We are going to be adding the capability to group vulnerabilities for an application by combinations of type, severity, directory and file because those are the ways we have found to be most useful.  Check out the slide deck below for the pros and cons of different grouping approaches.

Also right now we are just providing information about the location of the specific vulnerabilities in the text of the defect – one or more AttackSurfaceLocations or CodeLocations, depending on how much information we have about the vulnerability.  The goal is to provide as much information as possible to the developer so they can track down the vulnerability quickly and fix it.  In the future we will also be including type-specific remediation recommendations in the defect text and this should also be customizable to the organization using Vulnerability Manager so it will communicate your standards for fixes to the developers.

For more information on where we are coming from, you can check out the slides from a presentation I did at the 2010 SANS Application Security Summit titled “Treating Security Vulnerabilities as Software Defects”

Treating Security Vulnerabilities As Software Defects

View more presentations from Denim Group.

 

Contact us for more information about treating security vulnerabilities as software defects.

 

--Dan

dan _at_ denimgroup.com

@danielcornell

Posted via email from Denim Group's Posterous

Vulnerability Manager: Tracking Attacks

By Dan Cornell

One of the things we added shortly before the Vulnerability Manager Technology Preview release was the ability to track attacks that were firing previously-generated IDS/IPS or WAF rules.  Right now this only works for generated Snort rules, but it should be pretty straightforward to extend to other supported IDS/IPS and WAFs.

Basically, each rule that is generated receives a unique identifier.  When the rules fire the rule identifier is included in the log message.  By uploading these logs to Vulnerability Manager we can parse out alerts associated with rules we generated and associate those with the original vulnerabiltity that spawned the rule.

This is handy because it provides some visibility into vulnerabilities in your applications that are actually being attacked.  Based on this data you might choose to prioritize certain vulnerabilities over others for code-level remediation.

Right now you have to manually upload the log files to Vulnerability Manager.  We have talked with other IDS/IPS and WAF vendors and one suggestion has been to add syslog support so that is something we are considering for the future.  We could also programmatically SCP log files off of sensors with a little bit of effort.  So although this is a manual process right now it should be easy to automate in the future.

Contact us for more info about prioritizing vulnerabilities for remediation.

--Dan

dan _at_ denimgroup.com

@danielcornell

Posted via email from Denim Group's Posterous

Denim Group at RSA 2010

By Dan Cornell

John Dickson and I will be at RSA 2010 next week.  I will be running around giving demos of our Vulnerability Manager and John Dickson is running a P2P session on getting management buy-in for secure software initiatives.  We will also be at a bunch of the after-parties for OWASP, WhiteHat, etc.  Hope to see folks there.

Here is the info on the P2P session:

https://cm.rsaconference.com/US10/catalog/profile.do?SESSION_ID=5963

Session Code:		P2P-106A
Session Title:		Secure Software Initiatives: How Do You Get Management Buy-in?
Scheduled Date/Time:		Tuesday, March 02 01:00 PM Burgundy Room 222
Session Abstract:		The vast majority of information that exists about software security either focuses on technical means to build more secure applications, or strategies to put controls in a software development process. Unfortunately, there is a dearth of information regarding how managers should push secure software initiatives forward, convincing executives that software security is critical to trusted business operations. What can security officers do to build consensus around security for applications?
Facilitator:		John Dickson, CISSP Principal Denim Group, Ltd.

Please contact us if you are going to be at RSA 2010 and would like to meet up.

--Dan

dan _at_ denimgroup.com

@danielcornell

Posted via email from Denim Group's Posterous

Vulnerability Manager: Auto-Generating Virtual Patches

By Dan Cornell

This is the next in a series of blog posts where we will be going through the internals of Vulnerability Manager as well as our future plans.  The hope is to explain the approach we have taken as well as solicit thoughts on improvements or different approaches we may want to look into.  You can submit bugs and feature requests here.

An advantage of having all of your organization’s application vulnerability information centralized and in a structured and open format is that you can start manipulating this data.  One useful thing you can do is to generate real-time protection for you applications that sit behind IDS/IPS or web application firewalls (WAFs).

Most IDS/IPS systems are very focused on network and infrastructure protection so making their rule languages work for web applications can take a little bit of work but it is possible.  WAF rule languages tend to have better native constructs for monitoring HTTP traffic so those rules can be a bit easier.  But basically, once you have an attack surface location (URL, vulnerability type, possibly a parameter) as well as an idea of what attacks looks like you can create rules to look for that sort of traffic.

When we were originally looking at this functionality I wanted to make it completely automated.  You find vulnerabilities, upload them to the system and – WHAM – virtual patches are automatically sent out to your IDS/IPS sensors or WAFs.  Once you have the rules you want to deploy this is pretty simple, from a technical standpoint: SCP the rules to a device, send a process a HUP signal and the rules get reloaded.  Or some XML-RPC variation on that theme.  All the software developers I described this to thought it was totally cool.  All the IT Operations and Security Operations folks I described this to looked at me like I was crazy.  Apparently most folks aren’t quite ready to take the human totally out of the loop in the virtual patching process.  Fair enough.

Based on that feedback the “tech preview” version of the Vulnerability Manager generates the virtual patches as text in the browser so that they can be copied and pasted into a separate file, inspected by a human, and then pushed live on the blocking device.  This doesn’t make for as cool of a demo as a fully automated solution, but it ended up being more usable for the way organizations are using these technologies today.

We’ve been talking with some folks who might be more comfortable taking the person out of the loop so down the road when people are more comfortable with the interaction between the different technologies we will look again at making this totally automated.

Contact us for more information about virtual patching.

--Dan

dan _at_ denimgroup.com

@danielcornell

Posted via email from Denim Group's Posterous

Vulnerability Manager: A Unified Approach to Vulnerability Data

By Dan Cornell

This another in a series of blog posts where we will be going through the internals of the Vulnerability Manager as well as our future plans.  The hope is to explain the approach we have taken as well as solicit thoughts on improvements or different approaches we may want to look into.  You can submit bugs and feature requests here.

As the name implies, vulnerabilities have an important role to play in Vulnerability Manager.  I posted earlier about our tracking of Applications in the system.  This post describes the Vulnerabilities that get attached to Applications.  Our immediate requirements were:

· Create a schema allowing us to import from all the leading static and dynamic analysis tools

· Be able to identify duplicate vulnerabilities (ie if the same results file with 10 results is uploaded twice there should be only 10 vulnerabilities in the system)

· Be able to support automated and semi-automated merging of vulnerability results from multiple tools

· Normalize imported vulnerabilities into a common classification scheme

· Normalize imported vulnerabilities into a common severity scheme

We started by looking at the sort of information we can extract from the scanning tools in order to determine the maximum amount of data we could extract – either directly or via some interpretation – from every tool.  This provided us with a baseline that can be extended with additional metadata when more data was available, but with a fundamental set of constructs we could design the major functionality around.

For dynamic scanning tools we created an AttackSurfaceLocation object with a VulnerabilityType, a relative URL and, depending on the VulnerabilityType, a parameter.  This is heavily web-focused and that is because most of the market-leading dynamic testing tools are specifically for web-based applications.  We may look down the road at incorporating fuzzers or other dynamic testing tools, but for now a web-based dynamic attack surface serves our needs well.

For static scanning tools we created a CodeLocation object with a VulnerabilityType, a code filename and a line number.  This supports both web and non-web based applications.  Many static analysis tools include source code snippets and column information with their results so those can be imported and stored, but they are not required for the basic operation of the system.  However they are required to do cooler merging operations later, so tools that provide more data in their output files tend to have better support in other areas.

We created a VulnerabilitySourceInfo object to describe a result pulled from a scanning tool.  Each scanning tool gets a subclass (XYZVulnerabilitySourceInfo) that must provide either a AttackSurfaceLocation object or a CodeLocation object (or optionally both, although we do not have any importers that provide both yet).

However, what the system actually cares about are Vulnerabilities so we created a Vulnerability object that has one or more VulnerabilitySourceInfo objects attached to it.  When importing a new results file, a series of VulnerabilitySourceInfo objects are created.  They are compared to all other VulnerabilitySourceInfo objects for all Vulnerabilities in the Application.  Exact matches – that is VulnerabilitySourceInfo objects of the same subclass with either an identical AttackSurfaceLocation or CodeLocation – are discarded because we have already seen a result from the same scanning technology for the same vulnerability.  If there isn’t an exact match already in the system then a new Vulnerability object is created and the VulnerabilitySourceInfo object is attached to it.

We also do some automated and manual merging where existing Vulnerability objects are matched up with additional VulnerabilitySourceInfo objects, but we will discuss the merge process in a later post because it is somewhat tricky and we are evolving the way we do this.  This is really cool when it works and we are looking forward to collecting more real-world data on success rates for doing this automagically.

With regard to normalizing the classification of vulnerabilities, we took the approach of creating an internal classification scheme that is basically a superset of all the vulnerability types supported by all the importers created thusfar.  When we add a new importer we either map its vulnerability types to existing vulnerabilities or add new ones for those not already supported.  All of this is done in an ever-expanding Java enum.

Obviously, this is a sub-optimal approach because it is proprietary, messy and more than a little bit brittle.  For future releases we are moving toward using the MITRE CWE with possible custom annotations where required.  This has the advantage of being a classification scheme that is well-known and accepted.  Down the road it should facilitate industry benchmarking.  Also we’re moving the implementation into the database so the data lives in the database and not in the code.  This work is currently underway.

Normalizing the vulnerabilities to a common severity scheme is also still in progress.  It actually isn’t in the “tech preview” release but we have it running on internal builds.  We went with a scheme where we classify vulnerabilities as High, Medium and Low (there is also an Info level for non-Vulnerability data stored alongside scan results).  Basically we just created a mapping between each supported tool’s native severity scheme to our High, Medium and Low levels.  This works, but we are not ecstatic about it.  Anyone have suggestions for a better scheme?  As with our move to CWE we would like something industry-recognized that can have a first-guess automatically pulled or interpreted from scan results, with the option to re-classify manually later.  Please feel free to suggest ideas here.

Future considerations include supporting imports from manual testing and providing better reporting and metric tracking.  Given our internal data structures none of these should be terribly difficult to incorporate – we just have some other things to finish up first.  Keep watching this blog for more posts about Vulnerability Manager internals, and please share any thoughts or suggestions.

Contact us for more information about finding and fixing your application vulnerabilities.

--Dan

dan _at_ denimgroup.com

@danielcornell

Posted via email from Denim Group's Posterous

Cybersecurity and Privacy "Great Debate" on Television

By Dan Cornell

Denim Group’s John Dickson was the conference chair for the Texas Lyceum conference on Cybersecurity and Privacy, held February 5^th and 6^th in San Antonio, TX.  One of the major parts of the weekend was a “Great Debate” discussion held Friday night.  That debate will be broadcast on a variety of television stations later this week.

Here is a video from John talking about the conferene, the “Great Debate” as well as other panel discussions held throughout the weekend:

And here is a list of scheduled times for the Great Debate being televised in various markets:

·         Amarillo > KACV > Thursday, February 18 at 8:00 p.m.

·         Austin > KLRU > Thursday, February 18 at 8:00 p.m.

·         Corpus Christi > KEDT > Thursday, February 18 at 7:00 p.m.

·         El Paso > KCOS > Sunday, February 21 at 1:00 p.m.

·         Killeen > KNCT > will air it in April

·         Houston > KUHT > Sunday, February 21 at 4:00 p.m.

·         Lubbock > KTXT > Sunday, February 21 at 1:00 p.m.

·         Midland/Odessa > KPBT > Thursday, February 18 at 8 p.m.

·         San Antonio > KLRN > Thursday, Feb. 18 at 8:00 p.m.

·         Waco > KWBU > Sunday, February 21 at 1:30 p.m.

·         Dallas, Harlingen and College Station are still looking for a place to schedule it.

Contact us for more information about protecting your organization’s online assets.

--Dan

dan _at_ denimgroup.com

@danielcornell

Posted via email from Denim Group's Posterous

Denim Group Sponsoring Austin Security Professionals Happy Hour March 18, 2010

By Dan Cornell

Denim Group will be sponsoring the Austin Security Professionals Happy Hour on Thursday, March 18 from 5-7 PM.  The happy hour will be held at Sherlock's Baker St Pub (183 and Burnet).  It's far enough from downtown that parking and crowds shouldn't be a problem, but close enough that you can easily attend the happy hour and still make it downtown for the SxSW shows that start at 8 PM.

View Larger Map

Please contact us if you are planning to attend.  Hope to see folks there.

--Dan

dan _at_ denimgroup.com

@danielcornell

Posted via email from Denim Group's Posterous

Denim Group Featured in CNBC Story on Cybersecurity

By Dan Cornell

CNBC did a story on cybersecurity and specifically how San Antonio is a leading city for cybersecurity professionals and firms.  They dropped by the Denim Group offices to speak John Dickson (about 0:50 into the video)

Also - enjoy the Texas-kitchy music and rodeo imagery.

Contact us for help addressing your cybersecurity concerns.

--Dan

dan _at_ denimgroup.com

@danielcornell

Posted via email from Denim Group's Posterous

Vulnerability Manager: How It Tracks Applications in Your Portfolio

By Dan Cornell

This is the first in a series of blog posts where we will be going through the internals of the Vulnerability Manager as well as our future plans.  The hope is to explain the approach we have taken as well as solicit thoughts on improvements or different approaches we may want to look into.  You can submit bugs and feature requests here.

Application portfolio tracking is pretty simple in the “Technology Preview” edition of the Vulnerability Manager.  Basically all we track are a name for the application and the URL.  And the only reason we need to keep track of the URL is to help de-multiplex results that come back from the WhiteHat Sentinel XML API.

Portfolio tracking will be expanded going forward to track important application metadata such as development platforms, characteristics of development teams, and sensitivity of data under management.  We are considering using a “tagging” approach to this – how Web 2.0!

Also currently applications URLs are basically tied to a hostname, but going forward we are considering allowing applications to be tied to a full URL with a directory.  This supports the way many organizations deal with their larger systems, so there can be separate “applications” hosted at http://www.site.com/Application1/ and http://www.site.com/Application2/

We have received questions from a number of folks asking if Vulnerability Manager is only for web based applications.  The answer is unequivocably “no”  It is true that some of the functionality of the Vulnerability Manager – such as virtual patching by generating Web Application Firewall (WAF) rules – is only for … web applications.  But most features – such as bundling vulnerabilites for submission to defect tracking systems and tracking team practice maturity – work for any application.  In addition to being able to import results from web application scanners we can also import results from code scanning tools that work for a variety of types of applications.  So no matter what type of applications are in your portfolio you should be able to derive value from Vulnerability Manager.

Any thoughts or comments on how we’re doing application portfolio tracking?  Please submit here.

Contact us for help enumerating and risk-ranking your applications.

--Dan

dan _at_ denimgroup.com

@danielcornell

Posted via email from Denim Group's Posterous

SANS AppSec 2010: Slides from Treating Security Vulnerabilities As Software Defects

By Dan Cornell

The slides from my lunch and learn presentation at SANS AppSec 2010 “Treating Security Vulnerabilities as Software Defects” are online here:

Treating Security Vulnerabilities As Software Defects

View more presentations from Denim Group.

We had some great questions during and after the presentation and I even had time for a brief Vulnerability Manager demonstration.  I also had an extended talk with two gentlemen from the State of California about freely-available tools to help .NET developers develop secure code.  I will pull all of that information into a separate blog post shortly.

Contact us for more information about getting security and development teams to work together.

--Dan

dan _at_ denimgroup.com

@danielcornell

Posted via email from Denim Group's Posterous