A more complete description of the system and its capabilities is:
ThreadFix is a software vulnerability aggregation and management system that reduces the time it takes to fix software vulnerabilities. ThreadFix imports the results from dynamic, static and manual testing to provide a centralized view of software security defects across development teams and applications. The system allows companies to correlate testing results and streamline software remediation efforts by simplifying feeds to software issue trackers. By auto generating application firewall rules, this tool allows organizations to continue remediation work uninterrupted. ThreadFix empowers managers with vulnerability trending reports that show progress over time, giving them justification for their efforts.
So what’s next? You can download ThreadFix here. The current beta is available as a pre-built Tomcat install (the final release will likely be a pre-configured virtual machine). Just unzip and run. During the beta period we will be looking to push new versions on a weekly basis. Also we have a “Getting Started” guide that runs through the major functionality as well as other documentation on the wiki to describe the different parts of the system.
There is still work to do – we need to improve the stability of the scan importers, we need to tighten up the security of parts of the system, and we need to build out the REST API. If you run into any issues using ThreadFix or if you have feature requests please use the online bug tracking system here or email me at the address listed below.
Also you can follow ThreadFix on Twitter: @threadfix
I sat down witih Jeanne Friedman last week and recorded a short podcast preview for my upcoming presentation at RSA 2012 “Remediation Statistics: What Does Fixing Application Vulnerabilities Cost?” You can see her post online here and you can download the 8 minute podcast here. The actual session is ASEC-302 and will be Thursday, March 1st, 2012 at 9:30am in Room 302. Hope to see folks there!
The good news is that if you got your stakeholders aligned in an Inception phase and if you know what you are going to do and how you are going to do it from a Planning phase, actually fixing the vulnerabilities is usually pretty easy: you just do what you said you were going to do. That said, there are some potential pittfalls.
For starters - you can’t always just start fixing vulnerabilities. You first have to have a working development environment where you can make changes and test them. For applications being actively developed with good configuration management this should be pretty easy. For the 10 year old end-of-lifed ASP Classic application that no one has touched in five years it can be more challenging. This can be a huge time waster and wading through a couple of messy projects has renewed my belief in the value of configuration management and environment documentation.
Also once you fix vulnerabilities you have to both make sure your fixes worked and make sure you didn’t break the application. The way of the world is that the last person to touch the code is assumed to be responsible for anything that goes wrong, so you want to be sure to be able to demonstrate that security fixes aren’t responsible for any downtime. If your application has a suite of automated unit and functional tests they will really be paying dividends at this point. If you agreed that a “fix” would be considered successful if it no longer turned up in a scan then you’ll need to re-run a scan. If your fix stanards were more involved then you will need to runt through and confirm that the vulnerabilities have been addressed. Again – automation can be a big help for certain types of vulnerabilities.
Finally you can’t declare victory until your updated – and tested – code is pushed into production. Using an existing update window is preferable or you may have to do an out-of-cycle release. The important thing is that the code goes live. If vulnerabilities are supposed to be fixed and the next round of testing from your external auditors indicates that they haven’t been you should expect to get a nasty phone call. So far every time I have seen this situation in a well-run remediation project it has been because code updates or mandated configuration changes had not actually been pushed into the production environment.
Here is a short video where we talk a bit about the Execution phase for software security remediation projects:
Here is a presentation outlining some of our research on the actual costs of remediating different classes of vulnerabilities as well as where time gets spent during the execution of remediation projects:
You can read the full HOWTO Guide for Software Security Vulnerability Remediation here:
Normally information security creative writing is reserved for marketing brochures talking about Advanced Persistent Threats, but I thought I would share a little something I’ve been working on. This is an excerpt from an infosec romance novel I have in progress titled “The Remediation Diaries”
After we sent out our assessment report we hadn’t heard a peep from the application team until Alice the development manager came to me and complained one of her developers, Bob, was behind schedul and had been “screwing around” all week doing some sort of “security crap.” Wandering the halls until I found him, a premonition hit me that the conversation was going to take a frightening turn. I felt a hollowness in the pit of my stomach as I looked into the puppy dog eyes of a developer who wanted nothing more than to prove his competence and win the aproval of the security team.
“The report was pretty long and a little confusing, but in there it said the XSS vulns were because we weren’t escaping text before we sent it to the browser.” I breathed a small sigh of relief. At least Bob read the report and seemed to understand the important parts. Things were looking up. “So I wrote an ‘escape’ function that turns all the left and right brackets into the escape sequences.” My heart started racing – did Bob really think that was a sufficient answer? “Then I went around and pasted it in the code to fix a bunch of the pages – I think I got them all.” My throat closed up. This was not going well. “I didn’t want to bother you so I went and did that for all the injection vulnerabilities, too. Can we run another test?” My heart stopped beating and I was convinced I was in full-on cardiac arrest. I smelled toast burning – was I having a stroke?
I struggled to keep my cheerful poker face. What was the kindest way to handle this? How could I encourage Bob’s enthusiasm without crushing his soul? My brain was shouting: That’s a crappy way to make an HTML escape function! And it doesn’t work for HTML attributes! And why would anyone think that HTML escaping was the same as SQL escaping? And why wouldn’t you just use parameterized queries to fix SQL injection properly? Why didn’t they know how to fix this stuff right the first time? If only I had helped the developers plan a little bit before they ran off on their own…
[Side note: What do you think? Do I have a future as a novelist? Or should I stick with my day job?]
On a more serious note, this story typifies the types of unproductive efforts we see with poorly-planned remediation efforts. Just as the Inception phase is needed to make sure that the stakeholders are on the same page, efforts needs to be planned so that there is unambiguous agreement on issues such as:
Which specific vulnerabilities are going to be fixed and when?
Specifically how should the vulnerabilities be fixed?
What sort of follow-on testing is going to be performed and what is the standard by which vulnerabilities will be considered to be successfully addressed?
Developers are smart people. The vast majority of them also have the best of intentions – they want to write good code and they want to write secure code and, if given the choice and the time, they would prefer to do things the “right” way. But if you rile them up and don’t provide guidance – especially when it comes to unfamiliar territory like security fixes – you are likely to be disappointed with the results. And if you are disappointed then you can be sure that development managers and other stakeholders are going to be both disappointed and angry. In addition, they aren’t going to be interested in funding your next remediation effort. Up-front planning and explicit guidance means the difference between successful vulnerability remediation and ineffective time-wasting so spend the time with developers to get this right the first time.
Imagine this scenario: Your development team builds an application and puts it into production. Down the road, a customer asks you to do a security assessment. You run a scanner against the application and perhaps even do some manual penetration testing. The result is you end up with a long list of vulnerabilities and the customer wants them fixed. So the security team meets with the development team and the exchange goes something like this:
Thanks, security team! Very helpful! Come back any time…
If you’re going to spend the time and resources diverting development teams from building new features to fix security vulnerabilities all parties involved owe it to themselves to make sure the effort is successful. Based on our experience doing software security remediation projects the ones that are approached in a thoughtful and structured manner tend to do far better than ones based on a mandate of “FIX IT!” We’ve developed a HOW-TO guide for software security remediation projects outlining just such a structure, and these projects start with an Inception phase.
The Inception phase is used to get all the stakeholders together and on the same page. Software security remediation projects are typically software development projects, not security testing projects and they need to be estimated and project managed as such. They also force people from different parts of an organization with different goals to work together. Before moving forward, teams need to agree on things like:
Approximate budget and where the budget is coming from
Desired (but realistic) timeline
Specific compliance or audit issues that must be addressed
Initial project success criteria (“fix all the CRITICAL and HIGH vulnerabilities” or “fix all public-facing cross-site scripting”)
Given this shared understanding the involved parties can start to work on planning the actual remediation effort, but in the absence of a consensus the remediation project likely does not have a clear mandate and this is a recipe for project failure.
Here is a short video talking a bit about the Inception phase for software security remediation projects. Hopefully you find it to be a bit more constructive than the previous one.
Also, you can read the full HOWTO Guide for Software Security Vulnerability Remediation here:
There is a lot of focus in the security industry on finding vulnerabilities so you see a lot of material out there about penetration testing, fuzzing, static analysis and so on. That is all well and good but with so much focus placed on finding vulnerabilities a lot of organizations forget about the need to actually fix vulnerabilities. We do a lot of software security remediation work at Denim Group and we’re trying to start to make more of the lessons we’ve learned publicly available to make it easier for organizations to fix the vulnerabilities they find in their software. This is the first in a series of blog posts about a framework we’ve created for software security remediation projects. You can find more in-depth information available in the HOWTO Guide we put together on the subject.
You can read the full HOWTO Guide for Software Security Vulnerability Remediation here:
As we have posted previously, there is data out there you can use in planning your remediation strategy. We are hoping that if organizations have a structure for their remediaiton projects as well as data-backed estimates for the level of effort involved that it will be easier to secure budget for these important initiatives. We’ll be following up with more in-depth information on the different steps in the remediation process as well as some tips on practices we have found to be successful.
We are starting to see some data from the industry about how long it takes most organizations to fix vulnerabilities they’ve identified in their software. Two useful sources are:
These provide information about the prevalence of different types of vulnerabilities as well as how how long vulnerabilities tend to stay in software and is a reflection of the calendar time that these vulnerabilities exist. At Denim Group we have also released some of our data on how long it takes to fix different types of vulnerabilities and our data reflects the level of effort required to make fixes. The data we released can be found online here:
The combination of these types of data sources should be helpful for organizations trying to craft a strategy for addressing vulnerabilities in their software. The data about vulnerability lifespans can help you to benchmark yourself against industry peers and set goals for what sort of exposure window you are willing to accept in your organization (although I would argue that software security vulnerability lifespans are far too long right now). The data about the level of effort required for fixes can help you to plan the resources required for remediation projects. Availability of data sets like this allows security analysts to have “grown up” conversations with management. Think along the lines of “to keep pace with peers in our industry we should be doing these things…” versus “cross-site scripting is scary…” More “grown up” conversations should lead to better-allocated budgets and ultimately to better-managed risk.
The slides from my SOURCE Boston 2011 presentation “The Real Cost of Software Remediation” are now online:
We’ve been doing remediation work for a number of years so I’m happy to start talking about more of the things we’ve seen. I’ve maintained for a while that finding vulnerabilities is usually pretty easy and that fixing vulnerabilities is where organizations need to focus more effort. Hopefully some of the lessons we have learned will help other organizations start to plan and execute remediation projects of their own. I think the remediation project framework we put together, when combined with some of the remediation statistics we are releasing should help. We have these and a number of other resources in our online Remediation Resource Center.
As expected, the Security BSides Austin conference was a blast. There were a lot of great folks in attendance and plenty of great talks to go around. And of course there were Hackers on a Duck Friday night. Take a look at Jack Daniel’s photos from BSides Austin to get a feel for the event.
Everyone loves the coffee truck servicing Austin Security BSides
For some reason (mostly thanks to Ben Tomhave’s trickery) I ended up doing a number of talks during the conference. The slides should now all be up online:
John Steven lays down the law about Threat Modeling
I really enjoy the BSides conferences. They’re less vendor-driven that some of the big security conferences like RSA and BlackHat. The talks also tend to be a good mix of in-the-weeds technical topics and interesting perspectives on other security industry topics. Hopefully future BSides Austin events will be further integrated with South By Southwest (SxSW). Getting the security message out to developers and social media
There are a tremendous number of resources available for organizations wanting to test the security of software or build secure software from the ground up. A critical area that is often overlooked is how organizations can address the existing vulnerabilities in their software portfolios.
The guide lays out a project structure for remediation and has tips and recommendations drawn from our work with a variety of organizations struggling with huge volumes of security vulnerabilities. It steps through the major phases: Inception, Planning and Execution and looks at what teams can do in order to most efficiently reduce their exposure and risk due to vulnerable software.
