By Dan Cornell
Jeremiah Grossman from WhiteHat Security recently posted a list of firms willing and able to fix vulnerable code for you. Denim Group was on the list - we've been doing software security remediation for quite some time now - and we really appreciate the mention. The point of this blog post is to start a discussion about situations where outsourced security remediation makes sense - as well as where it doesn't. We've had the opportunity to do a lot of software security remediation projects, and we've also talked folks out of using our services on more than one occasion where it wasn't their best way forward. Outsourcing can be a powerful option, but it is one that comes with tradeoffs.
The good:
- Security-Smart Developers - If you rely on an outside firm to do your security fixes you can (and should) expect that the developers working on your project are trained in secure coding, understand the issues they are fixing and are in a position to make recommendations on your best options for fixing vulnerabilities as efficiently as possible. Ideally all of the developers on your in-house teams would have these capabilities but the reality is that there is still a pretty significant knowledge gap in this area. A classic reason for hiring a consultant is to get access to skills that you don't have - or don't have enough of - in-house and this holds true in these circumstances. One important thing to focus on if you make the decision to outsource security fixes is what sort of knowledge transfer you should expect so that your in-house development teams can learn lessons based on past mistakes.
- Flexible Capacity - I've never met a development team that felt like they had too many resources or too few bugs or feature requests. Good developers are expensive and as a result most organizations tend to keep theirs pretty busy. In a case where you have to get things fixed on a specific timeframe, outsourcing can be a great option. In one organization, the development manager told me "All of my people are fully committed for the next six months; we simply don't have the time to devote to fixing these issues." That's a tough situation to be in and outsourcing security fixes can make a lot of sense if you find yourself in it.
The bad:
- Unfamiliarity With the Application - Though it might seem like a great idea to parachute in a team of secure coding ninjas just be done with it, you also need to remember that there is going to be a learning curve for any developer working on a new application. If in-house developers understand the application to be remediated then using an outsourced team might not make a lot of sense because you're likely going to have to pay them to get up to speed.
- Unfamiliarity With the Environment - Making changes to code to fix vulnerabilities is great. However you really only get the benefits of those code changes when those changes get pushed live. In addition to understanding the application, the people making security fixes are probably going to have to understand the environment in which the application is deployed so this represents another ramp-up cost that applies in outsourcing situations but might not apply if the fixes were performed by in-house development teams.
The ugly:
- Fixes Can't Be Done in a Vacuum - At Denim Group we use an outside firm to deliver bottled water and to fill up our water coolers and this is a pretty "hands-off" relationship. They show up from time to time, drop off bottled water and, I assume, they eventually invoice us and we pay them. Super-easy. Outsourcing security fixes is not like this. As mentioned above, the team making the security fixes is going to need information about the application they are fixing and they are going to need help getting updated code deployed. As an external developer, I don't want to have (nor should you want me to have) the root password to your production servers. Outsourcing security remediation can allow you to make a lot of progress in a short amount of time, but you will still need to devote some internal resources to making these projects successful. Ask up-front about how much time will be required from your team and what support you should expect to provide.
So when does outsourcing security fixes make sense? Here are a couple of example situations:
- Overwhelmed Development Teams - If you have vulnerabilities that must be fixed, either because of a compliance mandate or because identified vulnerabilities are so high-risk that you cannot let them remain, then it can make a lot of sense to call in a 3rd party to do your security remediation. It will cost you some money, but you can get things fixed quickly and you have minimal disruption for your internal teams.
- End-Of-Life Applications - We once did a security remediation engagement for an application that was 10 years old and had been end-of-lifed for the last five of those years. Anyone who knew anything about the application was long-gone and everyone was afraid to touch it. Outsourcing remediation made a lot of sense in that case because anyone would have had to learn how the application worked - why waste the time of the internal development team to get up to speed on an application they all hoped they'd never have to touch again.
And when does outsourcing security fixes not make sense? Here's a counter-example:
- Applications With Strong Agile/DevOps Practices - A number of Agile and DevOps practices such as continuous integration, automated testing and one-click deployment both help reduce the cost of fixing security issues as well as make it much easier to address security issues in-house. If environment set-up, testing and deployment are very low-cost for your in-house teams then the costs of outsourcing become even more pronounced.
There is more than one way to approach fixing security vulnerabilities and more than one way to approach outsourced security remediation. This list of pros and cons isn't exhaustive, but it should start to provide some background for making decisions. Also we've released a free HOWTO Guide that should be helpful whether you decide to do your remediation in-house or via a 3rd party:
Contact us to have a frank discussion about your options for getting vulnerabilities fixed.
--Dan
dan _at_ denimgroup.com
Comments