Amrit Williams, BigFix CTO, discusses the role of education in security with Jack Danahy, founder of Ounce Labs, and now the World Wide Security Executive for the Rational division of IBM.

Subscribe in iTunes:

Subscribe with XML:

FULL TRANSCRIPT

Amrit Williams: Welcome! This is Amrit Williams, your host on Beyond the Perimeter, and today I am joined by Jack Danahy. Jack was the Founder and CTO/CEO of Ounce Labs, which was acquired by IBM in July 28^th of 2009. Jack, thanks for joining me today!

Jack Danahy: Happy to do so!

Amrit Williams: And today you are what is called the Worldwide Security Executive for the Rational Division inside of IBM. I don’t know a lot about what that probably means, but I imagine that means you do a lot of speaking, a lot of thinking, a lot of writing, lot of trying to get message out about security.

So talk a little bit about what your role is inside of IBM and what type of things you are trying to drive the market towards?

Jack Danahy: You are right, it’s all of those things that you described; there is quite a bit of education really that has to go on inside the marketplace. Security remains really amorphous, but rapidly growing space. New problem types are popping up all the time, new approaches to solving those problems arise, and lot of my responsibility is understanding how to frame those changes and those new problems and solutions in a language that our clients can understand.

Within Rational, many of the customers have typically looked at IBM to be their partner in developing applications, making sure that they work the correct way, making sure that they were developed efficiently, the productivity was high. The centralization of lot of the traditional development functions happen through Rational products. And so security is really a new entrant to a lot of those clients.

The security team has traditionally been pretty operational, and so as IBM clients and prospects begin their maturation into moving security further back in the lifecycle, don’t just test the product when you are about to deploy it, why not start building some security into it, well, those customers are looking for some information and some experience in terms of the best ways, the most efficient ways to introduce security earlier on as they are designing and building the applications that they rely on IBM to help them with.

And so my job is to figure out how do I integrate, how do we integrate security and security experience into the tools and the processes that IBM customers use to build things securely.

So the second piece is that this introduction of security in the development phase is most useful when I can mesh it with the existing investments people are making in operational security. There are a lot of excellent technologies that are already helping people with things like patch management and monitoring, authentication, access control; really more traditional security disciplines that are better understood.

And trying to help the people who are security folks understand the development teams, areas of responsibility, and their strengths and weaknesses, helps the organizations that we work with have a much better communication about security in general, not communication about security in two very discrete pockets; one being operational, one being developmental. And it’s my job to sort of help folks bridge that gap and share our experiences as IBM with them.

Amrit Williams: And it really is — today we are facing a pretty big challenge in the conversations being elevated into different aspects of the business, it’s being expanded out to different domain groups within an organization. And as we know, security has tended to have its own language and its own way of communicating either issues or problems or methods for resolution, those are not, as you had mentioned to me earlier, very digestible by the sort of the primary folks inside of a business that are trying to make the decisions, that have the funding or can actually effect change, and so there is a lot of need for that.

What are some of the ways that you are seeing that evolve better inside of the software development lifecycle as part of app development, how are you seeing that evolve and get to a point where folks in security and outside of security can actually have a comprehensive conversation about what needs to be done without too much fear and uncertainty and doubt?

Jack Danahy: It’s really about a common language. When development teams think about problems in the code, they are typically talking about bugs or flaws. When security folks are thinking about what can go wrong inside an infrastructure, they are thinking about vulnerabilities and exploits. And there really has not historically been a tight mapping between those two things.

The security team had so many responsibilities around operational management, of things like log files and system configuration, what have you, they were very busy there. And the development teams were typically being pounded for performance and functionality and getting things built on time. And so they really hadn’t ever taken the time to organize these into a single and comprehensive set of requirements for the development of an overarching view of what security would mean across both of these organizational boundaries.

And so what we have done is by understanding that the security problem evidences itself as a type of software defect, we can now communicate to the developer. When we tell them that there is a problem with this program, we can show them both what the programmatic issue is, i.e., sort of how they have done memory management or input validation, what have you, and they understand that really, really well.

(00:05:04)

But we can also help them understand what the net effect is of that, that will cause people to be able to do things they shouldn’t, or it will cause data to go places where it shouldn’t. So now the development team understands that the programmatic issues that they were having evidences itself from the security concern up top.

In the same way, the security team can understand which of the errors and the issues and the vulnerabilities that they see are really related to programming problems as opposed to system configuration problems or base level update sorts of problems. Now they can say, you know what, I see this type of problem, it looks to me as though it may be a validation problem or encoding problem, and now they can take that information, knowing that it can be characterized as something that exists within the software itself, and now they can communicate to development, that this is a concern that potentially the development team should be worrying about.

So number one, by sort of unifying the language of that conversation around terms like vulnerability, which are understandable from both perspectives; the perspective of a security team and the perspective of an individual developer perhaps, now I can get these folks to talk about it.

Mechanically, having tools which can introduce this language and these issues into both frameworks, into the capacity to translate a security finding of some sort into information that is useful for a security person and understanding its impact, and the same feature being introduced sort of non-disruptively into the development lifecycle, i.e., populating things like defect tracking systems, quality systems, into the developers’ desktops.

Well, now what I have done is I have created not only a common language, but common delivery mechanisms that they are used to, whether I am pushing data into reporting infrastructures with which the security team is comfortable and proficient, or whether I am pushing information results and recommended fixes done to developers in the IDE that look just like the recommendations that you get for quality related fixes. Well, now everybody is pretty happy, because I have taken security, I have given them the capacity to communicate in a common language, and I have introduced that information relatively non-disruptively into the interfaces they have been using forever.

Amrit Williams: And that makes a lot of sense. Do you find that — real quick, I think bridging that gap between how security can communicate with the folks on the operational side that actually have to effect change, whether we are talking about modifications to code or modifications or configurations to settings to network or computing devices, there has still been a pretty big gap there between how they communicate.

So definitely you need the ability to work inside of the ecosystems that these groups are both comfortable in, and a common language. Have you found that the resistance is stronger on one side versus the other, because they still tend to be pretty myopic in their views around what they need to communicate, and reaching out to the other side has always been quite challenging?

Jack Danahy: I think that’s really an insightful point, because I think on the face of it, everyone wants to do the right thing, and if I could, I would make little air quotes in the air, but they want to do the right thing when it comes to security. It’s hard to find an individual or an organization where they are intentionally ignoring security.

But the inside is exactly square on. The security team tends to believe that the development groups should just build things which are secure. They believe it is as natural as creating food that isn’t poisonous. So for them, we have seen some resistance, a little bit of washing of hands that says, listen, when that thing comes over the wall to me, it better be secure, because it’s my job to make sure that I am deploying secure infrastructure.

So security can, before they really get informed in this way, this mutual communication, they can say, my job is to find out if this thing is broken, and if I do find it’s broken, I mark it up and I throw it back over the wall, and I make negative comments about what’s going on in development, and it’s somebody else’s fault to make that right. So they draw this artificial line between the responsibilities of the people who are going to be creating the software and the people who are going to be responsible for deploying it.

In the same way, the security team can take a very defensive approach. We have seen this in terms of the early adoption of some of these technologies, where a development team who for the first time has an application reviewed by security, if what they basically get is a laundry list of why they are all stupid, they tend to resist. They tend to see this really as an encroachment upon their capability as developers, upon their ability to deliver what they are asked to deliver, and it becomes now really a pretty painful conflict.

The development team typically will say, and rightfully, they have built what they were told to build. There was no articulation of specific security requirements. There was no focus or exposition of how it would be tested for security, and so they feel they did their job according to the requirements they were given and the fault exists within the requirements.

(00:10:05)

The security team on the other side is holding up their end and saying, listen, we tested it, it’s too insecure to run, and now the organization finds itself in a pretty painful stalemate; and this is largely the result of a lack of education. In fact, not really at either of those two levels; it is not — the security team, it’s not their responsibility for being undereducated, it’s not the development team’s responsibility for being undereducated, there exists above both of them some organizational hierarchy, which was supposed to get them to talk in the same language.

Someone was supposed to say, this application has to be secured, and it means the following, and there is a list of things, and a lot of those things will be populated with information from the security team, so they have to participate in terms of setting that context and setting those requirements.

And then the higher level organization, as they are mandating what development will do, has got to sort of rearticulate those in development and design terms, so the team that’s actually building the software knows how to do it correctly, and where correctly means securely.

And so it’s actually that level that is sort of one above, if you think of it as being one above both security/operational and development/quality, that should have specified a language or a goal at least that both these organizations could understand, and then they work together to satisfy that higher level. As opposed to, I give one set of requirements to security that says, your job is to make sure that insecure things are not deployed. And I give another set of requirements to development that says, your job is to go build this functional thing. And then I sort of sit on top and I complain when there is a massive collision between those two sort of conflicting priorities for the groups.

Amrit Williams: Yeah. You said something very interesting, you said it has to be secured, it means the following things. I don’t think that there is almost any agreement on what those things are. I mean, I think there is the ideal and the ultimate of what they should be, but in the context of a specific situation, a specific application, given all of the conditions that, that application or the company finds it in, it’s very difficult to come to consensus there. And I think one of the — I do want to come back and touch on that in a minute, but I think one of the challenges is, is that traditionally, and even today, a lot of the output of the security tools tend to be fairly overwhelming, almost the shock in all reports, where it’s like this thing is so insecure, there are 1,500 ways to exploit it.

Development gets very defensive about that type of stuff, because it really is — not only is it pretty overwhelming for someone to deal with, it’s not oriented always to the resolution, it’s oriented towards the problem, which is generally where security likes to sit.

And I really like the concept and have always pushed for the concept of security needs to be part of an element’s lifecycle, whether it’s an application or a network device or a computing device, but we still really have a problem with language. We still really have a problem with the output of the tools themselves providing information that can be consumed by both the security professionals and the operational or development professionals that need to effect change.

What do you think is the — I mean, inside of the application development lifecycle you have mentioned a couple of things, but what do you think is going to radically change this, or are we just slowly going to trudge towards adoption?

Jack Danahy: I am hoping, there is always the inspirationally aspirational, it is my hope that this changes, because it’s something that is almost unrelated to technology at all, it’s acquisition behavior.

The same people who will be responsible for cleaning up the mess after a bad security incident, at some level, the same people who are responsible to shareholders for what happens as a consequence of something bad happening in security, are the same people who are budgeting the expense for new infrastructure and for new applications, what have you, and that is acquisition behavior.

So I think that if we can change the dynamics of security to being something which is much different than a technical approach to worrying about whether somebody did good memory management or not, and change it into something that says that security is absolutely as fundamental a characteristic about software as is performance or features, then we can change this very much. Because the acquisition, and by acquisition I mean it could be internal budgeting for resources to build something, offshore, outsourced, service managed, doesn’t really make any difference, but we can really change this by making security a characteristic by which acquisition is influenced.

So if I am the person who is buying a piece of software, I would never think to go buy Apple software, and I have a base of Windows machines, because everyone realizes it’s that sort of silly, or I would never think about something that was built to handle a 100 users when I have a user population of 10,000. There are different characteristics of products.

Security today has not really arrived there, and I think mainly because the impacts of security have both been a little bit muted, they have tended to be managed internally, and they have been obfuscated, the causes of the problems have been obfuscated to such an extent, it was harder for there to be a direct relationship between insecurity and inefficiency in cost and acquisition.

(00:15:13)

I think that this market changes and I think that people’s emphasis on security as a component of their infrastructure changes as it becomes the requirements of the acquisition lifecycle.

If I tell a vendor, I am not paying for this unless it is able to be asserted that it is secured, and I use the words carefully because you can’t really say certify, because it’s hard to find a certifying agent, but it has to be positioned in a way that is meaningful as secure, I think that that is where things change.

Amrit Williams: But that’s one of the — sorry to cut you off real quick Jack, but I want you to drill down on that. I think that’s one of the fundamental problems is, a lot of people don’t know how to describe or quantify what it means to include security.

When I was at Gartner we did — I put out a note that discussed, and I think this was in 2006 or 2005 or something like that, that by 2010 most organizations would include security as a critical requirement as part of software acquisition. But we still have not seen enough instantiation for organizations to be able to quantify what that means. They can quantify performance, they can quantify cost, they can quantify availability, they can quantify a lot of different aspects about what they are trying to acquire, but when it comes to security, there is a very abstract world out there where people just say, it needs to be secure, but very few people can actually say what that needs to be at any given point in time for any acquirer of said thing.

So how do we get past that and what do you think is really broken there that’s causing such a problem for people to be able to have a very easy conversation about what security is, because they have those easy conversations in almost every other aspect of software or networking devices or whatever it is.

Jack Danahy: I think the basic problem is one of quantification, that I can describe security qualitatively and I know sort of internally what I mean by it, and capturing that in contract language is really very difficult. As soon as I begin creating a specific list of security enablers, you will use encryption for this, you will use access control for that, you will update according to the following set of schedules, I am automatically creating a list of things they don’t have to do, because you are never, to your point earlier, you are never going to be able to fully enumerate everything that security means inside an application, even given a fair amount of context.

Where we have seen people be successful with this is where they tend to treat it more behaviorally. I will give you a great example that was done by one of our clients about four years ago.

They actually created contract language for customized code, in which they reserved the right to assess whether or not this application met the following set of criteria, i.e., user controllable behavior cannot cause misbehavior on part of the application, and there will not be malicious code anywhere that lives inside here. So the terms that defined what the insecurity was were fairly bland, right? They could have been applied in many different ways. And the terminology, which was exact, was around what would happen and the rights that were reserved to perform that level of assessment.

So from a purely contractual view, I believe that there was a small amount of extra money paid to get this into the contract, but at the end of the day there was full cost recovery, because what ended up happening was, there were penalties with it that said, if you agree to this, and we assert our rights to examine the code and we find anything which is bad, well then, you are not getting paid. And in fact, they were getting fined some percentage for certain types of problems and then not paid at all for others until they were remediated.

While the definitions of security were relatively planned and open-ended, the changes in the vendor’s behavior were very concrete, very measurable. They suddenly put into place methodologies through which they would first understand what the client was going to do in terms of their own testing internally and mimic that through their own organization.

They changed the priority of the way in which they addressed different kinds of issues, with an eye towards closing the security things, because a functional gap, you are allowed to patch. If I am contractually obligated to fix security things before I get paid, it changes the prioritization of those, and by defining the requirements for security in a relatively generic way, it really forced the vendor to do more process-oriented things, so that they could demonstrate what they had done. They could influence the perception of the way in which they had delivered a secure product and then they could have a much more harmonious relationship with their client, because there were no surprises at the end of the day.

So the client claimed full cost recovery against the means that they use for assessment, because they didn’t have the patching cost, they had a markedly reduced number of vulnerabilities discovered in user acceptance test. And the other great thing about it was, this was a custom version of a COTS product, and so every client of that COTS product also saw the benefits of this. So I think it was just sort of a win-win, but it changed the discussion from trying to be specific about security, to being specific about sort of legal remediation.

Amrit Williams: I like that Jack. I really appreciate you joining me today. Jack Danahy, Worldwide Security Executive with IBM’s Rational Division. Thanks Jack!

Jack Danahy: Thanks so much.

Announcer: You have just listened to Beyond the Perimeter, sponsored by BigFix Inc. Views expressed on this podcast are the personal opinions of podcast participants and do not reflect official positions of their employers or BigFix.

Thanks for listening.