Cenzic 232 Patent
Paid Advertising
web application security lab

Silver Bullet Metric

No, I don’t believe there is a silver bullet. But, I came up with an interesting thought exercise while I was at RSAcon that I like to call the silver bullet metric. I asked a number of notable security experts, vendors and analysis and everyone had almost the same reaction, which is that this is worth thinking about, but a hugely complex task to complete. So I thought I’d throw it out there and let the community think about it too. Let’s take a theoretical situation where we looked at any single security vendor out there and give them essentially as much money as they needed to do a complete global deployment of one of their security products. So if it was an anti-virus vendor, you’d give them enough to put AV on every desktop. If it were a firewall, it would be at every endpoint, and so on. Now, the metric is a combination of two scores a) how much is the total cost of ownership and b) what percentage of global online fraud has it decreased. Let’s take a few examples.

If you put Anti-virus on every desktop in the world, would you stop viruses from existing? I think any reasonable person who understands how viruses work would say no. It will, however, make the bad guys work harder and iterate faster to get by the filters (boutique malware). So there is actually a diminishing return once you get above a certain level of deployment. On the other hand, at the very lowest end, if only a few people had anti-virus they would be pretty well protected, because the virus authors wouldn’t bother trying to figure out a way around it. Of course everyone else who doesn’t have the AV is screwed in that scenario. So the right percentage of deployment for anti-virus isn’t global, it somewhere in the middle in that simple example.

If we’re talking about firewalls doing proper egress filtering, that would stop some worms from propagating, but it probably wouldn’t solve enough of the problems compared to the other options out there. If we’re talking about whitelisting applications that can run on computers, that would probably solve a much bigger percentage of the problems compared to firewalls, but the total cost of ownership is through the roof - and who is going to monitor and create all those whitelists. Eesh!

But back to AV for a second - AV has the hidden benefit outside of security that theoretically increases longevity of computers. So AV increases the lifetime of the computer, although the decrease in usability of the computer because of the resources that are being used might offset that number. Anyway, all of that factors into the total cost of ownership. Once we go through that exercise (which is probably best left for the product managers of each product line to do) you come up with a few interesting metrics. The first is the silver bullet metric, and the second is exactly what the maximum level of deployment that product or service should get to before it stops being an effective tool for the money - because TCO might change depending on how widely it is deployed as well (economies of scale, diminishing returns, etc…).

I’m not at all saying I have the right answer, or that I do believe there is a single best product out there, but to be the devil’s advocate, what if we did find that one product or service had the best silver bullet metric - what then? Why would we back any other technologies at that point? Anyway, it’s a fun thing to think about. Perhaps it’s just another lens by which to look at the security industry through. Of course this exercise has it’s evil twin too - which is the types of exploits that can be performed and their own associated cost benefit analysis.

13 Responses to “Silver Bullet Metric”

  1. LonerVamp Says:

    I agree, I think that is worth thinking about. Not just on a global scale, but also on a scale of a single enterprise and their own risk posture. I like the idea of thinking about full deployments where cost is not a factor. It puts our efforts into perspective quickly.

    It might be clearer for your metric if you scoped the goal. For instance, a goal of AV could either be wiping out all viruses or reducing fraud. The answers may be pretty different without a clear ideal goal.

    And like traditional crime, it might not be that we’ll ever achieve a goal of wiping it out, but if we can increase the costs such that some tool wipes out 90% of fraud, we create a much more favorable situation with cyber crime….hence your metric may have merit!

    One thing about silver bullets, and I’m jumping away from your term and back onto the more traditional view of a single perfect solution, is that if one pops up, it’s going to be an, “oh fuck me yes!” moment. Like that light bulb going off in our heads where something is obviously the solution and holy crap why didn’t we do this before? Even if something has a huge impact that is close to being a silver bullet, I think it would seem obvious to us experts.

  2. Dominic Cronin Says:

    I am reminded of http://en.wikipedia.org/wiki/Betz_limit

    Briefly, if there weren’t any wind coming out of the back of a windmill, it wouldn’t work, because the air wouldn’t be moving. This means that no windmill can capture all the energy of the wind going through it.

    It doesn’t mean that windmills aren’t useful, nor, in your thought experiment does it mean that av, firewalls etc. aren’t useful; only that they can’t capture all the energy.

  3. Sebastien Duquette Says:

    I’d like to know what makes you say that AVs could extend life of computers ? Maybe it’s because it’s late but I can’t figure out where that idea might come from. All that I can think of is that AVs restrict malicious usages of the CPU but even over large periods of time the gain would stay quite marginal. Can you shed some light on this for me please ?

    As for silver bullets, I can’t keep myself from thinking about the Mythical Man-Month when someone is talking about them. Brooks’ assertion that “no single development would achieve even one order-of-magnitude improvement within a decade in software engineering” still holds true today. Heck, if that wasn’t possible to MAKE software, how would that be possible to PREVENT it from breaking ? The first is a necessary condition for the second in my opinion.

  4. AbiusX Says:

    Hey there, It somehow right, But somehow wrong.
    Vendors already know all this, And that’s why they manufacture quite a few softwares based on a single code (e.g AVs)

    I think the solution to all that is OpenSource Software, Since it will eventually grow and flows would cease to exists on many systems, So there would be all levels of security on stuff.

    But with OSS, People should know things :D Not only use’em

    gl

  5. Kai Sellgren Says:

    Let’s imagine that we have ten apple trees. One of them produce bad apples that make you sick. Now, should ten children eat apples from one tree or should they all eat apples of different trees? If they all eat apples of the same tree, they might all get sick. If they all choose a different tree, only one will get sick. The question is: which is a better approach? Which approach produces less sick children? Statistically, they both are equal.

    In a world where one AV is used and installed on each desktop, we would have balanced the cost of security breaches a lot.

    Just my thoughts.

  6. LonerVamp Says:

    @Sebastien
    I read that part about AV extending the life of computers as being a perception thing. Many consumers and enterprise workers force the cycle of new computers because “they’re too slow.” Often, this is caused by malware taking up cycles on the system.

    That was my take on it, but I admit that’s me reading a lot between the lines. Rob may have had a totally different scenario in mind.

  7. RSnake Says:

    @Sebastien Duquette - I meant it not as a perception thing, but rather a lot of companies just force the computers to be completely re-installed when they find that a computer has been infected. So I really meant the lifetime of the OS install before it has to be re-installed. Sorry, I should have phrased that a little better.

  8. Rob Lewis Says:

    @Lonervamp(24th),

    Sorry Michael, when you say “Even if something has a huge impact that is close to being a silver bullet, I think it would seem obvious to us experts.”

    I don’t think that you “experts” would recognize the silver bullet if it fell on your heads.

    I will always remember Guy Kawasaki’s comment that “those on the first curve are unable to comprehend, let alone embrace, the second curve”. (Don’t let the security bozos get you down- the art of innovation key note)

    In my experience, I have seen that to be true. One tends to look for the same patterns that you are used to. Since we have a broken infosec model, why wouldn’t one look for something that works differently?

  9. Rob Lewis Says:

    Robert,

    Thinking about your post and some of the other comments, the following comes to mind.

    In a 2005 Security Focus interview Marcus Ranum stated the following:

    “To really secure systems, everything needs to be done 100% right at application layer, kernel layer, network layer, and at the boundary of the network. That’s a huge undertaking and nobody has made any effort to tackle it directly because the resulting system would probably be unusable.”

    So what products measure up using this criteria?

    If there is no one product, what would be the ingredients for such a silver bullet then?

    Shouldn’t a silver bullet:

    Work from the firmware layer to the user layer;

    Prevent unauthorized privilege escalation;

    Be least privilege, deny by default (white listing);

    Provide an authorization component post-authentication, or ZBAC access controls as Gunnar Peterson has been posting about;

    Be so fine-grained as to work at the document level;

    Have immutable audit logs;

    Bridge the gap between business operations and IT security policies so that there is more clarity and intuition in rule making;

    Automate administrative labelling functions according to the way data is used in the enterprise;

    Be cost-effective and manageable?

    Cheers.

  10. anonymous Says:

    There is not silver bullet, there is only silver bullshit.

  11. Rafal Los Says:

    Always interesting to hear what rattles around in that cavernous cranium of yours Rob. So what you’re essentially saying, in a round-about way, is much like Ranum said way back when - “Security is an unsolvable problem”… at least given the finite resource constraints we have to work with.

    Cool… at least we’ll all have jobs for the foreseeable future :)

  12. Brett Says:

    It seems that a critical assumption that is made is faulty. Early in the post, you state , “I think any reasonable person who understands how viruses work would say no. It will, however, make the bad guys work harder and iterate faster to get by the filters (boutique malware).”

    But would virus developers continue to develop? At some point all attackers, and computer scientists, are generally lazy. Attackers want to follow the path of least resistance. If every desktop in the world had Anti-virus installed, would the attackers actually want to “work harder and iterate faster?” or would they find some other way to make money?

  13. zuLu Says:

    @Brett

    “But would virus developers continue to develop? At some point all attackers, and computer scientists, are generally lazy. Attackers want to follow the path of least resistance. If every desktop in the world had Anti-virus installed, would the attackers actually want to “work harder and iterate faster?” or would they find some other way to make money?”

    “Virus developers.” Just like “Hackers.”

    Have lost there face in the sense that nothing is honorable anymore it’s not like the old days, where people did things for fun or did them for a name? Now it’s all about the money if we did have a “one in all” anti-virus that did what It was suppose to. I do believe people would skip the Idea of developing and go straight onto something else. Well of course until the “real” developers skip out of the wormwood and decide to try and develop something worth while.