Cenzic 232 Patent
Paid Advertising
web application security lab

Mozilla Says “Ten Fucking Days”

I’ll do a more thorough writeup of the craziness that is Blackhat, but this I thought should go out ahead of all the other stuff. I don’t have a lot of time so I’ll try to make this story short. Two days ago after Jeremiah and my talk (you can get the slides off of the WhiteHat site) a number of people from Mozilla came up and said they wanted to talk more about the issues we were finding and other suggestions we might have (I’m going to write this part up more thoroughly later in a separate post as well). We were also invited to the Mozilla “milk and cookies pajama party” which is pretty much exactly as it sounds.

We showed up, and nearly immediately I was surrounded by the bulk of the Mozilla QA and security team that was attending Blackhat. They asked me lots of questions, and gave me lots of info. It was a pretty equitable trade of information. Clearly, they acknowledge that they need help from the community but they also feel confident that once things come to their attention it’s simply a matter of days to close their holes. They said the recent rollouts were actually slower than they would have liked them to be, even though they were only a week and a half apart. Further, they said that they could roll out any critical patches within 10 days. Not one to let challenges go untested I called BS.

At this point Mike Shaver threw down the gauntlet. He gave me his business card with a hand written note on it, laying his claim on the line. The claim being - with responsible disclosure Mozilla can patch and deploy any critical severity holes within “Ten Fucking Days”:

I told him I would post his card - and he didn’t flinch. No, he wasn’t drunk. He’s serious. I’ve always been a fan of Mozilla and Firefox however this is a pretty bold claim for a company of any shape or size. I shopped the business card around to some various people while I was at the Microsoft party the next day to get people’s reaction. The consensus was that it was funny, very difficult to achieve and in one case, one of the head guys of security at Amazon simply doubted that the patches would be of sufficient quality. I’m not going to comment on my personal feelings on this matter except to say that I’d love to see Mozilla back up their promise.

58 Responses to “Mozilla Says “Ten Fucking Days””

  1. Lurker Says:

    What gets measured gets improved … of course, 10d > 0d.

  2. Nick Says:

    They’ll pull it off.

  3. Abe Says:

    This is almost meaningless…

    Yes, I believe Mozilla could usually come up with a good, tested fix and release in 10 days. And they’ll do their best to come up with a fix as soon as possible.

    However, bugs can be arbitrarily difficult to diagnose and fix. When you combine this with the issue being in code that is only understood by one or two people (some areas of the layout engine seem to be like this), and combine this with the fact that sometimes people happen to be unavailable to fix things due to vacations etc., it becomes clear that you simply cannot always roll a fix within an arbitrary, short time period.

    Rather than get fixated at ten fucking days I’ll be confident knowing they’ll roll a fix as quickly as they can, which is usually very quickly indeed.

  4. Spyware Says:

    Rome wasn’t build in one day, but heck, firefox isn’t rome. And Mozilla has ten whole days. I don’t know, put 20 geeks in front of a computer for ten days and just watch them go.

    It could be done, depends on difficulty of the issues to fix though. I’d say give them ten days, if they patched, hail for mozilla, if they didn’t, laugh out loud, ask for cookies (and your money back).

    You should’ve made a bet.

  5. lucho Says:

    and Bill say’s “one fucking service pack”

  6. 0x000000 Says:

    I think ten years isn’t enough, look at the history. I don’t believe in “fixes” nor “patches” cause some where you’ll open up another hole. It’s hilarious, really. It’s okay to claim all that alleged FF security stuff, but I think Mozilla does a terrible job. I’m not ready to give into MSIE7.0, but I am close, because they have a few darn good features.

  7. zoiz Says:

    Really hope that they can produce something ‘new’ :)

  8. Dan Veditz Says:

    A release is much more than a guy banging out a patch and testing it. There’s a lot of time spent building and sanity-checking all the pieces that go into a release: 44 languages times 3 platforms is 132 packages to build and test, plus another 132 binary diff update packages and another 132 “full” update packages. And then all the update.rdf files served by aus.mozilla.org telling clients the update is available and where to get it.

    Sure, lots of that is automated, but it still takes a lot of time and you never quite trust the process not to glitch somewhere.

    For the Firefox 2.0.0.6 release there are 517 individual files hashed in http://releases.mozilla.org/pub/mozilla.org/firefox/releases/2.0.0.6/MD5SUMS

    The equivalent Thunderbird file has another 417 (fewer supported languages)

  9. John Black Says:

    I think it can be done. I’m not a Mozilla fan boy but they seem to have the best patch system in the world going right now. I don’t doubt that they can pull of miracles under file.

    Still, we won’t know until it is put to the test. Hopefully that kind of need won’t happen soon.

  10. J.C. Says:

    two words - MEMORY LEAK

    Maybe it isn’t a security problem but after 2+ years I think the 10 days has passed

  11. J.C. Says:

    Two words - memory leak

  12. Your Mom Says:

    Meh. I’ve posted critical security bugs to Mozilla before, (like ones that will post information to the wrong sites) with exploits, just to have them downgrade them to non-critical, and not fix them for almost a year. Easy way to weasel out of them.

    Anyway, when Mozilla has idiots like Darin Fisher who implement things like the PING attribute in the A tag that makes it easy for websites to track you against your will, and that *not a single user on the planet wants* it’s obvious that Mozilla doesn’t care for your privacy or security. Only when this PING tag is removed will I have any respect for them again.

  13. Jesse Ruderman Says:

    What was Mike Shaver’s promise? That Mozilla will fix security bugs you find within 10 days? That Mozilla will fix security bugs within 10 days, regardless of who reports it, if asked to do so by the reporter?

  14. Anonymous Says:

    “one fucking service pack”…in “half a fucking year.”

  15. Tips Dr.com Says:

    Todays Tech August 4, 2007

    New Wi-Fi network proves critical in Minneapolis bridge disaster A new Wi-Fi network in Minneapolis — only partially completed and just two months old — is nonetheless giving the city critical help in responding to this week’s collaps…

  16. anathema Says:

    I think the thing of note is the “responsible disclosure” this will rule out a lot of exploits found, as quite a few are just coded into a POC and then released into the wild and the vendor only knowing after the fact.

    It will be interesting though when their claim becomes tested. I think ten days is a decent timeframe to aspire to but not realistic in all cases.

  17. Friedbeef Says:

    Don’t know bout u - but I feel they are just inviting trouble with this claim…

  18. Max Kanat-Alexander Says:

    I’d believe it. The Firefox team has been very responsive to serious security holes, and has probably already done this 10 days thing several times. You could easily check by looking at the date a serious security issue was reported versus the date where there was a release with the fix. It wouldn’t always be 10 days, but it might be in some cases.

    -Max

  19. Ronald van den Heetkamp Says:

    It’s easy for me to kicm ‘em about it, I understand that before anyone else is going to say it, but if a moron like me can still make firefox stall on a simple thing like this:

    function {

    location . href = “uri”;

    function call

    }

    I think that is something to be ashamed of, really.

  20. Ronald van den Heetkamp Says:

    But, it goes way offtopic.

    It’s about Mozilla fixing stuff in ten days, and they can do. Like we saw in the latetst update.

  21. Irvan Says:

    is it called core dump, huh ??? :D

    -IT-

  22. Howard Beale Says:

    Firefox used to be the shiznit. . . for a while. The last few “upgrades” have been wonky. I agree about the mem leak. Come on guys!
    I’ll be back in “ten fucking days”. LOL //:

  23. Adrian Says:

    Of course they can do it.

  24. mroblivious1bmf Says:

    2 words: memory leak
    3 words: never had it

    :D

  25. Bk Says:

    When I did an analysis last year of how long IE and Firefox were vulnerable to unpatched exploits, it turned out that IE was open to known flaws for 284 days in 2006. Mozilla, on the other hand, had just nine days of exposure.

    See:

    http://blog.washingtonpost.com/securityfix/2007/01/internet_explorer_unsafe_for_2.html

  26. Deat by Fire Says:

    Oh wow you think your fuzzing tools will really pump out a bug so dangerous it can’t be fixed?

    You’re not a real coder, you’re a fucking PHP HACK. Go run your metasploit some more times, I’m sure you can get that remote shellcode working.

  27. RSnake Says:

    @Deat by Fire - I’m not sure if you’re speaking to me or someone else, or why you have hostility, but nothing I find uses any automated tools. I’m not trying to beat Mozilla up over anything. What I find aren’t even really considered vulns (in the traditional sense). what I work on is primarily logic flaws and (mis-)use cases which is an interesting problem. The reason it’s interesting is that I don’t need traditional vulns (buffer over flows for example since you mentioned meta-sploit) to do what I do. The Google Desktop exploit I found, for instance, had nothing to do with anything an automated tool could find - it had to do with strange interactions between client side and server side and man in the middle code and forced browsing.

    One of the weird things that came up after the talk is “Whats next?” Both Jeremiah and I are having a hard time even working with the browser guys in the same way more exploit authors do - because what we are finding is sort of systemic to the way browsers naturally work. Also, we’ve broken so much of the cross domain policy mantra that it’s not important to find vulns because it’s already so broken there is really less of a need.

    The issues Jeremiah and I work on aren’t (for the most part) something you can simply do an additional string compare for and be on your way. They generally require re-thinks of how the architecture of some part of the system works. That’s why 10 days is a weird thing for us - while it may be a great line in the sand for traditional exploit writers who use the fuzzers and pre-canned exploit generation tools you’re talking about, it really can’t apply to most of the problems I work on.

    So to answer your question - it’s not a matter of “so dangerous it can’t be fixed” it’s a matter of “so fundamental to how things work it’s a pain to fix”. Danger is all relative - you’re not going to own a box with the issues we find. However, you could write worms that infect millions of user’s webmail for instance.

    As an easy example since it came up this week, the CSS history hack that Jeremiah found almost a year ago and that I re-wrote to not require JavaScript is still unfixed. The JavaScript version of that hack probably will be fixed soon according to Mike - but that’s not exactly ten days. But to his credit it’s not a “critical” vulnerability, so they are under less of an obligatory time crunch to fix - consumers just don’t care as much about those issues even though they do have a lot of power when combined with other attacks. I hope that’s more clear.

    Simply put: for me, while 10 days is admirable - it’s almost completely non-applicable to everything Jeremiah and I have found. Not to say that Mozilla doesn’t have plans to fix them and not to say they don’t see a need to, they’re just harder to fix because it hurts the consumer almost as much as the exploit. Minimizing that impact is critical, and adds lots of time to the patches.

  28. Davide Denicolo Says:

    You should darken more his bunisess card…it’s hacked :P

  29. Davide Denicolo Says:

    – removed –

  30. David Kierznowski Says:

    That sounds incredibly unprofessional if he mean’t it in the contact you are portraying.

    Davide, nice image hack :)

  31. Chris D Says:

    Either way, I give them lots of respect for the attempt - even saying it shows they’re dedicated.

    I won’t be one to eat them alive if a few of them take eleven. That’s still pretty damned impressive in my book.

  32. K M D Says:

    I have to totally agree with Chris D here. Mem leaks (which make me tear my hair out daily) and other suggestions aside - they have my respect for having the cojones to make this claim. The number “10″ is merely symbolic in my book.

  33. RSnake Says:

    @David - nice one - I removed the link but very clever. Would you mind emailing me how you did it? I love that kind of stuff. I had to remove it because I’m trying to be nice to Mike. He didn’t say I had to white out the text, but I didn’t want him to get 10,000 calls on his cell either. ;)

  34. cenourinha Says:

    Ahahah… great!

    Mozilla Rox!

  35. softspade Says:

    I’m not sure if David did it this way, but my guess is that it’s form the EXIF data. Check out this exif viewer.

  36. RSnake Says:

    That’s exactly right. Pretty cool hack actually. Probably very useful for more sensitive applications. I appreciate the link! He also emailed me offline with the same info.

  37. Giorgio Maone Says:

    @RSnake: last time I checked your image still contained its EXIF thumbnail, while Jeremiah’s didn’t.
    I friendly blogged about this difference tonight, and also attached a quick recipe for (batch) stripping away EXIF/IPTC metadata from JPEGs before publishing them: http://hackademix.net/2007/08/05/two-faces-same-card/

    Just in case you really didn’t do this on purpose, and David didn’t already email you this info as well ;)

  38. Davide Denicolo Says:

    What info didn’t I email to him? About how to remove thumbnail?
    I explained him to read this article http://exploit.blogosfere.it/2007/07/quando-le-immagini-non-mostrano-verita.html
    or http://no.spam.ee/~tonu/exif/ for better understanding but I did’t receive answer to continue. But is the image with exif metadata still here?

  39. Okkio Says:

    I know this is going to seem the thickest thing ever posted here, but what exactly is memory leak and why on earth do people view it as such a big problem?

  40. Ronald van den Heetkamp Says:

    Excellent stuff Giorgio, I figure I can do the same in photoshop which I normally use to view watermarks & other metadata

  41. Ronald van den Heetkamp Says:

    It seems Photoshop does this automatically when you save for the web inside photoshop. It’s my favorite app, so that is nice to know :)

  42. Mike Shaver Says:

    (I thought I commented here on Friday, but I was working from my Blackberry, which is not especially web-friendly. Bleh.)

    Glad you enjoyed the party, Robert. To clarify, I was making a personal commitment, not a Mozilla one, that you could redeem that card if there was a vulnerability that you believed needed to be turned around in 10 days. I didn’t consider at the time that it would be taken as a Mozilla policy statement — even *I* don’t make new policy announcements at late-night parties in Vegas :) — but it seems to have been read that way, which I can understand in hindsight. I’m sure I’ll be answering for my potty mouth and apparent lack of clarity for a while…

  43. RSnake Says:

    @Giorgio - good writeup! But Jeremiah has suffered the same flaw - I’m not sure what you mean: http://regex.info/exif.cgi?dummy=on&url=http%3A%2F%2Fbp3.blogger.com%2F_JdybrokZBAk%2FRrY4PBXLuzI%2FAAAAAAAAAkY%2FRgil8CwLI8M%2Fs1600%2Ften-fucking-days.jpg

    I think you were looking at the wrong URL. Anyway - I’ve cleaned mine up - no, I wasn’t trying to be evil. As I said in the post I was in a big hurry as I wrote it trying to get back to DefCon. Why does everyone think I’m out to screw Mozilla? Alas - browser companies are my friend!

    @Mike - yes, it was really nice meeting you and I owe you a few emails and follow up posts. Honestly, I think while not policy, it shows confidence in your ability to do the right thing. It may have been perceived badly by few, but I think most of the people who matter get it. I saw Window’s response as well: http://blog.mozilla.com/security/2007/08/06/mike-shaver-ten-days-and-expletives/

    I know full well it isn’t Mozilla’s actual policy - but if nothing else, it shows people you and your team take this very seriously. In my mind this is nothing but a kick in the butt in the right direction. I wouldn’t have removed the expletive for anything.

    There probably is still an interesting follow-up conversation which I think we talked about briefly. That being how do we define open source when bugs are hidden from view? To be more clear: is it better to allow people to try to patch themselves or to allow or obfuscate the issues until a patch is ready for everyone - thereby delaying the patching for those who may be able to do it themselves? Clearly I can reverse engineer fixes, but that essentially just changes the problem from everyone knowing the problem to just a select few determined people. It’s a tough problem, and not one I expect to get “answered” but it is an interesting discussion.

  44. Giorgio Maone Says:

    @RSnake:
    My fault, I didn’t notice the picture inlined in Jeremiah’s post was just an incidentally washed up (by Photoshop’s “Save for Web”?) thumbnail:

    http://regex.info/exif.cgi?dummy=on&url=http://bp3.blogger.com/_JdybrokZBAk/RrY4PBXLuzI/AAAAAAAAAkY/Rgil8CwLI8M/s200/ten-fucking-days.jpg

    As you noticed, it is linked to your revealing original :P

    @Ronald:
    Yes, Photoshop’s “Save for web” can strip out some metadata (it’s probably what happened with Jeremiah’s thumbnail), but every time you use that function you encode the JPEG again, losing quality.
    IrfanView method is lossless, allows batch processing and is affordable even for the not so rich hackers ;)

  45. Ronald van den Heetkamp Says:

    Cool stuff, I didn’t know that. And I’m into graphis design all my life :D

    As far as the discussion, I can see the humor in the business card. If I would work for Mozilla, I probably would do the same at that particular moment. Let’s not wash out the circumstances it was presented ;)

    IMHO, I still think Mozilla can do a better job. When they rely on types like us, and highly skilled programmers and hackers I really think I may question that. Most bugs are things that could have been anticipated on, and would have been found with better testing & fuzzing. Not only that, I think certain security design principles are just denied only to comply with usability and the ease of using Firefox.

    And why all the quick media releases, like sitting in the front seat before microsoft does? Is Google pushing you guys? why the competition anyways, in the end it’s about the end users and not about a battle of intellects. Everyone with a few braincells likes Firefox more then MSIE and that will never change, so relax ;)

  46. JLS Says:

    Interesting methodology, however.

  47. TT Says:

    It seems Shaver pledge hasn’t even lasted 10 days.

    Mozilla: 10-day patch guarantee ‘not our policy’

    http://www.infoworld.com/article/07/08/06/Mozilla-10-day-patch-guarantee-not-our-policy_1.html

    “The open source browser maker was forced to issue a statement Monday, retracting a pledge attributed to the company’s director of ecosystem development, Mike Schaver, to fix any critical security bugs in the browser within “Ten ****ing Days.”

  48. anathema Says:

    As he said “it is not and never was Firefox’s policy”, It was his commitment to fix any bug that RSnake thought needed urgent attention.
    RSnake and Jeremiah should be chuffed, not many security professionals (even the other major corps) would get an offer like that.

    It’s great to see a more solid working relationship between vendor and consultant - rather than them thinking we are the all on the “evil” dark side, rather than what most of us really do.
    Walk a line on the grey side of life.

  49. Mensajero Says:

    A few more fucking days, then?

  50. Technocrat Says:

    He couldn’t be drunk, wasn’t the Mozilla party dry?

    Personally, I think Shaver might have gone a little too far in attempting to prove a worthy point. Mozilla is serious about fixing security issues and would like to work with researchers on these issues.

    Unlike other vendors, they are not going to throw out blanket security responses (looking at you Apple) and hope to improve Firefox with the help of researchers that report vulnerabilities.

    Ten days or not, it is a good stance to work toward.

  51. mo. Says:

    fortunately you were allowed to publish a scan of the card on your blog. somehow it’s a funny contemporary document.

  52. Alun Jones Says:

    Hey, I could fix almost any bug in ten fucking days, but if the bug was anything more than a typo, I would worry that I had introduced something worse.
    Because if I had failed to test on a wide enough array of test systems, I would have increased the risk that thousands of customers would be calling my support team on the phone first thing tomorrow morning. And then I’d have some ’splainin’ to do.

  53. Adam Says:

    This is incredibly unprofessional.

  54. aljuk Says:

    If only they’d get around to fixing the memory leak I might actually start using their bloated heap of junk again…

  55. hackathology Says:

    Lets see what happens and keep us updated.

  56. GMGJ Says:

    When you get to be old and gray and have been jumped on repeatedly you tend to back away from making objective claims. It also starts to sink in that that the reward for actually doing something is frequently outweighed by the pain and aggravation you get for for doing it.

    Then you go and work with ISO …

    My personal thanks to the folks who encouraged more carefully considered remarks, and to the folks who find the holes and to the folks that fix them, 10 days or not.

    And to “Corporate America”, thanks for almost nothing.

  57. Fawaz Iqbal Says:

    I think it will easily get done as he is saying. This is Mozilla mind you and not Microsoft!
    Atleast even if they get the job done in 11 days or 12 days, doesn’t matter. They are faster than the rest at the least.

  58. chillervalley Says:

    This is FUCKING AWESOME!

    i like it director of an company goes in this ways.

    i would like to have this “Ten Fucking Days” Card.

    This rocks so much. Mozilla rocks. TEN FUCKING DAYS.

    (P.s.: I can make a girl pregnant within THEN FUCKING DAYS!)