Cenzic 232 Patent
Paid Advertising
web application security lab

To Refer or Not to Refer

I’ve been thinking a lot about referring URLs over the last few days - mainly as a security measure. I’ve had a long standing internal battle about whether I think referring URLs have value or not as a security mechanism. Most of the time I think it’s not particularly useful since it can be spoofed, but really, let’s think about this for a second. When you look at traffic hitting your website, some fragment of it is spoofed (referrer spam) some of it is someone lying to hide their tracks and sometimes it’s just plain not there for some reason. It’s actually this third scenario that I’m most interested in and I’ll share why in a minute. First let’s look at when it isn’t there the majority of the time:

META Refresh: Whenever someone uses a meta refresh (in headers or in HTML) it strips out the referring URL. This is often used when you are building sites that have quickly changing content, or to redirect people to another page after some amount of time (sorta a backup for JavaScript redirection these days).

JavaScript: There are a number of cases where JavaScript will strip out referring URLs. Although there are lots of scenarios where this is true, it’s not particularly useful to strip it out in most cases.

Local addresses: If a local URL makes a request to a remote site it will strip out the referring URL. This is to protect people from having their information stolen (drive information). There are bugs in this, but those are rare and for the most part this works as designed. Note that this does not apply to localhost type URLs, just HTML pages on the system itself.

Security tools: There are a number of desktop security tools that sanitize URLs. They also mess with other headers too, so in a way they are easy to detect, but I’m not going to get into that.

Those are the vast majority of cases where a referring URL won’t be there. So let’s talk about why you’d want to see a referring URL. You may want it to make a decision about whether to perform a function or not - probably a bad idea given that it can be spoofed. You may want to see where your traffic is coming from - a decent idea if you don’t make decisions based on it given that people can spoof it (referral spam). Lastly, if you want to see where a bulk of your traffic is coming from in the case of mass exploitation you may want to see a referring URL. Therein lies the single reason I think it may be worthwhile to start sending referring URLs on each request.

Now mind you, I know for a fact that there will be browsers and software that will always cleanse the referring URL. But if you change meta referesh, and JavaScript to always send referring URLs, (not the other two scenarios mentioned above) you make it impossible for an attacker to cleanse their attack origin. Currently today, attackers have a conduit to sending as much traffic as they like without letting the victim know where the traffic started. For privacy advocates they can create an interim redirection that changes the referring URL. So rather than have no referring URL it will be a referring URL to a page that has no value to an attacker attempting to derive information from the user.

Yes, referring URLs are for amusement purposes only in a lot of ways, but I think making those two changes could really yield a lot more security benefits with only a minor hit to website development.

13 Responses to “To Refer or Not to Refer”

  1. Dave Lewis Says:

    You make some very sound points. There are all kinds of unscrupulous characters that use trackback spam and it can be a real pain in the butt trying to clean up after a deluge. A tool that I use to great success is Spam Karma 2 (link: http://unknowngenius.com/blog/wordpress/spam-karma ). It filters out the detritus with a surprising success rate. This particular plugin is Wordpress specific. It has made my life easier with respect to track backs on my site.

    Cheers

  2. Wladimir Palant Says:

    You forgot that HTTPS referrers aren’t transferred either.

  3. RSnake Says:

    Ah, yes, I knew I was forgetting something, thank you, Wladimir!

  4. RSnake Says:

    @Dave - I actually get no trackback spam anymore (maybe one or two a day at most). Much better than the thousands that I was getting. It’s only a 10-15 line script that I wrote to block the vast majority, and I still manage to get real trackbacks, just none of the garbage. So I’m not too worried about that. But also, I wasn’t talking about trackback spam, I’m talking about referral spam. Those two things are not really close, as trackbacks don’t have referring URLs in the HTTP header since they are robotic in nature. They submit a URL but that’s very different.

  5. Dave Lewis Says:

    @RSnake - (slapping my forehead) Sorry, that’s what I get for not paying proper attention. Thanks for setting me straight.

  6. another anonymous bastard Says:

    Referrer as a security measure … it’s possible, if you use your own referrer method.
    Say you are using sessions (cookie by client stored on the server, that’s it), each page your client browse know who it is, i mean when you request /a/page.cgi, “page.cgi” know his name and it is not fakable, so this page can fill his name in the session.

    Using that method you are able to have “unfakable” referrer (if your server has not been rooted yet, if you are not using PHP in module and another user in the same server is not bugging it, etc).

  7. Chris Snyder Says:

    Conventional wisdom is that referrers are useful in preventing cross-site request forgery, since the victim’s browser isn’t going to spoof the referrer. So there’s some value right there…

  8. kaes Says:

    FYI, Opera doesn’t clear the referer-header on a META refresh. so it’s not a surefire way to get rid of the http-referer. For example, if you use some sort of massive XSS that targets generic visitors of a site to exploit another, the fraction of users that uses Opera will still send the referer-header, so the site from which the hack is originating will still be able to be found in the serverlogs (for ex., in the mass-exploitation example you gave)
    To surely clear http-referer, you’d need to use a combination of javascript (for Opera) and META refresh (for other browsers). But you can never really be sure that someone with a weird browser comes along and leaks the info afterall.

  9. Jeremiah Grossman Says:

    Thats what I though until Amit Klein twice demonstrated otherwise.

    “Forging HTTP request headers with Flash”
    http://www.webappsec.org/lists/websecurity/archive/2006-07/msg00069.html

  10. Chris Snyder Says:

    @Jeremiah Erp, that’s what I love about this lab. Conventional wisdom is regularly punctured and tossed into the corner. God bless Macromedia for giving us another environment in which to make cross-domain requests.

  11. Ironic Says:

    Also, would great to retrieve from where the referral is coming.

    it’ss from an img tag?, it’s from a link of css?, it’s a link?

    The only use i see for referral is just to know where ur traffic is coming from.

    But i found it very easy to modified as you like.

  12. dusoft Says:

    Referers are often used a security measure for an action to start (e.g. script checks if the previous page is the one that it expects it to be etc.)

    of course, it has to be implemented only as one part of the solution and can’t be itself used for critical operation checking.

  13. Dipson Says:

    I need a help.
    I need a script to be placed in my site-1 that redirects to site-2 and the reeferring URL is site-1.
    Please mail me the script at xtranophilist@toughguy.net if you can.
    I would be very thankful.