I’ve been thinking a lot about referring URLs over the last few days - mainly as a security measure. I’ve had a long standing internal battle about whether I think referring URLs have value or not as a security mechanism. Most of the time I think it’s not particularly useful since it can be spoofed, but really, let’s think about this for a second. When you look at traffic hitting your website, some fragment of it is spoofed (referrer spam) some of it is someone lying to hide their tracks and sometimes it’s just plain not there for some reason. It’s actually this third scenario that I’m most interested in and I’ll share why in a minute. First let’s look at when it isn’t there the majority of the time:
Local addresses: If a local URL makes a request to a remote site it will strip out the referring URL. This is to protect people from having their information stolen (drive information). There are bugs in this, but those are rare and for the most part this works as designed. Note that this does not apply to localhost type URLs, just HTML pages on the system itself.
Security tools: There are a number of desktop security tools that sanitize URLs. They also mess with other headers too, so in a way they are easy to detect, but I’m not going to get into that.
Those are the vast majority of cases where a referring URL won’t be there. So let’s talk about why you’d want to see a referring URL. You may want it to make a decision about whether to perform a function or not - probably a bad idea given that it can be spoofed. You may want to see where your traffic is coming from - a decent idea if you don’t make decisions based on it given that people can spoof it (referral spam). Lastly, if you want to see where a bulk of your traffic is coming from in the case of mass exploitation you may want to see a referring URL. Therein lies the single reason I think it may be worthwhile to start sending referring URLs on each request.
Yes, referring URLs are for amusement purposes only in a lot of ways, but I think making those two changes could really yield a lot more security benefits with only a minor hit to website development.