An empty cs(Referer), Options

[Mark2]

Hi,

ASP problem need your help:

From my log file, I have noticed that sometimes cs(Referer) for a page (say A.asp) is empty. This page is supposed come from a link
<a href="A.asp">......</a> in the first page Index.asp or other pages in my site.
I know one can directly type/paste the address of A.asp and causes an empty cs(Referer), but it would be much easier to click a link in the first page Index.asp or other pages in my site.

Can you guess why sometimes cs(Referer) for the page is empty?

[pandy]

Referer is an optional header. Browsers don't have to send it. Some browsers let you turn it off right from the menu.

[Christian J]

I've turned it off in some of my browsers.

More commonly, I don't think links that the user opens in new window will send one. Ditto for bookmarked URLs.

[Mark2]

Thanks a lot for helps from you two experts.

That means an empty referer is possible for a good visiter.....
I was trying to block a bad visiter who seems use an empty referer.

The bad visiter keep trying to post sex advertisments in my website for a few years and I don't know how he can get so many IPs, I cannot block him by IPs.

He also posted a lot similar sex advertisments in many websites around the world but seems nobody can or is willing to stop him!

[Christian J]

I'm no expert, but you're welcome anyway...

One way to obstruct spambots (but not real, human spammers) is with javascript encryption, like this one: http://www.jottings.com/obfuscator/

Note that the above is for email links. If you use a form you need to edit the document.write line so that it prints the form's ACTION attribute value instead of an email link. You also need to change the URL of the form's server-side script, since its current URL is known by spammers.

Users without javascript will of course be unable to post this way. If you're really ambitious you might let such users post, but keep their posts hidden until you've previewed them. This can be done by not hiding the ACTION value, but instead let the javascript populate a hidden form field with a "password". If the hidden field sends the "password", the server-side script will show the post right away. If the hidden field is empty (because JS is not used), the post is hidden until you've previewed it manually.

[pandy]

No, you can never assume that an empty referer is a bad guy. It's a common mistake with for instance scripts that are preventing image leeching, to allow a list of approved domains but forget the empty referer.

[Mark2]

To Christian J,

Thank you for the way to obstruct spambots.
However, what I meant was not email links .... For example, a bad guy clicked the "NEWTOPIC" button in this page, and then posted many sex page links here. Every time the content of the sex page links were not repeated so difficult to block them.

[Mark2]

No, you can never assume that an empty referer is a bad guy. It's a common mistake with for instance scripts that are preventing image leeching, to allow a list of approved domains but forget the empty referer.

To pandy,

OK, thank you. I will try something else.

[Christian J]

To Christian J,

Thank you for the way to obstruct spambots.
However, what I meant was not email links .... For example, a bad guy clicked the "NEWTOPIC" button in this page, and then posted many sex page links here. Every time the content of the sex page links were not repeated so difficult to block them.

Yes, that's why you must adapt the javascript to work with forms (see my previous post).

[pandy]

You can ban by IP if he's always using the same. Not with JavaScript but with a .htaccess file if you are on Apache.

But "New Topic" sounds like a forum and they more often than not have banning features built in, so you can probably just add his IP address to the filters from the Admin section.

[Mark2]

Dear Pandy,

He has been using hundreds different IPs. It is difficult to block him by IPs.

[Christian J]

that's why you must adapt the javascript to work with forms (see my previous post).

Here's a simple example. First rename the ASP script file. Then give the FORM an ID, and remove the URL in the ACTION attribute:

CODE

<form id="f" method="post" action="">
...
</form>

Next generate the javascript from http://www.jottings.com/obfuscator/ but put the new URL of your ASP script (instead of an email address) in the generator. Then place the generated javascript after the form on your web page, and finally replace this line:

CODE

document.write("<a href='mailto:"+link+"'>"+link+"</a>")

with this:

CODE

document.getElementById('f').action=link;

Now you have a form that will only submit to the correct ASP URL when javascript is enabled.

(In my experience many spambots still don't understand javascript - at least the javascript-obfuscated forms I've tried seem to block them - but that can of course change in the future. The widespread use of CAPTCHA:s may indicate that some spambots may already understand javascript. Personally I'd avoid CAPTCHA if possible, since they are a hassle for human users too.)

[Mark2]

Christian J,

Thank you so much and I understand what you mean now. I will have a try!