Help - Search - Members - Calendar
Full Version: Lynx cannot see my site
HTMLHelp Forums > General Interest > Site Review Requests
dpatel
Hi - I've created my first website (www.purpletin.co.uk). I've tried viewing it in Lynx but get the following error back:

QUOTE
Not Acceptable

An appropriate representation of the requested resource / could not be
found on this server.

Additionally, a 404 Not Found error was encountered while trying to
use an ErrorDocument to handle the request.


I can see it through 'w3m' and 'links' on my linux PC but not lynx.

I'm concerned I've done something wrong and won't get listed on search engines.

I've used Joomla 1.5 to design my website - could this be anything to do with it. I'm not sure it is because 1000s of people are using Joomla.
Darin McGrew
The server is sniffing browsers based on the user-agent string. When I configure Lynx to send a more "acceptable" user-agent string, the server stops sending 406 Not Acceptable.

The solution is to get rid of the browser sniffing.
dpatel
QUOTE(Darin McGrew @ Jan 13 2009, 06:22 PM) *

The server is sniffing browsers based on the user-agent string. When I configure Lynx to send a more "acceptable" user-agent string, the server stops sending 406 Not Acceptable.

The solution is to get rid of the browser sniffing.


Hi Darin - thanks for the reply. How do I get rid of the browser sniffing? Also, I assume this doesn't then effect whether or not my site can be crawled?
Darin McGrew
QUOTE
Hi Darin - thanks for the reply. How do I get rid of the browser sniffing?
You'll need to change your server configuration. The details depend on how the browser sniffing is implemented.

QUOTE
Also, I assume this doesn't then effect whether or not my site can be crawled
Maybe. Maybe not. What does the browser sniffing do with a "Googlebot" user-agent string?
dpatel
I'm a little new to all this, so you might have to talk me through it.

I am on shared hosting with Inmotionhosting. Would this explain why I can see it through w3m & links but not lynx?

Using Awstats, it is showing that Googlebot is showing up. I have no idea how to interrogate how browser sniffing affects Googlebot - can you help?

Thanks for the replies! smile.gif
pandy
Looks like it's Joomla's doing.

http://www.google.com/search?q=joomla+lynx+406
pandy
Or it's mod_security alone. I know nothing about mod_security.

Found this http://tutorials.zen-cart.com/index.php?article=390 .

And this. "lynx" is in there several times.
http://www.gotroot.com/downloads/ftp/mod_security/rules.conf
Brian Chandler
QUOTE(pandy @ Jan 16 2009, 12:24 PM) *

Or it's mod_security alone. I know nothing about mod_security.

Found this http://tutorials.zen-cart.com/index.php?article=390 .

And this. "lynx" is in there several times.
http://www.gotroot.com/downloads/ftp/mod_security/rules.conf


It all appears to be "security" in the modern style -- instead of writing the program properly to check its input, you just paste on a layer afterwards that filters out what look like "keywords". Goodness knows why "Lynx" counts as a dangerous keyword.
dpatel
Many thanks to all for replying.

I guess I'm not comfortable turning off security features so I'll have to see what happens in terms of the site being crawled/indexed or not. I'm hoping it'll be OK since I can at least see my site with other text browsers (links and w3m).
Brian Chandler
QUOTE(dpatel @ Jan 16 2009, 08:40 PM) *

Many thanks to all for replying.

I guess I'm not comfortable turning off security features so I'll have to see what happens in terms of the site being crawled/indexed or not. I'm hoping it'll be OK since I can at least see my site with other text browsers (links and w3m).


There is a big problem with the notion of "security features" -- this name makes them sound wonderful, but actually this is just a horrible kludge, causing almost as many problems as it solves. If you are using Joomla, perhaps you should investigate why and if it really needs this sort of sticking plaster. If it does, I would strongly recommend finding something better written.

But possibly this is mostly paranoia... you will need to probe the Joomla community to find out.

Darin McGrew
QUOTE(Brian Chandler @ Jan 15 2009, 08:54 PM) *
Goodness knows why "Lynx" counts as a dangerous keyword.
Because "Lynx/2.8.6rel.5 libwww-FM/2.14" is so much more dangerous than "Mozilla/4.0 (compatible; not MSIE 6.0; CSC 1107 Monitor System; UNIVAC 1107)" of course...
dpatel
Hi all, just thought I'd post that my site is now being indexed by Google & Yahoo. Best as I could tell is my host must has have been blocking Lynx for some reason.

If anyone else is having this issue try w3m or links instead.
Darin McGrew
QUOTE
If anyone else is having this issue try w3m or links instead.
Yeah, because everyone loves being told to use a different browser...
This is a "lo-fi" version of our main content. To view the full version with more information, formatting and images, please click here.
Invision Power Board © 2001-2014 Invision Power Services, Inc.