Annoying 404 redirect

[Pete]

Our web site (running Apache) has a special '404 not found' page that
tries to be helpful in suggesting what might have been mistyped and
providing a local search engine. However, I was just in the local
public library, and wanted to look something up on our site, but of
course mistyped it. Instead of getting our help page, I got shunted
over to (apparently) the library's service provider 404 page!

(I've just checked our logs, and sure enough the request arrived at
our machine (which was correctly entered) before getting lost on my
typo.)

Where is this diversion happening? I thought that 404 redirects were
done on the target if appropriate. I suppose that our server was
dutifully supplying the help, but the 404 tag caused the library's ISP
to interfere.

I don't suppose there's any way to clobber their actions, is there?

-- Pete --

--
================================================== ==========================
The address in the header is a Spam Bucket -- don't bother replying to it...
(If you do need to email, replace the account name with my true name.)
================================================== ==========================

[Leif K-Brooks]

Pete wrote:

Our web site (running Apache) has a special '404 not found' page that
tries to be helpful in suggesting what might have been mistyped and
providing a local search engine. However, I was just in the local
public library, and wanted to look something up on our site, but of
course mistyped it. Instead of getting our help page, I got shunted
over to (apparently) the library's service provider 404 page!

Are you sure it wasn't Internet Explorer's default 404 message? It
displays it instead of your custom message when the 404 page is smaller
than 512 bytes. If that's what's happening, the obvious fix would be to
add enough padding (meaningless comments or whitespace, for instance) to
your page so that IE will show it.

http://support.microsoft.com/kb/q218155/

[Andy Dingley]

On 10 Jan 2005 15:44:34 -0800, ne*******@jwgibbs.cchem.berkeley.edu
(Pete) wrote:

I don't suppose there's any way to clobber their actions, is there?

Don't return a 404. Show the "404 page" you already have, but return
something like a 202 success code. After all, your server has
"Accepted" the request, it just can't return the content they were
after.

This may confuse some spiders though.

--
Smert' spamionam

[Jukka K. Korpela]

Andy Dingley <di*****@codesmiths.com> wrote:

Don't return a 404. Show the "404 page" you already have, but return
something like a 202 success code.
Oh no! That's similar to the misbehavior that some sites exhibit, and it's
a gross error. Well, maybe not as gross as sending 200 code and "error
page", but still.
After all, your server has
"Accepted" the request, it just can't return the content they were
after.
The HTTP protocol describes 202 as follows:
"Its purpose is to allow a server to accept a request for some other
process (perhaps a batch-oriented process that is only run once per day)
without requiring that the user agent's connection to the server persist
until the process is completed. The entity returned with this response
SHOULD include an indication of the request's current status and either a
pointer to a status monitor or some estimate of when the user can expect
the request to be fulfilled."

So how would you give the actual error message? Any HTML document that your
server sends along with the 202 code is _not_ an error message, by
definition, but just an explanation of what's going on with the request.
This may confuse some spiders though.

It will, and other user agents and users as well.

I'm pretty sure Leif gave the correct analysis, though I could only be real
sure if the URL of the problematic page and a URL of a screenshot had been
posted. But putting dummy content into an error message is almost always
pointless; any _useful_ error page is, by its very nature, longer than
needed to overcome IE's "helpfulness"; for some ideas on this, see
http://www.cs.tut.fi/~jkorpela/www/404.html

Note that IE behavior depends on in its settings; in the Internet settings,
there's a checkbox about showing HTTP error messages in "helpful" format,
and it's probably checked by default. The "helpful" format means that IE
ignores the error message sent by the server if it thinks it's too short,
and shows its own generic babble instead.

--
Yucca, http://www.cs.tut.fi/~jkorpela/

[Alan J. Flavell]

On Tue, 11 Jan 2005, Jukka K. Korpela wrote:

pointless; any _useful_ error page is, by its very nature, longer than
needed to overcome IE's "helpfulness";

Not necessarily, in my experience. I had to deal with a report of an
error page (IIRC it was produced by mod_speling) which contained a
menu of two likely best-matches for the miss-spelled URL that this
particular user had presented. IE had seen fit to suppress this
useful menu and brief explanation, and substitute its own.

This error page wasn't exactly chatty, I admit, but IMHO it was quite
adequate to its purpose, in WWW terms - however, it had to be padded
out with woffle to pacify IE.

[Jukka K. Korpela]

"Alan J. Flavell" <fl*****@ph.gla.ac.uk> wrote:

On Tue, 11 Jan 2005, Jukka K. Korpela wrote:

pointless; any _useful_ error page is, by its very nature, longer than
needed to overcome IE's "helpfulness";

Not necessarily, in my experience. I had to deal with a report of an
error page (IIRC it was produced by mod_speling) which contained a
menu of two likely best-matches for the miss-spelled URL that this
particular user had presented.

Since the error page should be an HTML document that complies with the
specification (needing a DOCTYPE etc.) and since the simple menu should be
accompanied with a textual explanation*), I don't think it's particularly
difficult to exceed the 512 character limit.

*) Whenever we make a guess, even an intelligent guess, we should remember
that we might guess wrong. I would even say that this is part of making an
_intelligent_ guess. So a simple menu should be followed, for example,
by a statement like

<p>The address (URL) specified does not correspond to any
existing resource. Probably the address is a misspelling
of either of the two address above. If this is not the case,
please use our <a href="search.html">search page</a> to
locate the resource you're looking for.</p>

(which is about 276 characters already).

--
Yucca, http://www.cs.tut.fi/~jkorpela/

[Pete]

In article <34*************@individual.net>,
Leif K-Brooks <eu*****@ecritters.biz> wrote:

Pete wrote:

Are you sure it wasn't Internet Explorer's default 404 message? It
displays it instead of your custom message when the 404 page is smaller
than 512 bytes. If that's what's happening, the obvious fix would be to
add enough padding (meaningless comments or whitespace, for instance) to
your page so that IE will show it.

Interesting [something I didn't know], but I don't think that was the
cause in this case.

For one thing, I noticed some string in the URL gadget like
"www.serverhosts.com/notfound.php?...." [No, it wasn't that --
it just was sort of in that form. I'll note it more specifically
if I get a chance.]

Secondly, our 404 message is certainly longer that 512 bytes.
Not sure where that length is measured, but if I look at the raw
HTML received, it's nearly 2K (and if I save just the text from lynx,
it's still 1K).

Maybe that particular version of IE has been tweaked to redirect
*all* 404s?

-- Pete --

--
================================================== ==========================
The address in the header is a Spam Bucket -- don't bother replying to it...
(If you do need to email, replace the account name with my true name.)
================================================== ==========================

[Jukka K. Korpela]

ne*******@jwgibbs.cchem.berkeley.edu (Pete) wrote:

For one thing, I noticed some string in the URL gadget like
"www.serverhosts.com/notfound.php?...." [No, it wasn't that --
it just was sort of in that form. I'll note it more specifically
if I get a chance.]
That doesn't sound very specific, and doesn't sound like relating to the
problem you've described.
Secondly, our 404 message is certainly longer that 512 bytes.
For Pete's sake, why do you _still_ refrain from telling the URL?
WWW resources are addressed by URLs, you know.
Maybe that particular version of IE has been tweaked to redirect
*all* 404s?

That would be odd, but you could test that simply by using it to access
some nonexistent URL in some other site.

--
Yucca, http://www.cs.tut.fi/~jkorpela/

[Pete]

In article <Xn*****************************@193.229.0.31>,
Jukka K. Korpela <jk******@cs.tut.fi> wrote:

ne*******@jwgibbs.cchem.berkeley.edu (Pete) wrote:

For one thing, I noticed some string in the URL gadget like
"www.serverhosts.com/notfound.php?...." [No, it wasn't that --
it just was sort of in that form. I'll note it more specifically
if I get a chance.]
That doesn't sound very specific, and doesn't sound like relating to the
problem you've described.

Umm -- I just said... I DON'T REMEMBER THE BLOODY THING!!
What I'm saying is that the 404 page returned had a URL -- it didn't
seem to be internal. Next time I happen to be by the library, and
if there's a free machine, I will carefully note the exact string...

Secondly, our 404 message is certainly longer that 512 bytes.
For Pete's sake, why do you _still_ refrain from telling the URL?
WWW resources are addressed by URLs, you know.

*What* URL?? The typo I entered? (:-/) Wouldn't do you any good unless
you happened to visit the Berkeley Public Library! And to repeat myself
again, I don't REMEMBER the 404 URL.

Maybe that particular version of IE has been tweaked to redirect
*all* 404s?

That would be odd, but you could test that simply by using it to access
some nonexistent URL in some other site.

That I will do, but I suspect one always ends up at the same page.

-- Pete --

--
================================================== ==========================
The address in the header is a Spam Bucket -- don't bother replying to it...
(If you do need to email, replace the account name with my true name.)
================================================== ==========================