Correct HTTP response status code on "Access Denied" page.

[diarmaid]

Hi,

I have a password-protected area on my site. I'm not using any server
authentication methods: Instead I prompt the user for a username and
password using an HTML form and check these against a database of
registered users.

I was thinking that this page should be returning a HTTP status code of
"401 Unauthorized" instead of its current "200 OK". My reasoning being
that this would let search engines know what's going on when they
request a protected page and get redirected to the "Access Denied"
page.

I experimented with adding response.status = "401 Unauthorized" to the
"denied" page, but now it displays a (Windows/IE) username & password
prompt instead. The HTTP sniffer shows that the www-authenticate field
is set to "Basic".

My question: How can I indicate to the search engines that the page it
is requesting requires authentication, yet have it not display the
login/password dialog box?

Thanks in advance,

--Diarmaid McGleenan