How to keep a site out of the search engines?

[Steve]

Hi;

I have a site that I do not want the search engines to pick up
on.....attracts people and problems I do not want.

Is there a tag ( or some other means ) of preventing this?

Thanks

Steve

[jojo]

Alan Silver schrieb:

In article <11************ **********@i40g 2000cwc.googleg roups.com>,
"Andy Dingley <di*****@codesm iths.com>" <di*****@codesm iths.com> writes

Google Toolbar (for just one) is a backchannel that feeds the URLs of
"hidden" web sites back to Google, where they then get spidered.

Also, Google has taken to looking at newly registered domain names to
see if there is a web site there. This means that even if your site
doesn't have any links to it and you don't use the Google toolbar,
Google could still find it!!

Google will always be able to "find" your page. You can just tell Google
not to list your page.
You can use either robots.txt or <meta>-Tags to keep out a site from the
indexes of most search-engines. Although there are Spiders who do not
follow your rules in robots.txt, most of the searchbots do.

If you just want a single page not to be listed by search-engines insert
the following tag into your HTML-head:
<meta name="robots" content="noinde x">

If you want a whole directory not to be listed you have to create a text
file called "robots.txt " in the main directory of your domain. In this
file you write:
User-agent: *
Disallow: /DIRECTORY/
(Replace DIRECTORY with the name of the directory you want to disallow.)

Hope I could help you.

[deko]

>>Google Toolbar (for just one) is a backchannel that feeds the URLs of

>>"hidden" web sites back to Google, where they then get spidered.

Also, Google has taken to looking at newly registered domain names to see if
there is a web site there. This means that even if your site doesn't have any
links to it and you don't use the Google toolbar, Google could still find
it!!

Google will always be able to "find" your page. You can just tell Google not
to list your page.
You can use either robots.txt or <meta>-Tags to keep out a site from the
indexes of most search-engines. Although there are Spiders who do not follow
your rules in robots.txt, most of the searchbots do.

If you just want a single page not to be listed by search-engines insert the
following tag into your HTML-head:
<meta name="robots" content="noinde x">

If you want a whole directory not to be listed you have to create a text file
called "robots.txt " in the main directory of your domain. In this file you
write:
User-agent: *
Disallow: /DIRECTORY/
(Replace DIRECTORY with the name of the directory you want to disallow.)

You can request that your URL be removed from Google's index at
http://services.google.com:8882/urlc...&lastcmd=login
and read more about Google's webmaster's guidelines at
http://www.google.com/support/webmas...y?answer=35769

Google generally plays by the rules, so a Disallow instruction in robots.txt
should work - for google's bot. But don't expect all bots to heed your
instructions (many will ignore robots.txt entirely). It's like you are on a
crowded public street telling people not to look at you. As long as you're in
sight, there's nothing preventing people (good, bad, and indifferent) from
looking.