473,289 Members | 1,808 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,289 software developers and data experts.

Google Bot problems?

I have worked on a couple of sites which google's bot visits, partially lists
and then goes away again.

MSN and Yahoo are fine and working.

Can anyone please suggest what, if anything, is wrong with these sites? Frankly
I am a bit baffled at the moment! I am wondering if there is a problem with the
page headers and googlebot?

All suggestions appreciated.

Thanks in advance,


Steve

Sites & Google results are:-

http://tinyurl.com/6tr2b

http://tinyurl.com/5dj6w
Jul 23 '05 #1
29 3175
me
"Steve" <pl***************@ireland.com> wrote in message
news:_m*******************@news.indigo.ie...
I have worked on a couple of sites which google's bot visits, partially lists and then goes away again.

MSN and Yahoo are fine and working.

Can anyone please suggest what, if anything, is wrong with these sites? Frankly I am a bit baffled at the moment! I am wondering if there is a problem with the page headers and googlebot?
All suggestions appreciated.
Thanks in advance,
Steve

Sites & Google results are:-

http://tinyurl.com/6tr2b

http://tinyurl.com/5dj6w


Place this in the head of every page:

<meta name="robots" content="ALL">

IIRC bots may enter from any page so this may help.
Good Luck,
me
Jul 23 '05 #2
>
Place this in the head of every page:

<meta name="robots" content="ALL">

IIRC bots may enter from any page so this may help.
Good Luck,
me


Thanks me...will give it a go!

Steve
Jul 23 '05 #3
On Wed, 09 Mar 2005 19:07:34 +0000, Steve
<pl***************@ireland.com> wrote:

Place this in the head of every page:

<meta name="robots" content="ALL">

IIRC bots may enter from any page so this may help.
Good Luck,
me


Thanks me...will give it a go!

Steve


Thoughtful of me to mentin it but it won't have any effect. Please may
we see the robots txt for each site.

BB
--
www.kruse.co.uk/ SE*@kruse.demon.co.uk
Affordable SEO!
--
Jul 23 '05 #4
On Wed, 9 Mar 2005 09:01:36 -0600, "me" <anonymous@_.com> wrote:
"Steve" <pl***************@ireland.com> wrote in message
news:_m*******************@news.indigo.ie...
I have worked on a couple of sites which google's bot visits, partially

lists
and then goes away again.

MSN and Yahoo are fine and working.

Can anyone please suggest what, if anything, is wrong with these sites?

Frankly
I am a bit baffled at the moment! I am wondering if there is a problem

with the
page headers and googlebot?
All suggestions appreciated.
Thanks in advance,
Steve

Sites & Google results are:-

http://tinyurl.com/6tr2b

http://tinyurl.com/5dj6w


Place this in the head of every page:

<meta name="robots" content="ALL">

IIRC bots may enter from any page so this may help.
Good Luck,
me


It won't do anything. Robots ignore meta tags like that as they index
what they can anyway by default. It would be useful if the relevant
robots.txts were displayed here.

BB.

--
www.kruse.co.uk/ SE*@kruse.demon.co.uk
Affordable SEO!
--
Jul 23 '05 #5
Gazing into my crystal ball I observed Big Bill <kr***@cityscape.co.uk>
writing in news:18********************************@4ax.com:
On Wed, 9 Mar 2005 09:01:36 -0600, "me" <anonymous@_.com> wrote:
"Steve" <pl***************@ireland.com> wrote in message
news:_m*******************@news.indigo.ie...
I have worked on a couple of sites which google's bot visits, partially
lists and then goes away again.

MSN and Yahoo are fine and working.

Can anyone please suggest what, if anything, is wrong with these sites?
Frankly I am a bit baffled at the moment! I am wondering if there is a
problem with the page headers and googlebot?
All suggestions appreciated.
Thanks in advance,
Steve

Sites & Google results are:-

http://tinyurl.com/6tr2b

http://tinyurl.com/5dj6w


Place this in the head of every page:

<meta name="robots" content="ALL">

IIRC bots may enter from any page so this may help.
Good Luck,
me


It won't do anything. Robots ignore meta tags like that as they index
what they can anyway by default. It would be useful if the relevant
robots.txts were displayed here.

BB.

--
www.kruse.co.uk/ SE*@kruse.demon.co.uk
Affordable SEO!
--


1. http://www.barrabooks.com/robots.txt

User-agent: *
Disallow: /picture_library/
Disallow: /Store/
Disallow: /CCS/
Disallow: /webstat/
Disallow: /plesk-stat/
Disallow: /php/

2. http://www.stevenhenson.com/robots.txt

User-agent: *
Disallow: /picture_library/
Disallow: /Store/
Disallow: /CCS/
Disallow: /webstat/
Disallow: /plesk-stat/
Disallow: /php/

I don't see anything unusual do you?

--
Adrienne Boswell
http://www.cavalcade-of-coding.info
Please respond to the group so others can share
Jul 23 '05 #6
> 1. http://www.barrabooks.com/robots.txt

User-agent: *
Disallow: /picture_library/
Disallow: /Store/
Disallow: /CCS/
Disallow: /webstat/
Disallow: /plesk-stat/
Disallow: /php/

2. http://www.stevenhenson.com/robots.txt

User-agent: *
Disallow: /picture_library/
Disallow: /Store/
Disallow: /CCS/
Disallow: /webstat/
Disallow: /plesk-stat/
Disallow: /php/

I don't see anything unusual do you?


Thank you Adrienne - I see nothing nasty in the robot.txt files as well! It
still leaves me baffled as to why Google is behaving this way.


Steve
Jul 23 '05 #7
> I have worked on a couple of sites which google's bot visits,
partially lists
and then goes away again.
MSN and Yahoo are fine and working.


How long have you waited? You do have to be patient. MSN have made
some claims about indexing more often than Google and I believe this
could be true. Google does find the front page for one of the sites.

(BB, the robots.txt files are at http://www.barrabooks.com/robots.txt
and http://www.stevenhenson.com/robots.txt.)

--Phil.

Jul 23 '05 #8
me
"Steve" <pl***************@ireland.com> wrote in message
news:_m*******************@news.indigo.ie...
I have worked on a couple of sites which google's bot visits, partially lists and then goes away again.

MSN and Yahoo are fine and working.

Can anyone please suggest what, if anything, is wrong with these sites? Frankly I am a bit baffled at the moment! I am wondering if there is a problem with the page headers and googlebot?
All suggestions appreciated.
Thanks in advance,

Steve

Sites & Google results are:-

http://tinyurl.com/6tr2b

http://tinyurl.com/5dj6w


Don't assume that if you submit your site to one engine the others will also
pick it up (unless it's Google). Submit your site to Google directly:

http://www.google.com/addurl/

Incidentally the others search engines do eventually pick up any site that
Google lists, they know who the top search engine is even if MS *still*
doesn't. ;-)

This page still does not have <meta name="robots" content="ALL"> in the head
so I assume there may be others that don't. It may not be absolute necessity
to have this tag on every page but what can it hurt? I always plan for the
worst and hope for the best.

http://www.stevenhenson.com/spider_map.htm

There's also a trailing slash after the word "ALL" in the tag you're using.
I don't know if this will cause a problem but I would omit it just in case.

If you have submitted your site recently it may (surely?) take two weeks to
several months before it gets listed.
Good Luck,
me
Jul 23 '05 #9
Gazing into my crystal ball I observed Steve
<pl***************@ireland.com> writing in
news:Fc*******************@news.indigo.ie:
1. http://www.barrabooks.com/robots.txt

User-agent: *
Disallow: /picture_library/
Disallow: /Store/
Disallow: /CCS/
Disallow: /webstat/
Disallow: /plesk-stat/
Disallow: /php/

2. http://www.stevenhenson.com/robots.txt

User-agent: *
Disallow: /picture_library/
Disallow: /Store/
Disallow: /CCS/
Disallow: /webstat/
Disallow: /plesk-stat/
Disallow: /php/

I don't see anything unusual do you?


Thank you Adrienne - I see nothing nasty in the robot.txt files as
well! It still leaves me baffled as to why Google is behaving this way.


Not sure if this is what is causing it, but on www.stevenhenson.com there
are some markup errors that might be confusing Google. Google _might_
think it is going in an endless loop, therefore getting out before it
sticks its toe in the water.<http://validator.w3.org/check?uri=http%3A%2F%
2Fwww.stevenhenson.com%2F&charset=%28detect+automa tically%29&doctype=%
28detect+automatically%29&ss=1>

www.barrabooks.com does not have any markup errors, but it could be
upgraded to stylesheets instead of tables, and use semantic markup, eg:
<span class="style2 xbig"><strong>Welcome to Barra Books</strong></span>
should be
<h1>Welcome to Barra Books</h1>

--
Adrienne Boswell
http://www.cavalcade-of-coding.info
Please respond to the group so others can share
Jul 23 '05 #10
me wrote:
"Steve" <pl***************@ireland.com> wrote in message
news:_m*******************@news.indigo.ie...
I have worked on a couple of sites which google's bot visits, partially


lists
and then goes away again.

MSN and Yahoo are fine and working.

Can anyone please suggest what, if anything, is wrong with these sites?


Frankly
I am a bit baffled at the moment! I am wondering if there is a problem


with the
page headers and googlebot?
All suggestions appreciated.
Thanks in advance,

Steve

Sites & Google results are:-

http://tinyurl.com/6tr2b

http://tinyurl.com/5dj6w

Don't assume that if you submit your site to one engine the others will also
pick it up (unless it's Google). Submit your site to Google directly:

http://www.google.com/addurl/

Incidentally the others search engines do eventually pick up any site that
Google lists, they know who the top search engine is even if MS *still*
doesn't. ;-)

This page still does not have <meta name="robots" content="ALL"> in the head
so I assume there may be others that don't. It may not be absolute necessity
to have this tag on every page but what can it hurt? I always plan for the
worst and hope for the best.

http://www.stevenhenson.com/spider_map.htm

There's also a trailing slash after the word "ALL" in the tag you're using.
I don't know if this will cause a problem but I would omit it just in case.

If you have submitted your site recently it may (surely?) take two weeks to
several months before it gets listed.
Good Luck,
me

Thank you!

I have added the robots ALL tag to the spider_map page and submitted that page
to Google. Not sure if this will work but the spider_map page is a page
deliberately designed to make life easy for the spiders...so hopefully we might
make some headway!

Google keeps visiting the sites, looks at one or two pages (according to the
logs) and then goes away again.

Every now and then Google shows the sites as having a couple of pages listed -
and then reduces then down to the minimum information (i.e. just one page). Its
rather like Google cannot make its mind up to include the site and keeps coming
back for a nibble!
Steve
Jul 23 '05 #11
On Thu, 10 Mar 2005 07:22:54 GMT, Adrienne <ar********@sbcglobal.net>
wrote:
Gazing into my crystal ball I observed Big Bill <kr***@cityscape.co.uk>
writing in news:18********************************@4ax.com:
On Wed, 9 Mar 2005 09:01:36 -0600, "me" <anonymous@_.com> wrote:
"Steve" <pl***************@ireland.com> wrote in message
news:_m*******************@news.indigo.ie...
I have worked on a couple of sites which google's bot visits, partially
lists and then goes away again.

MSN and Yahoo are fine and working.

Can anyone please suggest what, if anything, is wrong with these sites?
Frankly I am a bit baffled at the moment! I am wondering if there is a
problem with the page headers and googlebot?
All suggestions appreciated.
Thanks in advance,
Steve

Sites & Google results are:-

http://tinyurl.com/6tr2b

http://tinyurl.com/5dj6w

Place this in the head of every page:

<meta name="robots" content="ALL">

IIRC bots may enter from any page so this may help.
Good Luck,
me


It won't do anything. Robots ignore meta tags like that as they index
what they can anyway by default. It would be useful if the relevant
robots.txts were displayed here.

BB.

--
www.kruse.co.uk/ SE*@kruse.demon.co.uk
Affordable SEO!
--


1. http://www.barrabooks.com/robots.txt

User-agent: *
Disallow: /picture_library/
Disallow: /Store/
Disallow: /CCS/
Disallow: /webstat/
Disallow: /plesk-stat/
Disallow: /php/

2. http://www.stevenhenson.com/robots.txt

User-agent: *
Disallow: /picture_library/
Disallow: /Store/
Disallow: /CCS/
Disallow: /webstat/
Disallow: /plesk-stat/
Disallow: /php/

I don't see anything unusual do you?


No but something's somewhere. I'll have to come back to this as I'm
trying to download a windows update and watch Zev fron Lexx in the
shower just at the mo.

BB
--
www.kruse.co.uk/ SE*@kruse.demon.co.uk
Affordable SEO!
--
Jul 23 '05 #12
On Thu, 10 Mar 2005 21:24:11 +0000, Steve
<pl***************@ireland.com> wrote:
me wrote:
"Steve" <pl***************@ireland.com> wrote in message
news:_m*******************@news.indigo.ie...
I have worked on a couple of sites which google's bot visits, partially
lists
and then goes away again.

MSN and Yahoo are fine and working.

Can anyone please suggest what, if anything, is wrong with these sites?


Frankly
I am a bit baffled at the moment! I am wondering if there is a problem


with the
page headers and googlebot?
All suggestions appreciated.
Thanks in advance,

Steve

Sites & Google results are:-

http://tinyurl.com/6tr2b

http://tinyurl.com/5dj6w

Don't assume that if you submit your site to one engine the others will also
pick it up (unless it's Google). Submit your site to Google directly:

http://www.google.com/addurl/

Incidentally the others search engines do eventually pick up any site that
Google lists, they know who the top search engine is even if MS *still*
doesn't. ;-)

This page still does not have <meta name="robots" content="ALL"> in the head
so I assume there may be others that don't. It may not be absolute necessity
to have this tag on every page but what can it hurt? I always plan for the
worst and hope for the best.

http://www.stevenhenson.com/spider_map.htm

There's also a trailing slash after the word "ALL" in the tag you're using.
I don't know if this will cause a problem but I would omit it just in case.

If you have submitted your site recently it may (surely?) take two weeks to
several months before it gets listed.
Good Luck,
me

Thank you!

I have added the robots ALL tag to the spider_map page and submitted that page
to Google. Not sure if this will work


It won't. It's pointless other than it takes up space.
but the spider_map page is a page
deliberately designed to make life easy for the spiders...so hopefully we might
make some headway!


How many links on it, by the way?

BB
--
www.kruse.co.uk/ SE*@kruse.demon.co.uk
Affordable SEO!
--
Jul 23 '05 #13
On Thu, 10 Mar 2005 08:23:50 -0600, "me" <anonymous@_.com> wrote:
"Steve" <pl***************@ireland.com> wrote in message
news:_m*******************@news.indigo.ie...
I have worked on a couple of sites which google's bot visits, partially

lists
and then goes away again.

MSN and Yahoo are fine and working.

Can anyone please suggest what, if anything, is wrong with these sites?

Frankly
I am a bit baffled at the moment! I am wondering if there is a problem

with the
page headers and googlebot?
All suggestions appreciated.
Thanks in advance,

Steve

Sites & Google results are:-

http://tinyurl.com/6tr2b

http://tinyurl.com/5dj6w


Don't assume that if you submit your site to one engine the others will also
pick it up (unless it's Google). Submit your site to Google directly:

http://www.google.com/addurl/

Incidentally the others search engines do eventually pick up any site that
Google lists, they know who the top search engine is even if MS *still*
doesn't. ;-)

This page still does not have <meta name="robots" content="ALL"> in the head
so I assume there may be others that don't. It may not be absolute necessity
to have this tag on every page but what can it hurt? I always plan for the
worst and hope for the best.

http://www.stevenhenson.com/spider_map.htm

There's also a trailing slash after the word "ALL" in the tag you're using.
I don't know if this will cause a problem but I would omit it just in case.

If you have submitted your site recently it may (surely?) take two weeks to
several months before it gets listed.
Good Luck,
me


(?)

BB
--
www.kruse.co.uk/ SE*@kruse.demon.co.uk
Affordable SEO!
--
Jul 23 '05 #14
Tim
"me" <anonymous@_.com> wrote:
Place this in the head of every page:

<meta name="robots" content="ALL">

Big Bill <kr***@cityscape.co.uk> posted:
It won't do anything. Robots ignore meta tags like that as they index
what they can anyway by default.


Perhaps that particular example might be ignored, but some robots do pay
attention to robot meta statements in the HTML head. Mostly about what
they should ignore, in some way, rather than what they should look at.

I recommend having a look through <http://www.google.com/webmasters/> for
more information about the robots, as well as why Google might be ignoring
the site. For instance, your wad of meta keywords might be doing you more
harm than good.

--
If you insist on e-mailing me, use the reply-to address (it's real but
temporary). But please reply to the group, like you're supposed to.

This message was sent without a virus, please delete some files yourself.
Jul 23 '05 #15
Tim
On Thu, 10 Mar 2005 08:23:50 -0600,
"me" <anonymous@_.com> posted:
There's also a trailing slash after the word "ALL" in the tag you're using.
I don't know if this will cause a problem but I would omit it just in case.


If the page really is the XHTML that it claims to be, the slash at the end
of the meta element belongs where it is, and should *not* be removed.

--
If you insist on e-mailing me, use the reply-to address (it's real but
temporary). But please reply to the group, like you're supposed to.

This message was sent without a virus, please delete some files yourself.
Jul 23 '05 #16
Tim wrote:

I recommend having a look through <http://www.google.com/webmasters/> for
more information about the robots, as well as why Google might be ignoring
the site. For instance, your wad of meta keywords might be doing you more
harm than good.


Thanks Tim. I am going to slim down the meta keywords just as soon as I can get
the PR people to agree to this........I never wanted that many in the first place!

Have checked out the webmasters stuff from Google site. At the moment I do not
see anything that stands out as being a glaring error.

Regards,

Steve
Jul 23 '05 #17
Tim wrote:
On Thu, 10 Mar 2005 08:23:50 -0600,
"me" <anonymous@_.com> posted:

There's also a trailing slash after the word "ALL" in the tag you're using.
I don't know if this will cause a problem but I would omit it just in case.

If the page really is the XHTML that it claims to be, the slash at the end
of the meta element belongs where it is, and should *not* be removed.

It was set-up at XHTML 1.0 Transitional. Point noted.
Jul 23 '05 #18
> How many links on it, by the way?

BB
--
www.kruse.co.uk/ SE*@kruse.demon.co.uk
Affordable SEO!
--


Bill,

Its all links and page title information. Is that an issue?

Steve
Jul 23 '05 #19
me
"Steve" <pl***************@ireland.com> wrote in message
news:Ln*******************@news.indigo.ie...
Tim wrote:
On Thu, 10 Mar 2005 08:23:50 -0600,
"me" <anonymous@_.com> posted:

There's also a trailing slash after the word "ALL" in the tag you're using.I don't know if this will cause a problem but I would omit it just in
case.

If the page really is the XHTML that it claims to be, the slash at the end of the meta element belongs where it is, and should *not* be removed.

It was set-up at XHTML 1.0 Transitional. Point noted.


I'm curious, how does setting up that page as XHTML benefit you, what
specifically does it do?
Signed,
me
Jul 23 '05 #20
>>It was set-up at XHTML 1.0 Transitional. Point noted.


I'm curious, how does setting up that page as XHTML benefit you, what
specifically does it do?
Signed,
me

I am not sure - that is the way the guy before me had set the whole thing up in,
dare I say it, dreamweaver.

As I understand it XHTML 1.0 trans is just a reformulation of HTML 4.0 - but in
XML. This page explains....its been out for about 3 years now I beleive!

http://www.w3.org/TR/xhtml1/#diffs

Steve
Jul 23 '05 #21
Tim
Tim wrote:
I recommend having a look through <http://www.google.com/webmasters/>
for more information about the robots, as well as why Google might be
ignoring the site. For instance, your wad of meta keywords might be
doing you more harm than good.

Steve wrote:
Thanks Tim. I am going to slim down the meta keywords just as soon as I
can get the PR people to agree to this........I never wanted that many in
the first place!


They're mostly useless, anyway. None of the worthwhile search engines pay
any attention to them, now (so most people say). And search engines will
eventually get better at relating terms against queries (i.e. they'll have
a table of alternatives for the same things).

But really, keywords and descriptions should be about the page that
they're on, not about other pages within the website. Search engines will
find them, by themselves. And no matter what you do, people will arrive
via a search engine directly at the page that seems most appropriate to
the search, not the homepage.

--
If you insist on e-mailing me, use the reply-to address (it's real but
temporary). But please reply to the group, like you're supposed to.

This message was sent without a virus, please delete some files yourself.

Jul 23 '05 #22
Tim
On Fri, 11 Mar 2005 08:28:21 -0600, me wrote:
I'm curious, how does setting up that page as XHTML benefit you


It doesn't benefit anyone. XHTML offers nothing as an improvement at this
stage, except more authoring, webserving, and browsing problems.

--
If you insist on e-mailing me, use the reply-to address (it's real but
temporary). But please reply to the group, like you're supposed to.

This message was sent without a virus, please delete some files yourself.

Jul 23 '05 #23
On Fri, 11 Mar 2005 05:36:45 GMT, Big Bill <kr***@cityscape.co.uk>
wrote:
On Thu, 10 Mar 2005 07:22:54 GMT, Adrienne <ar********@sbcglobal.net>
wrote:
Gazing into my crystal ball I observed Big Bill <kr***@cityscape.co.uk>
writing in news:18********************************@4ax.com:
On Wed, 9 Mar 2005 09:01:36 -0600, "me" <anonymous@_.com> wrote:

"Steve" <pl***************@ireland.com> wrote in message
news:_m*******************@news.indigo.ie...
> I have worked on a couple of sites which google's bot visits, partially
> lists and then goes away again.
>
> MSN and Yahoo are fine and working.
>
> Can anyone please suggest what, if anything, is wrong with these sites?
> Frankly I am a bit baffled at the moment! I am wondering if there is a
> problem with the page headers and googlebot?
> All suggestions appreciated.
> Thanks in advance,
> Steve
>
> Sites & Google results are:-
>
> http://tinyurl.com/6tr2b
>
> http://tinyurl.com/5dj6w

Place this in the head of every page:

<meta name="robots" content="ALL">

IIRC bots may enter from any page so this may help.
Good Luck,
me

It won't do anything. Robots ignore meta tags like that as they index
what they can anyway by default. It would be useful if the relevant
robots.txts were displayed here.

BB.

--
www.kruse.co.uk/ SE*@kruse.demon.co.uk
Affordable SEO!
--


1. http://www.barrabooks.com/robots.txt

User-agent: *
Disallow: /picture_library/
Disallow: /Store/
Disallow: /CCS/
Disallow: /webstat/
Disallow: /plesk-stat/
Disallow: /php/

2. http://www.stevenhenson.com/robots.txt

User-agent: *
Disallow: /picture_library/
Disallow: /Store/
Disallow: /CCS/
Disallow: /webstat/
Disallow: /plesk-stat/
Disallow: /php/

I don't see anything unusual do you?


No but something's somewhere. I'll have to come back to this as I'm
trying to download a windows update and watch Zev fron Lexx in the
shower just at the mo.

BB


Phew! Well, they both validate ok.

BB
--
www.kruse.co.uk/ SE*@kruse.demon.co.uk
Affordable SEO!
--
Jul 23 '05 #24
On Fri, 11 Mar 2005 20:42:36 +1030, Tim <ti*@mail.localhost.invalid>
wrote:
"me" <anonymous@_.com> wrote:
Place this in the head of every page:

<meta name="robots" content="ALL">

Big Bill <kr***@cityscape.co.uk> posted:
It won't do anything. Robots ignore meta tags like that as they index
what they can anyway by default.


Perhaps that particular example might be ignored, but some robots do pay
attention to robot meta statements in the HTML head. Mostly about what
they should ignore, in some way, rather than what they should look at.

I recommend having a look through <http://www.google.com/webmasters/> for
more information about the robots,


Have done. Every now and then though, out in the real world away from
what engines fondly imagine is determined by their guidelines, you
hear from reputable sources of instances where robots meta tags are
blindly ignored.

BB
as well as why Google might be ignoring
the site. For instance, your wad of meta keywords might be doing you more
harm than good.


--
www.kruse.co.uk/ SE*@kruse.demon.co.uk
Affordable SEO!
--
Jul 23 '05 #25
On Fri, 11 Mar 2005 10:28:15 +0000, Steve
<pl***************@ireland.com> wrote:
How many links on it, by the way?

BB
--
www.kruse.co.uk/ SE*@kruse.demon.co.uk
Affordable SEO!
--


Bill,

Its all links and page title information. Is that an issue?


How many though? Google doesn't seem too keen on monster links pages,
site maps or not.

BB

--
www.kruse.co.uk/ SE*@kruse.demon.co.uk
Affordable SEO!
--
Jul 23 '05 #26
Tim
Tim wrote:
I recommend having a look through <http://www.google.com/webmasters/> for
more information about the robots,


Big Bill <kr***@cityscape.co.uk> posted:
Have done. Every now and then though, out in the real world away from
what engines fondly imagine is determined by their guidelines, you
hear from reputable sources of instances where robots meta tags are
blindly ignored.


Since the question was about the Googlebot, it's probably going to be the
first place to look, though, to find out why Google mightn't be indexing
pages that it's apparently had the chance to (i.e. it's probably more to do
with the contents, e.g. bad HTML authoring techiques and search engine
scamming methods, than messing with robot instructions).

Yes, there's robots which ignore instructions to ignore parts of sites
because there might be something juicy there if the webmaster's trying to
hide it. But you've got Buckley's chance of inducing a robot to look at a
page that it'd already ignored, merely by putting tempting robot
instructions on a page.

--
If you insist on e-mailing me, use the reply-to address (it's real but
temporary). But please reply to the group, like you're supposed to.

This message was sent without a virus, please delete some files yourself.
Jul 23 '05 #27
me
"Tim" <ti*@mail.localhost.invalid> wrote in message
news:pa****************************@mail.localhost .invalid...
On Fri, 11 Mar 2005 08:28:21 -0600, me wrote:
I'm curious, how does setting up that page as XHTML benefit you


It doesn't benefit anyone. XHTML offers nothing as an improvement at this
stage, except more authoring, webserving, and browsing problems.


Then why bother?
Signed,
me
Jul 23 '05 #28
In article <11*************@corp.supernews.com>, anonymous@_.com enlightened
us with...

It doesn't benefit anyone. XHTML offers nothing as an improvement at this
stage, except more authoring, webserving, and browsing problems.


Then why bother?
Signed,
me


To look cool? ;)
--
--
~kaeli~
Hey, if you got it flaunt it! If you don't, stare at
someone who does. Just don't lick the TV screen, it leaves
streaks.
http://www.ipwebdesign.net/wildAtHeart
http://www.ipwebdesign.net/kaelisSpace

Jul 23 '05 #29
Tim
me wrote:
I'm curious, how does setting up that page as XHTML benefit you

"Tim" <ti*@mail.localhost.invalid> wrote
It doesn't benefit anyone. XHTML offers nothing as an improvement at this
stage, except more authoring, webserving, and browsing problems.

"me" <anonymous@_.com> posted:
Then why bother?


Good question. Usually people do it without any real clue about why.

At this stage in the game few browsers support it properly, so it's bad
news to publish pages that are going to get mangled by some browsers even
worse than they're already mangling ordinary HTML. To minimise this,
people serve it out as if it were HTML, hoping that it'll work in more
browsers. As such, it holds no advantages over serving it *as* HTML.

In the future it holds the *potential* for better authored webpages, when
more browsers support it better. *BUT* it looks highly likely that
browsers will be kludged to bits to support badly written XHTML, so that it
doesn't hold any advantage at all. It'll be just the same mess as current
tag-soup HTML parsing.

And even if browsers did manage to use HTML properly, authors do not. Just
ensuring that you've put your li tags properly within your ul tags, and so
on, is only half the task. You've actually got to use HTML elements for
their proper uses so that user-agents can make full use of the information
contained in them. Until people do that, they're still writing gibberish.

--
If you insist on e-mailing me, use the reply-to address (it's real but
temporary). But please reply to the group, like you're supposed to.

This message was sent without a virus, please delete some files yourself.
Jul 23 '05 #30

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

3
by: Stephen Ferg | last post by:
I use the newsgroup mainly via my Web browser and Google. Over the last few days, when I click on a link to a thread, I've been frequently getting the message "Unable to find thread." Does...
13
by: Mario | last post by:
Hello everybody, I am looking for a good script to work a searchprogram for Google .... How can tel me? Not with a logo from google THnx -- -------"""------- ---()--- °?----(_)-----?° |...
3
by: Alastair | last post by:
Hello guys, I've been building a search facility for an intranet site I'm part of developing and we've been building a search engine using Index Server. It mostly works, however there have been...
87
by: ziliath | last post by:
I recently tried out the Google "top coder" contest, as a C++ coder. I noticed immediately that they expected me to know STL. To which I say, what the fuck?! I may be missing something, but at...
18
by: smileplzz | last post by:
r there any files which we can download from this group. can we only ask questions in this group? no files stored like it is there in yahoo groups. i think it should be there in group services....
3
by: Johann Blake | last post by:
This aticle presents factual evidence that Google's PageRank, inbound links and keywords are irrelevent to your placement in the search results. It presents several case studies that show...
36
by: ern | last post by:
I inherited a huge C application and most of it is in one file. I'd like to modularize it into several different files. Is there a good resource for learning the best way to link C files...
0
by: Andrew_Vodo | last post by:
Hi, I'm developning ASP.Net 2.0 application which will use Google Map. The page with Google Map uses master page. As Google map uses client-side script, the first problem is where do I have...
2
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 7 Feb 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:30 (7.30PM). In this month's session, the creator of the excellent VBE...
0
by: DolphinDB | last post by:
The formulas of 101 quantitative trading alphas used by WorldQuant were presented in the paper 101 Formulaic Alphas. However, some formulas are complex, leading to challenges in calculation. Take...
0
by: DolphinDB | last post by:
Tired of spending countless mintues downsampling your data? Look no further! In this article, you’ll learn how to efficiently downsample 6.48 billion high-frequency records to 61 million...
0
by: Aftab Ahmad | last post by:
Hello Experts! I have written a code in MS Access for a cmd called "WhatsApp Message" to open WhatsApp using that very code but the problem is that it gives a popup message everytime I clicked on...
0
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 6 Mar 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:15 (7.15PM). In this month's session, we are pleased to welcome back...
1
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 6 Mar 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:15 (7.15PM). In this month's session, we are pleased to welcome back...
0
by: Vimpel783 | last post by:
Hello! Guys, I found this code on the Internet, but I need to modify it a little. It works well, the problem is this: Data is sent from only one cell, in this case B5, but it is necessary that data...
0
by: jfyes | last post by:
As a hardware engineer, after seeing that CEIWEI recently released a new tool for Modbus RTU Over TCP/UDP filtering and monitoring, I actively went to its official website to take a look. It turned...
0
by: ArrayDB | last post by:
The error message I've encountered is; ERROR:root:Error generating model response: exception: access violation writing 0x0000000000005140, which seems to be indicative of an access violation...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.