473,799 Members | 2,924 Online
Bytes | Software Development & Data Engineering Community
+ Post

Home Posts Topics Members FAQ

Google Bot problems?

I have worked on a couple of sites which google's bot visits, partially lists
and then goes away again.

MSN and Yahoo are fine and working.

Can anyone please suggest what, if anything, is wrong with these sites? Frankly
I am a bit baffled at the moment! I am wondering if there is a problem with the
page headers and googlebot?

All suggestions appreciated.

Thanks in advance,


Steve

Sites & Google results are:-

http://tinyurl.com/6tr2b

http://tinyurl.com/5dj6w
Jul 23 '05 #1
29 3276
me
"Steve" <pl************ ***@ireland.com > wrote in message
news:_m******** ***********@new s.indigo.ie...
I have worked on a couple of sites which google's bot visits, partially lists and then goes away again.

MSN and Yahoo are fine and working.

Can anyone please suggest what, if anything, is wrong with these sites? Frankly I am a bit baffled at the moment! I am wondering if there is a problem with the page headers and googlebot?
All suggestions appreciated.
Thanks in advance,
Steve

Sites & Google results are:-

http://tinyurl.com/6tr2b

http://tinyurl.com/5dj6w


Place this in the head of every page:

<meta name="robots" content="ALL">

IIRC bots may enter from any page so this may help.
Good Luck,
me
Jul 23 '05 #2
>
Place this in the head of every page:

<meta name="robots" content="ALL">

IIRC bots may enter from any page so this may help.
Good Luck,
me


Thanks me...will give it a go!

Steve
Jul 23 '05 #3
On Wed, 09 Mar 2005 19:07:34 +0000, Steve
<pl************ ***@ireland.com > wrote:

Place this in the head of every page:

<meta name="robots" content="ALL">

IIRC bots may enter from any page so this may help.
Good Luck,
me


Thanks me...will give it a go!

Steve


Thoughtful of me to mentin it but it won't have any effect. Please may
we see the robots txt for each site.

BB
--
www.kruse.co.uk/ SE*@kruse.demon .co.uk
Affordable SEO!
--
Jul 23 '05 #4
On Wed, 9 Mar 2005 09:01:36 -0600, "me" <anonymous@_.co m> wrote:
"Steve" <pl************ ***@ireland.com > wrote in message
news:_m******* ************@ne ws.indigo.ie...
I have worked on a couple of sites which google's bot visits, partially

lists
and then goes away again.

MSN and Yahoo are fine and working.

Can anyone please suggest what, if anything, is wrong with these sites?

Frankly
I am a bit baffled at the moment! I am wondering if there is a problem

with the
page headers and googlebot?
All suggestions appreciated.
Thanks in advance,
Steve

Sites & Google results are:-

http://tinyurl.com/6tr2b

http://tinyurl.com/5dj6w


Place this in the head of every page:

<meta name="robots" content="ALL">

IIRC bots may enter from any page so this may help.
Good Luck,
me


It won't do anything. Robots ignore meta tags like that as they index
what they can anyway by default. It would be useful if the relevant
robots.txts were displayed here.

BB.

--
www.kruse.co.uk/ SE*@kruse.demon .co.uk
Affordable SEO!
--
Jul 23 '05 #5
Gazing into my crystal ball I observed Big Bill <kr***@cityscap e.co.uk>
writing in news:18******** *************** *********@4ax.c om:
On Wed, 9 Mar 2005 09:01:36 -0600, "me" <anonymous@_.co m> wrote:
"Steve" <pl************ ***@ireland.com > wrote in message
news:_m****** *************@n ews.indigo.ie.. .
I have worked on a couple of sites which google's bot visits, partially
lists and then goes away again.

MSN and Yahoo are fine and working.

Can anyone please suggest what, if anything, is wrong with these sites?
Frankly I am a bit baffled at the moment! I am wondering if there is a
problem with the page headers and googlebot?
All suggestions appreciated.
Thanks in advance,
Steve

Sites & Google results are:-

http://tinyurl.com/6tr2b

http://tinyurl.com/5dj6w


Place this in the head of every page:

<meta name="robots" content="ALL">

IIRC bots may enter from any page so this may help.
Good Luck,
me


It won't do anything. Robots ignore meta tags like that as they index
what they can anyway by default. It would be useful if the relevant
robots.txts were displayed here.

BB.

--
www.kruse.co.uk/ SE*@kruse.demon .co.uk
Affordable SEO!
--


1. http://www.barrabooks.com/robots.txt

User-agent: *
Disallow: /picture_library/
Disallow: /Store/
Disallow: /CCS/
Disallow: /webstat/
Disallow: /plesk-stat/
Disallow: /php/

2. http://www.stevenhenson.com/robots.txt

User-agent: *
Disallow: /picture_library/
Disallow: /Store/
Disallow: /CCS/
Disallow: /webstat/
Disallow: /plesk-stat/
Disallow: /php/

I don't see anything unusual do you?

--
Adrienne Boswell
http://www.cavalcade-of-coding.info
Please respond to the group so others can share
Jul 23 '05 #6
> 1. http://www.barrabooks.com/robots.txt

User-agent: *
Disallow: /picture_library/
Disallow: /Store/
Disallow: /CCS/
Disallow: /webstat/
Disallow: /plesk-stat/
Disallow: /php/

2. http://www.stevenhenson.com/robots.txt

User-agent: *
Disallow: /picture_library/
Disallow: /Store/
Disallow: /CCS/
Disallow: /webstat/
Disallow: /plesk-stat/
Disallow: /php/

I don't see anything unusual do you?


Thank you Adrienne - I see nothing nasty in the robot.txt files as well! It
still leaves me baffled as to why Google is behaving this way.


Steve
Jul 23 '05 #7
> I have worked on a couple of sites which google's bot visits,
partially lists
and then goes away again.
MSN and Yahoo are fine and working.


How long have you waited? You do have to be patient. MSN have made
some claims about indexing more often than Google and I believe this
could be true. Google does find the front page for one of the sites.

(BB, the robots.txt files are at http://www.barrabooks.com/robots.txt
and http://www.stevenhenson.com/robots.txt.)

--Phil.

Jul 23 '05 #8
me
"Steve" <pl************ ***@ireland.com > wrote in message
news:_m******** ***********@new s.indigo.ie...
I have worked on a couple of sites which google's bot visits, partially lists and then goes away again.

MSN and Yahoo are fine and working.

Can anyone please suggest what, if anything, is wrong with these sites? Frankly I am a bit baffled at the moment! I am wondering if there is a problem with the page headers and googlebot?
All suggestions appreciated.
Thanks in advance,

Steve

Sites & Google results are:-

http://tinyurl.com/6tr2b

http://tinyurl.com/5dj6w


Don't assume that if you submit your site to one engine the others will also
pick it up (unless it's Google). Submit your site to Google directly:

http://www.google.com/addurl/

Incidentally the others search engines do eventually pick up any site that
Google lists, they know who the top search engine is even if MS *still*
doesn't. ;-)

This page still does not have <meta name="robots" content="ALL"> in the head
so I assume there may be others that don't. It may not be absolute necessity
to have this tag on every page but what can it hurt? I always plan for the
worst and hope for the best.

http://www.stevenhenson.com/spider_map.htm

There's also a trailing slash after the word "ALL" in the tag you're using.
I don't know if this will cause a problem but I would omit it just in case.

If you have submitted your site recently it may (surely?) take two weeks to
several months before it gets listed.
Good Luck,
me
Jul 23 '05 #9
Gazing into my crystal ball I observed Steve
<pl************ ***@ireland.com > writing in
news:Fc******** ***********@new s.indigo.ie:
1. http://www.barrabooks.com/robots.txt

User-agent: *
Disallow: /picture_library/
Disallow: /Store/
Disallow: /CCS/
Disallow: /webstat/
Disallow: /plesk-stat/
Disallow: /php/

2. http://www.stevenhenson.com/robots.txt

User-agent: *
Disallow: /picture_library/
Disallow: /Store/
Disallow: /CCS/
Disallow: /webstat/
Disallow: /plesk-stat/
Disallow: /php/

I don't see anything unusual do you?


Thank you Adrienne - I see nothing nasty in the robot.txt files as
well! It still leaves me baffled as to why Google is behaving this way.


Not sure if this is what is causing it, but on www.stevenhenson.com there
are some markup errors that might be confusing Google. Google _might_
think it is going in an endless loop, therefore getting out before it
sticks its toe in the water.<http://validator.w3.org/check?uri=http%3A%2F%
2Fwww.stevenhen son.com%2F&char set=%28detect+a utomatically%29 &doctype=%
28detect+automa tically%29&ss=1 >

www.barrabooks.com does not have any markup errors, but it could be
upgraded to stylesheets instead of tables, and use semantic markup, eg:
<span class="style2 xbig"><strong>W elcome to Barra Books</strong></span>
should be
<h1>Welcome to Barra Books</h1>

--
Adrienne Boswell
http://www.cavalcade-of-coding.info
Please respond to the group so others can share
Jul 23 '05 #10

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

3
1910
by: Stephen Ferg | last post by:
I use the newsgroup mainly via my Web browser and Google. Over the last few days, when I click on a link to a thread, I've been frequently getting the message "Unable to find thread." Does anybody know what's going on?
13
2371
by: Mario | last post by:
Hello everybody, I am looking for a good script to work a searchprogram for Google .... How can tel me? Not with a logo from google THnx -- -------"""------- ---()--- °?----(_)-----?° | Greetz Mario|
3
4209
by: Alastair | last post by:
Hello guys, I've been building a search facility for an intranet site I'm part of developing and we've been building a search engine using Index Server. It mostly works, however there have been a few niggling problems and fixing it seems to be a case of patching errors as we find them so I'm thinking that it might be worth starting the logic from scratch and rebuilding this again. Basically we have a simple search, which is simply a...
87
5264
by: ziliath | last post by:
I recently tried out the Google "top coder" contest, as a C++ coder. I noticed immediately that they expected me to know STL. To which I say, what the fuck?! I may be missing something, but at what point when learning C++ was I supposed to have picked up STL? I ask because at every job I've had, and every job interview for that matter, whenever STL comes up it's not I, but a manager who says effectively yuck, we don't use that crap.
18
3868
by: smileplzz | last post by:
r there any files which we can download from this group. can we only ask questions in this group? no files stored like it is there in yahoo groups. i think it should be there in group services. ur new group member Ram 20/m/India
3
3360
by: Johann Blake | last post by:
This aticle presents factual evidence that Google's PageRank, inbound links and keywords are irrelevent to your placement in the search results. It presents several case studies that show conclusively that the algorithm used by Google's search engine is flawed with serious "bugs" that deny many websites the high ranking that they should otherwise be granted while granting it to those who shouldn't have it. The article is entitled...
36
1837
by: ern | last post by:
I inherited a huge C application and most of it is in one file. I'd like to modularize it into several different files. Is there a good resource for learning the best way to link C files together... and maybe some good conventions to use? Thanks,
0
1311
by: Andrew_Vodo | last post by:
Hi, I'm developning ASP.Net 2.0 application which will use Google Map. The page with Google Map uses master page. As Google map uses client-side script, the first problem is where do I have to run Google Map creation and population script? I can't run it in master page on load, as this master page is used by another pages, which do not use Google map.
0
9685
marktang
by: marktang | last post by:
ONU (Optical Network Unit) is one of the key components for providing high-speed Internet services. Its primary function is to act as an endpoint device located at the user's premises. However, people are often confused as to whether an ONU can Work As a Router. In this blog post, we’ll explore What is ONU, What Is Router, ONU & Router’s main usage, and What is the difference between ONU and Router. Let’s take a closer look ! Part I. Meaning of...
0
10473
Oralloy
by: Oralloy | last post by:
Hello folks, I am unable to find appropriate documentation on the type promotion of bit-fields when using the generalised comparison operator "<=>". The problem is that using the GNU compilers, it seems that the internal comparison operator "<=>" tries to promote arguments from unsigned to signed. This is as boiled down as I can make it. Here is my compilation command: g++-12 -std=c++20 -Wnarrowing bit_field.cpp Here is the code in...
0
10249
jinu1996
by: jinu1996 | last post by:
In today's digital age, having a compelling online presence is paramount for businesses aiming to thrive in a competitive landscape. At the heart of this digital strategy lies an intricately woven tapestry of website design and digital marketing. It's not merely about having a website; it's about crafting an immersive digital experience that captivates audiences and drives business growth. The Art of Business Website Design Your website is...
1
10219
by: Hystou | last post by:
Overview: Windows 11 and 10 have less user interface control over operating system update behaviour than previous versions of Windows. In Windows 11 and 10, there is no way to turn off the Windows Update option using the Control Panel or Settings app; it automatically checks for updates and installs any it finds, whether you like it or not. For most users, this new feature is actually very convenient. If you want to control the update process,...
0
9068
agi2029
by: agi2029 | last post by:
Let's talk about the concept of autonomous AI software engineers and no-code agents. These AIs are designed to manage the entire lifecycle of a software development project—planning, coding, testing, and deployment—without human intervention. Imagine an AI that can take a project description, break it down, write the code, debug it, and then launch it, all on its own.... Now, this would greatly impact the work of software developers. The idea...
0
6804
by: conductexam | last post by:
I have .net C# application in which I am extracting data from word file and save it in database particularly. To store word all data as it is I am converting the whole word file firstly in HTML and then checking html paragraph one by one. At the time of converting from word file to html my equations which are in the word document file was convert into image. Globals.ThisAddIn.Application.ActiveDocument.Select();...
0
5461
by: TSSRALBI | last post by:
Hello I'm a network technician in training and I need your help. I am currently learning how to create and manage the different types of VPNs and I have a question about LAN-to-LAN VPNs. The last exercise I practiced was to create a LAN-to-LAN VPN between two Pfsense firewalls, by using IPSEC protocols. I succeeded, with both firewalls in the same network. But I'm wondering if it's possible to do the same thing, with 2 Pfsense firewalls...
0
5584
by: adsilva | last post by:
A Windows Forms form does not have the event Unload, like VB6. What one acts like?
3
2937
bsmnconsultancy
by: bsmnconsultancy | last post by:
In today's digital era, a well-designed website is crucial for businesses looking to succeed. Whether you're a small business owner or a large corporation in Toronto, having a strong online presence can significantly impact your brand's success. BSMN Consultancy, a leader in Website Development in Toronto offers valuable insights into creating effective websites that not only look great but also perform exceptionally well. In this comprehensive...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.