utf-8 and xhtml 1.0 strict

[The Bicycling Guitarist]

My web site has not been spidered by Googlebot since April 2003. The site in
question is at www.TheBicyclingGuitarist.net/ I received much help from this
NG and the stylesheets NG when updating the code before then.

My host's tech guy just sent me the following. Isn't it okay to specify
UTF-8 as the charset in the HTTP headers at the server level? Isn't it okay
to have validated XHTML 1.0 strict code?

*************** *************** *************** *************** *

If it was an misconfiguratio n with IIS the problem would be presenting
itself for every site that is hosted on that server under that instance of
IIS which isn't the case here. I have only been able to find two
differences between your site which Google isn't updated, and the sites that
are.

1) You have a custom charset also specified in the HTTP headers at the
server level
2) You are using XHTML strict.

I am curious why you chose XHTML strict rather than traditional? Here's a
quote from broadbandreport s.com with the full link at
http://www.broadbandreports.com/faq/...ebmonks?text=1

"If you are using XHTML you should strive to make your pages validate as
XHTML 1.0 Transitional. The XHTML 1.0 Strict standard is a bit too confining
for real world web sites."

My suggestion is still that you talk to Google to find out why their bot
both is getting a 406 error, and why it isn't updating the content it isn't
getting an error on. If you would like I would be happy to reset the HTTP
headers to the default setting so your site identically matches every other
site hosted on this server as far as IIS goes.

----- Original Message -----
From: Chris Watson
To: XXXXXXXXXXXXX
Sent: Tuesday, October 26, 2004 5:16 PM
Subject: RE: Jesse, you really need to see this.
I posted the information from your last two emails in one message at a
search engines newsgroup. What about this guy's answer? It's short and
sweet.

The Bicycling Guitarist wrote:

The following are two messages from the tech guy at my host concerning my
problems with Googlebot or vice versa.

The problem seems to be with your IIS configuration. Google sends
Accept: text/html,text/plain; which of course makes good sense for a
robot as it doesn't want anything else. Your IIS appears to be
incorrectly configured to send a 406 not acceptable message when it sees
this.

If you accept text/* you get your page. It doesn't seem to be linked to
the charset.

[Neal]

On Thu, 28 Oct 2004 06:26:27 GMT, The Bicycling Guitarist
<Ch***@TheBicyc lingGuitarist.n et> wrote:

My web site has not been spidered by Googlebot since April 2003. The
site in
question is at www.TheBicyclingGuitarist.net/ I received much help from
this
NG and the stylesheets NG when updating the code before then.

My host's tech guy just sent me the following. Isn't it okay to specify
UTF-8 as the charset in the HTTP headers at the server level? Isn't it
okay
to have validated XHTML 1.0 strict code?

I replied in alt.html - yep, UTF-8 and XHTML (served as text/html) has put
my site at PR4 and is on top for my keywords. The problem must lie
elsewhere.

[Andreas Prilop]

On Thu, 28 Oct 2004, The Bicycling Guitarist wrote:

My web site has not been spidered by Googlebot since April 2003.

My host's tech guy just sent me the following. Isn't it okay to specify
UTF-8 as the charset in the HTTP headers at the server level?
It certainly is.
<http://google.com/search?q=www.un ics.uni-hannover.de/nhtcapri/multilingual1.h tml>
Isn't it okay
to have validated XHTML 1.0 strict code?
I think it's okay. But why are you writing XHTML 1.0 instead of HTML 4.01?
Is there any good reason or is it just "kewl"?
2) You are using XHTML strict.
I am curious why you chose XHTML strict rather than traditional?

"If you are using XHTML you should strive to make your pages validate as
XHTML 1.0 Transitional. The XHTML 1.0 Strict standard is a bit too confining
for real world web sites."

This is absurd because any document that validates as [X]HTML Strict also
validates as [X]HTML Transitional. This person is clueless.

I have no indication that Google struggles with XHTML - but what's your
reason to write XHTML 1.0 rather than HTML 4.01 Strict?

--
Top-posting.
What's the most irritating thing on Usenet?

[The Bicycling Guitarist]

"Andreas Prilop" <nh******@rrz n-user.uni-hannover.de> wrote in message
news:Pine.GSO.4 .44.04102814391 80.12878-100000@s5b003.. .

On Thu, 28 Oct 2004, The Bicycling Guitarist wrote:

My web site has not been spidered by Googlebot since April 2003.

My host's tech guy just sent me the following. Isn't it >> 2) You are
using XHTML strict.
I am curious why you chose XHTML strict rather than traditional?

"If you are using XHTML you should strive to make your pages validate as
XHTML 1.0 Transitional. The XHTML 1.0 Strict standard is a bit too
confining
for real world web sites."
This is absurd because any document that validates as [X]HTML Strict also
validates as [X]HTML Transitional. This person is clueless.

I will point this out to him.

I have no indication that Google struggles with XHTML - but what's your
reason to write XHTML 1.0 rather than HTML 4.01 Strict?

For Duty and Humanity! I wish to make my pages more available to different
user-agents as they develop, and to make it easier to convert to xml in the
future.

If the tech really is clueless, then perhaps he hasn't configured the server
correctly even if he thinks he has. Can this be tested by somebody other
than him?

Chris Watson
www.TheBicyclingGuitarist.net/

..

[Alan J. Flavell]

On Thu, 28 Oct 2004, The Bicycling Guitarist wrote:

My host's tech guy just sent me the following.
Sounds to me like "grasp as any conceivable excuse and see if
the customer is clueless enough to swallow it".
Isn't it okay to specify UTF-8 as the charset in the HTTP headers at
the server level?
Not only is it "OK", but it's also strongly recommended.
Isn't it okay to have validated XHTML 1.0 strict code?
Only a super-purist (i.e me on a bad day) would complain that Appendix
C is hostile to the very intentions of XML. However, a more practical
person might counsel you to carry on using HTML/4.01 (by all means
"strict") until you're ready to go for the full XHTML+XML stuff. Or
rather, until the rest of the world is ready for you ;-))
If it was an misconfiguratio n with IIS
Oh, blimey. Is -that- the server they're using....
the problem would be presenting itself for every site that is hosted
on that server under that instance of IIS which isn't the case here.
The chap's plausible, at least... but is it technically well
founded?...
1) You have a custom charset also specified in the HTTP headers at the
server level
I wouldn't exactly refer to utf-8 as "custom" !!!
2) You are using XHTML strict.

I am curious why you chose XHTML strict rather than traditional?
I'm confident that this issue is irrelevant to your original question.
Here's a
quote from broadbandreport s.com [...]

"If you are using XHTML you should strive to make your pages validate as
XHTML 1.0 Transitional. The XHTML 1.0 Strict standard is a bit too confining
for real world web sites."
I would take the opposite point of view. If starting from HTML4.*
transitional, my first priority would be to head in the direction of
"strict" (with CSS for presentation), than to worry for the moment
about XHTML as such.

As far as I'm concerned, XHTML/1.0 "transition al" is merely a formal
rewriting into XML notation of an HTML/4.01 specification that was
already obsolescent at the time, and is surely inappropriate now for
new developments.
My suggestion is still that you talk to Google to find out why their
bot both is getting a 406 error,
Uh-uh, now we're getting to the -real- problem. Let's see this on the
lab bench:

$ telnet www.thebicyclingguitarist.net 80
Trying 216.229.101.149 ...
Connected to www.thebicyclingguitarist.net.
Escape character is '^]'.
GET / HTTP/1.0
Host: www.thebicyclingguitarist.net
Accept: text/html,text/plain

HTTP/1.1 406 No acceptable objects were found
Server: Microsoft-IIS/5.0
Date: Thu, 28 Oct 2004 13:51:57 GMT
Content-Length: 3906
Content-Type: text/html

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2 Final//EN">
<html dir=ltr>

[...and so on, yeuch...]
Once more with feeling:

$ telnet www.thebicyclingguitarist.net 80
Trying 216.229.101.149 ...
Connected to www.thebicyclingguitarist.net.
Escape character is '^]'.
GET / HTTP/1.0
Host: www.thebicyclingguitarist.net
Accept: text/html

HTTP/1.1 406 No acceptable objects were found
Server: Microsoft-IIS/5.0
Date: Thu, 28 Oct 2004 13:53:59 GMT

[...bleagh...]
You need to concentrate on why content-type negotiation is failing.

<prejudice type="Halloween Papers">If it was me, my first priority
would be to migrate to an Apache server.</>
If you would like I would be happy to reset the HTTP headers to the
default setting so your site identically matches every other site
hosted on this server as far as IIS goes.

Pffffffffffffff t.

This server is demonstrating its inability to implement content-type
negotiation correctly. As such, it's unfit for use on the WWW in this
state.

That's my best offer based on the evidence presented and the results
as far as I can see them.

[Invalid User]

Once upon a time *The Bicycling Guitarist* wrote:

"Andreas Prilop" <nh******@rrz n-user.uni-hannover.de> wrote in message
news:Pine.GSO.4 .44.04102814391 80.12878-100000@s5b003.. .

On Thu, 28 Oct 2004, The Bicycling Guitarist wrote:

My web site has not been spidered by Googlebot since April 2003.

My host's tech guy just sent me the following. Isn't it >> 2) You are
using XHTML strict.
I am curious why you chose XHTML strict rather than traditional?

"If you are using XHTML you should strive to make your pages validate as
XHTML 1.0 Transitional. The XHTML 1.0 Strict standard is a bit too
confining
for real world web sites."

This is absurd because any document that validates as [X]HTML Strict also
validates as [X]HTML Transitional. This person is clueless.

I will point this out to him.

I have small site in XHTML 1.1 and UTF-8 and I have no problem with
Google. So, your host's tech guy don't know what his talking about!

I have no indication that Google struggles with XHTML - but what's your
reason to write XHTML 1.0 rather than HTML 4.01 Strict?

For Duty and Humanity! I wish to make my pages more available to different
user-agents as they develop, and to make it easier to convert to xml in the
future.

I you want to use XHTML, why not go strait to XHTML 1.1 if you don't
need the transitional in 1.0 It's no big difference moving from 1.0
Strict to 1.1

In my opinion the transitional doctype should not be used in others than
frameset build sites, no matter if its XHTML 1.0 or HTML 4. The
"frameset" doctype must be used for the file containing the frameset,
and the transitional for other pages within the site makes the use of
the "target" attribut valid. But if you don't use frameset, you can go
to XHTML 1.1 directly

--
/Arne
http://w1.978.telia.com/~u97802964/

[Steve Pugh]

On Thu, 28 Oct 2004 14:53:30 GMT, Invalid User <us**@domain.in valid> wrote:

I have small site in XHTML 1.1 and UTF-8 and I have no problem with
Google. So, your host's tech guy don't know what his talking about!

But how do you get IE to handle it?

Or are you serving 1.1 as text/html ? Oh dear.

Steve

[Chris Morris]

Invalid User <us**@domain.in valid> writes:

Once upon a time *The Bicycling Guitarist* wrote:

"Andreas Prilop" <nh******@rrz n-user.uni-hannover.de> wrote in message

I have no indication that Google struggles with XHTML - but what's your
reason to write XHTML 1.0 rather than HTML 4.01 Strict?
For Duty and Humanity! I wish to make my pages more available to
different user-agents as they develop, and to make it easier to
convert to xml in the future.

Write XHTML and use tidy or similar to convert it to HTML before
serving? At the moment there are more user agents that support HTML
than there are that support XHTML, and I don't know of any that only
support XHTML (given that most pages don't validate to any doctype,
any that do exist are probably confined to the lab).

XHTML also has problems with incremental rendering.

Oh, and all those /s in XHTML add to the filesize. ;)
I you want to use XHTML, why not go strait to XHTML 1.1 if you don't
need the transitional in 1.0 It's no big difference moving from 1.0
Strict to 1.1

The Appendix C fix isn't allowed with XHTML 1.1, so you either serve
it against the specifications (and why go to something as obscure as
XHTML 1.1 if not to follow the specifications) , or you accept that
Internet Explorer users won't be able to view it.

--
Chris

[Leif K-Brooks]

The Bicycling Guitarist wrote:

My web site has not been spidered by Googlebot since April 2003. The site in
question is at www.TheBicyclingGuitarist.net/ I received much help from this
NG and the stylesheets NG when updating the code before then.

Seems to be sending the incorrect MIME type "text/*" instead of text/html:
[leif@localhost leif]$ telnet TheBicyclingGui tarist.net 80
Trying 216.229.101.149 ...
Connected to TheBicyclingGui tarist.net (216.229.101.14 9).
Escape character is '^]'.
GET / HTTP/1.1
Host: TheBicyclingGui tarist.net

HTTP/1.1 200 OK
Server: Microsoft-IIS/5.0
Content-Location: http://TheBicyclingGuitarist.net/index.htm
Date: Thu, 28 Oct 2004 18:24:11 GMT
Content-Type: text/*;charset=utf-8
Accept-Ranges: bytes
Last-Modified: Wed, 27 Oct 2004 19:48:04 GMT
ETag: "fab99adf5dbcc4 1:9f9"
Content-Length: 5169

<snip>

[Invalid User]

Once upon a time *Steve Pugh* wrote:

On Thu, 28 Oct 2004 14:53:30 GMT, Invalid User <us**@domain.in valid> wrote:

I have small site in XHTML 1.1 and UTF-8 and I have no problem with
Google. So, your host's tech guy don't know what his talking about!

But how do you get IE to handle it?

Or are you serving 1.1 as text/html ? Oh dear.

Yes, I have to if I want the IE users to have access to it.
What do you mean with "Oh dear"? Nothing wrong with text/html since it's
valid to do so, andf even recommended by W3C. However, it's just a small
personal "play ground" for testing purposes.

But basically, why use HTML 4.01 Strict, when XHTML is available. The
difference is not that big, but XHTML is more prepared for the future.
Therefore more XHTML coded sites is published almost every day now.

--
/Arne
http://w1.978.telia.com/~u97802964/