utf-8 and xhtml 1.0 strict

[The Bicycling Guitarist]

My web site has not been spidered by Googlebot since April 2003. The site in
question is at www.TheBicyclingGuitarist.net/ I received much help from this
NG and the stylesheets NG when updating the code before then.

My host's tech guy just sent me the following. Isn't it okay to specify
UTF-8 as the charset in the HTTP headers at the server level? Isn't it okay
to have validated XHTML 1.0 strict code?

*************** *************** *************** *************** *

If it was an misconfiguratio n with IIS the problem would be presenting
itself for every site that is hosted on that server under that instance of
IIS which isn't the case here. I have only been able to find two
differences between your site which Google isn't updated, and the sites that
are.

1) You have a custom charset also specified in the HTTP headers at the
server level
2) You are using XHTML strict.

I am curious why you chose XHTML strict rather than traditional? Here's a
quote from broadbandreport s.com with the full link at
http://www.broadbandreports.com/faq/...ebmonks?text=1

"If you are using XHTML you should strive to make your pages validate as
XHTML 1.0 Transitional. The XHTML 1.0 Strict standard is a bit too confining
for real world web sites."

My suggestion is still that you talk to Google to find out why their bot
both is getting a 406 error, and why it isn't updating the content it isn't
getting an error on. If you would like I would be happy to reset the HTTP
headers to the default setting so your site identically matches every other
site hosted on this server as far as IIS goes.

----- Original Message -----
From: Chris Watson
To: XXXXXXXXXXXXX
Sent: Tuesday, October 26, 2004 5:16 PM
Subject: RE: Jesse, you really need to see this.
I posted the information from your last two emails in one message at a
search engines newsgroup. What about this guy's answer? It's short and
sweet.

The Bicycling Guitarist wrote:

The following are two messages from the tech guy at my host concerning my
problems with Googlebot or vice versa.

The problem seems to be with your IIS configuration. Google sends
Accept: text/html,text/plain; which of course makes good sense for a
robot as it doesn't want anything else. Your IIS appears to be
incorrectly configured to send a 406 not acceptable message when it sees
this.

If you accept text/* you get your page. It doesn't seem to be linked to
the charset.

[Daniel R. Tobias]

Alan J. Flavell wrote:

My suspicion is that we're going to end up with XHTML-flavoured tag soup,
thus losing any of the benefits which XML was *supposed* to bring us.

We're seeing some of that already, in tag-soup pages in which
XHTML-style tags like <br /> are sprinkled randomly (interspersed with
their HTML-style equivalents like <br>).

--
== Dan ==
Dan's Mail Format Site: http://mailformat.dan.info/
Dan's Web Tips: http://webtips.dan.info/
Dan's Domain Site: http://domains.dan.info/

[Nick Kew]

In article <nc************ ****@newssvr13. news.prodigy.co m>,
"The Bicycling Guitarist" <Ch***@TheBicyc lingGuitarist.n et> writes:

The following is what my host's tech guy just sent me. Now he says it's the
fault of specifying the charset in the server header. I thought that's what
we're supposed to do. Is he doing it wrong?

Yes.

Well, I don't know IIS - it may be impossible to do it right.
But what Alan reported shows that it doesn't support HTTP, at least
as currently configured. If your host can't fix that, then they're
as fit for use on the Internet as a washing machine that only works
on 115 volts DC.

Hmmm, I recently ran an accessibility evaluation on an IIS site -
just checked it. They return "text/html; charset=UTF-8" and an HTML
page, regardless of any Accept header I send. Maybe that would be
an improvement for your host, if they lack the basic commonsense to
upgrade to Apache.

--
Nick Kew

[Rijk van Geijtenbeek]

On Thu, 28 Oct 2004 21:32:47 -0400, Daniel R. Tobias <da*@tobias.nam e>
wrote:

Alan J. Flavell wrote:

My suspicion is that we're going to end up with XHTML-flavoured tag
soup, thus losing any of the benefits which XML was *supposed* to bring
us.

We're seeing some of that already, in tag-soup pages in which
XHTML-style tags like <br /> are sprinkled randomly (interspersed with
their HTML-style equivalents like <br>).

Don't forget the XHTML pages with topmargin=0 added to the BODY tag...
Even systems geared to producing standards based markup like some blogging
tools are not vigorously checking input and comments, so you end up with
invalid pages anyway. Not a big deal, but this means that browsers will
not be able to treat XHTML different from HTML: it is and will stay tag
soup. There is nothing (more) wrong with sending XHTML tag soup instead of
sending HTML tag soup, from the browsers and readers point of view.

--
Rijk van Geijtenbeek

The Web is a procrastination apparatus:
It can absorb as much time as is required to ensure that you
won't get any real work done. - J.Nielsen

[The Bicycling Guitarist]

"Nick Kew" <ni**@hugin.web thing.com> wrote in message
news:7e******** ****@hugin.webt hing.com...

In article <nc************ ****@newssvr13. news.prodigy.co m>,
"The Bicycling Guitarist" <Ch***@TheBicyc lingGuitarist.n et> writes:

But what Alan reported shows that it doesn't support HTTP, at least
as currently configured. If your host can't fix that, then they're
as fit for use on the Internet as a washing machine that only works
on 115 volts DC.

Hmmm, I recently ran an accessibility evaluation on an IIS site - Nick Kew

Hi Nick. I like "fussy" mode. Your online tools have greatly assisted me
many times. I am very grateful for your assistance, and to everyone else who
has contributed to this thread. I may need another host soon.
Chris Watson a.k.a. "The Bicycling Guitarist"

[Nick Kew]

In article <op************ **@news.individ ual.net>,
"Rijk van Geijtenbeek" <ri**@operaremo vethiz.com> writes:

Even systems geared to producing standards based markup like some blogging
tools are not vigorously checking input and comments, so you end up with
invalid pages anyway.

Huh? Either tools generate valid markup, or they don't. You want to
accept markup from users and guarantee it's valid, see for example
http://www.apachetutor.org/apps/annot

--
Nick Kew

[Rijk van Geijtenbeek]

On Fri, 29 Oct 2004 12:08:42 +0100, Nick Kew <ni**@hugin.web thing.com>
wrote:

In article <op************ **@news.individ ual.net>,
"Rijk van Geijtenbeek" <ri**@operaremo vethiz.com> writes:

Even systems geared to producing standards based markup like some
blogging
tools are not vigorously checking input and comments, so you end up with
invalid pages anyway.
Huh? Either tools generate valid markup, or they don't.

Many don't. Even though they use nice standards-based table-less CSS
driven markup, that's what I meant.
You want to
accept markup from users and guarantee it's valid, see for example
http://www.apachetutor.org/apps/annot

--
Rijk van Geijtenbeek

The Web is a procrastination apparatus:
It can absorb as much time as is required to ensure that you
won't get any real work done. - J.Nielsen

[Pierre Goiffon]

"The Bicycling Guitarist" <Ch***@TheBicyc lingGuitarist.n et> a écrit
dans le message de news:nc******** ********@newssv r13.news.prodig y.com

The following is what my host's tech guy just sent me. (...) Chris, (...) When you telnet in and request the page manually with an accept of
"text/html" or "text/html,text/plain: the server tries to return
"text/html;charset=ut f-8" which isn't what the client requested so it
kicks back the 406 error.

Seems like there is a big confusion : the mime type value and the charset
share the same content type http header. For a content type negociation of
course only the mime type is compared. The charset information is, as told
before, very important and should always be returned.

[The Bicycling Guitarist]

"Pierre Goiffon" <pg******@nowhe re.invalid> wrote in message
news:41******** *************** @news.free.fr.. .

"The Bicycling Guitarist" <Ch***@TheBicyc lingGuitarist.n et> a écrit
dans le message de news:nc******** ********@newssv r13.news.prodig y.com

The following is what my host's tech guy just sent me.

(...)

Chris,

(...)

When you telnet in and request the page manually with an accept of
"text/html" or "text/html,text/plain: the server tries to return
"text/html;charset=ut f-8" which isn't what the client requested so it
kicks back the 406 error.

Seems like there is a big confusion : the mime type value and the charset
share the same content type http header. For a content type negociation of
course only the mime type is compared. The charset information is, as told
before, very important and should always be returned.

Should I also forward the above paragraph to my host's tech? Will *this*
finally fix things (assuming he understands it and can implement it on IIS)?
Chris Watson a.k.a. "The Bicycling Guitarist"

[Pierre Goiffon]

"The Bicycling Guitarist" <Ch***@TheBicyc lingGuitarist.n et> a écrit
dans le message de news:33******** ********@newssv r13.news.prodig y.com

Should I also forward the above paragraph to my host's tech?

Hu. Well no problem. You should just rephrase because I think my english is
far for perfect :)
The idea is that the content-type HTTP header contains two distincts MIME
informations : the MIME content-type itself and the charset. The content
negociation must be done with the _MIME_ content-type.

[Spartanicus]

Invalid User <us**@domain.in valid> wrote:

Therefore more XHTML coded sites is published almost every day now.

Every day more Lemmings are jumping off a cliff.
Mostly by people who don't understand the issues involved. Instead, like
you, they've been seduced the the "X" into thinking it's somehow cool or
modern.
Where do you think I copied this from:
<?xml version="1.0"?>
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN"
"http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" lang="en" xml:lang="en">

No, it's not any of my pages. It's from the W3C site! Don't even the
guys at W3C understand?

They do, it's a mistake to see their usage of xhtml served as text/html
as a recommendation. It's an aspiration that has proven to be pointless
for the foreseeable future. Even the things they do recommend should be
evaluated for value, usefulness and practicality. This evaluation is
left to the coding community and the verdict on xhtml is: don't. Look
through the archives of this group; every time it is discussed the
mythical benefits of xhtml have been dispelled by the people who've been
there and done that.
I know a lot more sites by people who know a lot
more about this than I do, and who use XHTML served as text/html

Lemmings, do you want to be one?

--
Spartanicus