character to HTML ampersand escape sequence converter

[SwordAngel]

Hello,
I'm looking for a program that converts characters of different
encodings (such as EUC-JP, Big5, GB-18030, etc.) into HTML ampersand
escape sequences. Anybody knows where I can find one?

thx.

[Henri Sivonen]

In article <Pi************ *************** ****@ppepc56.ph .gla.ac.uk>,
"Alan J. Flavell" <fl*****@ph.gla .ac.uk> wrote:

On Sat, 18 Dec 2004, Henri Sivonen wrote:

In article <fg************ @hugin.webthing .com>,
ni**@hugin.webt hing.com (Nick Kew) wrote:

Indeed. I was on the point of suggesting AN XML processor until I
saw that (libxml2 accepts HTML as well as XML input).
A quick glance at the API docs suggested that the HTML API is similar
but separate from the XML API. Is it so?

But does this matter, in the context of the original question?

Perhaps not. It was a new question in the spirit of "discussion
forum--not help desk". :-)
Surely, given any WWW-compatible HTML or XHTML data stream, one can
choose to convert any non-ascii coded character (or any selection of
non-ascii characters) to a unicode code point and thence into
&#bignumber; notation, purely at the character stream layer, without
parsing the rest of the material at all?

Yes, except comments change if they exist and contain non-ASCII.

--
Henri Sivonen
hs******@iki.fi
http://iki.fi/hsivonen/
Mozilla Web Author FAQ: http://mozilla.org/docs/web-developer/faq.html

[Bjoern Hoehrmann]

* Alan J. Flavell wrote in comp.infosystem s.www.authoring.html:

I don't dispute that in theory you can produce counter-examples where
the simple method described above gives the wrong result, for the
reasons you gave; but I'm interested if a real-life example can be
produced where this would matter.

Consider a HTML document with

<style type="text/css">
q:lang(no) { quotes: "«" "»" '"' '"' }
</style>

or consider HTML documents with scripts such as those in

http://www.rfs.jp/sitebuilder/javascript/01/08.html
--
Björn Höhrmann · mailto:bj****@h oehrmann.de · http://bjoern.hoehrmann.de
Weinh. Str. 22 · Telefon: [phone] · http://www.bjoernsworld.de
68309 Mannheim · PGP Pub. KeyID: 0xA4357E78 · http://www.websitedev.de/

[Alan J. Flavell]

On Sat, 18 Dec 2004, Bjoern Hoehrmann wrote:

Consider a HTML document with

<style type="text/css">
q:lang(no) { quotes: "«" "»" '"' '"' }
</style>

or consider HTML documents with scripts such as those in

http://www.rfs.jp/sitebuilder/javascript/01/08.html

OK, I concede.

Of course, if the target encoding was meant to be us-ascii with
&#bignumber; representations of non-ascii characters (which might have
been what the questioner had in mind, since I undestood the request to
be for &#bignumber; representation rather than actual utf-8-encoded
characters in the HTML part), then you'd need CSS-aware and
Javascript-aware converters to know how to represent those non-ascii
characters in their respective languages.

Indeed the W3C were wise in their XHTML documentation to recommend
moving those enclosures out into separate files rather than trying to
in-line them as CDATA ;-)

[Nick Kew]

In article <hs************ *************** *@news.dnainter net.net>,
Henri Sivonen <hs******@iki.f i> writes:

Indeed. I was on the point of suggesting AN XML processor until I saw
that (libxml2 accepts HTML as well as XML input).
A quick glance at the API docs suggested that the HTML API is similar
but separate from the XML API. Is it so?

Yes, that's a reasonably fair summary. The HTML parser is the XML
parser with tolerance of non-XML and knowledge of HTML4.
Is there an equivalent of SAX
filter or somesuch that would make HTML appear to the app as XHTML?
The HTML parser gives you either SAX or DOM, and will process either
HTML or XHTML input without distinction. HTML mode is also tolerant
of tag-soup, though not quite as forgiving as a typical browser.
There are a few bugs wrt the spec: most obviously, it only recognises
XML comment syntax (but then, so do the browsers).

As a corollary, you can use it to apply XML processing to HTML.
TagSoup on the Java side appears to the app as an XML parser parsing
XHTML.
I'm not familiar with that, but it's not uncommon.
Has anyone compared the tag slurping features of TagSoup and libxml2? I
Wonder which one is a better idea when writing in Python: using libxml2
with CPython or using TagSoup with Jython?

Couldn't tell you. But I'd venture a strong guess that libxml2 will be
not only a great deal faster than anything-java, but also no harder
and possibly easier to work with.
--
Nick Kew

[Henri Sivonen]

In article <fu************ @hugin.webthing .com>,
ni**@hugin.webt hing.com (Nick Kew) wrote:

In article <hs************ *************** *@news.dnainter net.net>,
Henri Sivonen <hs******@iki.f i> writes:

Indeed. I was on the point of suggesting AN XML processor until I saw
that (libxml2 accepts HTML as well as XML input).

The HTML parser gives you either SAX or DOM, and will process either
HTML or XHTML input without distinction.
Are the elements in the XHTML namespace or in no namespace? The good
thing about TagSoup is that it allows the app internals to be written
for XHTML, so the same app internals work for HTML, XHTML *and*
XHTML+FooML (using an XML parser). That is, the HTML/XHTML difference is
left on the parsing level and not carried over to higher levels as in
browsers.
But I'd venture a strong guess that libxml2 will be
not only a great deal faster than anything-java, but also no harder
and possibly easier to work with.

I think I read somewhere that the libxml2 wrapper gives the Python side
UTF-8 byte strings instead of Python Unicode strings.

--
Henri Sivonen
hs******@iki.fi
http://iki.fi/hsivonen/
Mozilla Web Author FAQ: http://mozilla.org/docs/web-developer/faq.html

[Nick Kew]

In article <hs************ *************** *@news.dnainter net.net>,
Henri Sivonen <hs******@iki.f i> writes:

In article <hs************ *************** *@news.dnainter net.net>,
Henri Sivonen <hs******@iki.f i> writes:

>> Indeed. I was on the point of suggesting AN XML processor until I saw
>> that (libxml2 accepts HTML as well as XML input).

The HTML parser gives you either SAX or DOM, and will process either
HTML or XHTML input without distinction.

Are the elements in the XHTML namespace or in no namespace?

They're not namespaced. At least not in the SAX parse mode, which is
where I've investigated the issue. At least, my preliminary experiments
trying to use the HTML parser in SAX2 mode were not successful, which
is not to say I won't return to the issue.
The good
thing about TagSoup is that it allows the app internals to be written
for XHTML, so the same app internals work for HTML, XHTML *and*
XHTML+FooML (using an XML parser). That is, the HTML/XHTML difference is
left on the parsing level and not carried over to higher levels as in
browsers.

Watch this space. That's what I'd like mod_publisher to do. OTOH,
how many people mix HTML (no X) with other namespaces in real life?
The full capability is at best a pathological edge-case.

BTW, if you're interested in namespace processing on the Web,
may I refer you to my recently-published article at
http://www.xml.com/pub/a/2004/12/15/...amespaces.html

--
Nick Kew

[Henri Sivonen]

In article <cq***********@ hugin.webthing. com>,
ni**@hugin.webt hing.com (Nick Kew) wrote:

In article <hs************ *************** *@news.dnainter net.net>,
Henri Sivonen <hs******@iki.f i> writes:

In article <hs************ *************** *@news.dnainter net.net>,
Henri Sivonen <hs******@iki.f i> writes:

>> Indeed. I was on the point of suggesting AN XML processor until I saw
>> that (libxml2 accepts HTML as well as XML input).

The HTML parser gives you either SAX or DOM, and will process either
HTML or XHTML input without distinction.

Are the elements in the XHTML namespace or in no namespace?

They're not namespaced.

That's a pity. Of course, it's possible to write a filter that takes
SAX1 events, adds the namespacing and emits SAX2 events, but it is
uncool to have to implement stuff that a library should be able to do
out of the box.

The good
thing about TagSoup is that it allows the app internals to be written
for XHTML, so the same app internals work for HTML, XHTML *and*
XHTML+FooML (using an XML parser). That is, the HTML/XHTML difference is
left on the parsing level and not carried over to higher levels as in
browsers.

Watch this space. That's what I'd like mod_publisher to do. OTOH,
how many people mix HTML (no X) with other namespaces in real life?

The people who export from MS Office?

I was not suggesting that namespaces in HTML should be supported. How
that would work isn't even defined.

However, I think it doesn't make sense to write the app internals for
namespaceless HTML so that massive rework is needed for XHTML+FooML. It
makes more sense to write the app internals for namespaced compound
documents and to convert HTML to XHTML at parse time. Using an XML
parser is the right way to go for XHTML and XHTML+FooML.
BTW, if you're interested in namespace processing on the Web,
may I refer you to my recently-published article at
http://www.xml.com/pub/a/2004/12/15/...amespaces.html

Interesting.

BTW, how do you reconcile the GPL and the Apache license?

--
Henri Sivonen
hs******@iki.fi
http://iki.fi/hsivonen/
Mozilla Web Author FAQ: http://mozilla.org/docs/web-developer/faq.html

[Nick Kew]

In article <hs************ *************** *@news.dnainter net.net>,
Henri Sivonen <hs******@iki.f i> writes:

Watch this space. That's what I'd like mod_publisher to do. OTOH,
how many people mix HTML (no X) with other namespaces in real life?
The people who export from MS Office?

Good catch. I'd forgotten that one. Don't they try/claim to be XHTML?
I was not suggesting that namespaces in HTML should be supported. How
that would work isn't even defined.

It would presumably work by treating it as XHTML. Like XPath, XSLT,
etc, which do work fine with HTML and the libxml2 parser.

BTW, if you're interested in namespace processing on the Web,
may I refer you to my recently-published article at
http://www.xml.com/pub/a/2004/12/15/...amespaces.html

Interesting.

BTW, how do you reconcile the GPL and the Apache license?

Why is that a problem? My work is GPL (if you want it free - dual
licensing available otherwise). Apache is ASF license. They are
distributed separately. Those Linux distros (and FreeBSD) that
package my GPL modules offer them to users as separate packages,
and don't have a problem with it. Even the fundamentalists at
Debian don't have a problem with it. Any more than they have a
problem distributing non-GPL apps like Apache to run on Linux itself.

--
Nick Kew

[Henri Sivonen]

In article <l8************ @hugin.webthing .com>,
ni**@hugin.webt hing.com (Nick Kew) wrote:

In article <hs************ *************** *@news.dnainter net.net>,
Henri Sivonen <hs******@iki.f i> writes:

Watch this space. That's what I'd like mod_publisher to do. OTOH,
how many people mix HTML (no X) with other namespaces in real life?
The people who export from MS Office?

Good catch. I'd forgotten that one. Don't they try/claim to be XHTML?

I don't think so. It's more like HTML tag soup spiced up with colonified
names and XML "data islands".

I was not suggesting that namespaces in HTML should be supported. How
that would work isn't even defined.

It would presumably work by treating it as XHTML.

With namespaces in HTML I meant this kind of Microsoftism:

<HTML xmlns:k='urn:ke wl-schema-urn'>
<HEAD>
<TITLE>Test</TITLE>
<xml>
<k:foo>
<k:bar/>
</k:foo>
</xml>
</HEAD>
<BODY>
....
</BODY>
</HTML>

(I suppose Microsoft has defined how that is supposed to work. So saying
it isn't defined was not entirely accurate.)
Why is that a problem?
The FSF lists the Apache licenses 1.0, 1.1 and 2.0 as GPL-incompatible
free software licenses.

http://www.fsf.org/licenses/license-...atibleLicenses
Even the fundamentalists at Debian don't have a problem with it.
That's surprising. :-)
Any more than they have a
problem distributing non-GPL apps like Apache to run on Linux itself.

IIRC, Linus Torvalds declared an exception when the subject came up.

--
Henri Sivonen
hs******@iki.fi
http://iki.fi/hsivonen/
Mozilla Web Author FAQ: http://mozilla.org/docs/web-developer/faq.html