character to HTML ampersand escape sequence converter

[SwordAngel]

Hello,
I'm looking for a program that converts characters of different
encodings (such as EUC-JP, Big5, GB-18030, etc.) into HTML ampersand
escape sequences. Anybody knows where I can find one?

thx.

[David Dorward]

SwordAngel wrote:

I'm looking for a program that converts characters of different
encodings (such as EUC-JP, Big5, GB-18030, etc.) into HTML ampersand
escape sequences. Anybody knows where I can find one?

IIRC Tidy will do that.

http://tidy.sf.net/

--
David Dorward <http://blog.dorward.me .uk/> <http://dorward.me.uk/>
Home is where the ~/.bashrc is

[Bjoern Hoehrmann]

* David Dorward wrote in comp.infosystem s.www.authoring.html:

SwordAngel wrote:

I'm looking for a program that converts characters of different
encodings (such as EUC-JP, Big5, GB-18030, etc.) into HTML ampersand
escape sequences. Anybody knows where I can find one?

IIRC Tidy will do that.

Well, yes, but only for character encodings it supports (and it does not
support any of the encodings SwordAngel listed to that extend). Windows
users can compile Tidy with an experimental feature that enables support
for all character encodings Windows / Internet Explorer support via the
TIDY_WIN32_MLAN G_SUPPORT #define, but it is generally better to use ex-
ternal tools such as iconv, piconv, uconv, recode, ... to convert the
document to UTF-8 and let Tidy process the document accordingly.
--
Björn Höhrmann · mailto:bj****@h oehrmann.de · http://bjoern.hoehrmann.de
Weinh. Str. 22 · Telefon: [phone] · http://www.bjoernsworld.de
68309 Mannheim · PGP Pub. KeyID: 0xA4357E78 · http://www.websitedev.de/

[Alan J. Flavell]

On Fri, 17 Dec 2004, SwordAngel wrote:

I'm looking for a program that converts characters of different
encodings (such as EUC-JP, Big5, GB-18030, etc.) into HTML ampersand
escape sequences. Anybody knows where I can find one?

"free recode" ? http://recode.progiciels-bpi.ca/

Call it with something like:
recode -d euc-jp..h4 < input.html > output.html

That won't do anything to tidy up the HTML, though, unlike Tidy ;-)

And don't forget that when you've translated language-specific
encodings into Han-unified Unicode characters, you should mark-up
the source with the correct language attribute in order to get
the right rendering of the unified characters. At least that's my
understanding (I can't actually read them myself).

[Nick Kew]

In article <41************ ****@news.bjoer n.hoehrmann.de> ,
Bjoern Hoehrmann <de*******@gmx. net> writes:

IIRC Tidy will do that.

Indeed. I was on the point of suggesting AN XML processor until I saw
that (libxml2 accepts HTML as well as XML input).
Well, yes, but only for character encodings it supports (and it does not
support any of the encodings SwordAngel listed to that extend).
Indeed, libxml2 (last time I checked) supports some but not all of
those encodings, so the same limitation applies.

Have you considered tying in iconv to Tidy to improve i18n support?
but it is generally better to use ex-
ternal tools such as iconv, piconv, uconv, recode, ... to convert the
document to UTF-8 and let Tidy process the document accordingly.

I believe OpenSP supports all the encodings named, though I'm
not entirely sure OTTOMH. So there may still be a one-stop
program for the conversion. But as Björn says, a transcoder
such as iconv is a more general solution.
--
Nick Kew

Nick's manifesto: http://www.htmlhelp.com/~nick/

[Bjoern Hoehrmann]

* Nick Kew wrote in comp.infosystem s.www.authoring.html:

Have you considered tying in iconv to Tidy to improve i18n support?

I wrote an experimental iconv wrapper which is included in the source
distribution, but it is not plugged into the code, i.e., you need to
change a few things in order to use it. Development of these features
was put on hold until a better interface for pluggable transcoders for
Tidy has been developed (which has not happend yet).
--
Björn Höhrmann · mailto:bj****@h oehrmann.de · http://bjoern.hoehrmann.de
Weinh. Str. 22 · Telefon: [phone] · http://www.bjoernsworld.de
68309 Mannheim · PGP Pub. KeyID: 0xA4357E78 · http://www.websitedev.de/

[Henri Sivonen]

In article <fg************ @hugin.webthing .com>,
ni**@hugin.webt hing.com (Nick Kew) wrote:

Indeed. I was on the point of suggesting AN XML processor until I saw
that (libxml2 accepts HTML as well as XML input).

A quick glance at the API docs suggested that the HTML API is similar
but separate from the XML API. Is it so? Is there an equivalent of SAX
filter or somesuch that would make HTML appear to the app as XHTML?

TagSoup on the Java side appears to the app as an XML parser parsing
XHTML.

Has anyone compared the tag slurping features of TagSoup and libxml2? I
Wonder which one is a better idea when writing in Python: using libxml2
with CPython or using TagSoup with Jython?

--
Henri Sivonen
hs******@iki.fi
http://iki.fi/hsivonen/
Mozilla Web Author FAQ: http://mozilla.org/docs/web-developer/faq.html

[Alan J. Flavell]

On Sat, 18 Dec 2004, Henri Sivonen wrote:

In article <fg************ @hugin.webthing .com>,
ni**@hugin.webt hing.com (Nick Kew) wrote:

Indeed. I was on the point of suggesting AN XML processor until I
saw that (libxml2 accepts HTML as well as XML input).

A quick glance at the API docs suggested that the HTML API is similar
but separate from the XML API. Is it so?

But does this matter, in the context of the original question?

Surely, given any WWW-compatible HTML or XHTML data stream, one can
choose to convert any non-ascii coded character (or any selection of
non-ascii characters) to a unicode code point and thence into
&#bignumber; notation, purely at the character stream layer, without
parsing the rest of the material at all?

[Bjoern Hoehrmann]

* Alan J. Flavell wrote in comp.infosystem s.www.authoring.html:

Surely, given any WWW-compatible HTML or XHTML data stream, one can
choose to convert any non-ascii coded character (or any selection of
non-ascii characters) to a unicode code point and thence into
&#bignumber; notation, purely at the character stream layer, without
parsing the rest of the material at all?

That does not work very well for comments, CDATA elements, processing
instructions, etc.
--
Björn Höhrmann · mailto:bj****@h oehrmann.de · http://bjoern.hoehrmann.de
Weinh. Str. 22 · Telefon: [phone] · http://www.bjoernsworld.de
68309 Mannheim · PGP Pub. KeyID: 0xA4357E78 · http://www.websitedev.de/

[Alan J. Flavell]

On Sat, 18 Dec 2004, Bjoern Hoehrmann wrote:

* Alan J. Flavell wrote in comp.infosystem s.www.authoring.html:

Surely, given any WWW-compatible HTML or XHTML data stream, one can
choose to convert any non-ascii coded character (or any selection of
non-ascii characters) to a unicode code point and thence into
&#bignumber; notation, purely at the character stream layer, without
parsing the rest of the material at all?
That does not work very well for comments,

Fortunately, HTML rendering agents don't need to interpret the content
of comments...
CDATA elements, processing instructions, etc.

Theoretically, of course, you are right; which is why I slipped-in
that qualification re. documents that are compatible with the WWW as
it exists.

I don't dispute that in theory you can produce counter-examples where
the simple method described above gives the wrong result, for the
reasons you gave; but I'm interested if a real-life example can be
produced where this would matter.

all the best