UTF-8 to HTML conversion

[Martin Trautmann]

Hi all,

is there any kind of 'hiconv' or other (unix-like) conversion tool that
would convert UTF-8 to HTML (ISO-Latin-1 and Unicode)?

The database output is UTF-8 or UTF-16 only - Thus almost every character
starts with ^@.

I've seen e.g.
http://aktuell.de.selfhtml.org/artik...64/utf8.htm#a5 as
JavaScript decoder - but maybe there's a recommended little helper that
could do:

- get rid of UTF-8 declarations where Latin is good enough
- convert others to most widely used HTML

such as Bulgarian and Russian charsets to &#10..;
.... or even ISO-Latin-1 8bits to HTML
ä -> ä
.... and maybe
EUR -> €

Thanks,
Martin

[Alan J. Flavell]

On Wed, 2 Mar 2005, Shmuel (Seymour J.) Metz wrote:

In <11************ **********@f14g 2000cwb.googleg roups.com>, on
03/01/2005
at 11:07 AM, ph*******@treef ic.com said:

Anyway, if you really want to convert Unicode to latin1 + html
character entities, I believe that GNU recode can do what you want:

What about bidi text?

What about it? There's no reason that a mere change of character
encoding should have any effect on bidi properties. Arabic and logical
Hebrew should simply *work* (at least as well as they ever work in
your choice of browser).

See also http://www.nirdagan.com/hebrew/compare

So where do you foresee a problem?

[Lachlan Hunt]

Martin Trautmann wrote:

On Wed, 02 Mar 2005 23:21:35 +1100, Lachlan Hunt wrote:

Stripping away one byte from a UTF-16 character does not necessarily
produce the equivalent UTF-8 character. That would only apply to the
US-ASCII subset of Unicode, for which the high-order octet is set to 0
and the low-order octet matches the US-ASCII character, but does not
apply to any other character.

True - but that's exactly the range where I wanted ASCII/Latin instead
of UTF-16.

It won't work for the latin range, only the US-ASCII range I said.

US-ASCII: Code positions 0 to 127 (decimal)
In UTF-8, these are encoded as single octets which are identical to US-ASCII
Latin: Code positions 128 to 255
In UTF-8, these (and everything above) are encoded using multiple
octets. I think all of this range is encoded as 2 octets, but they *do
not* match the UTF-16 encoding.

eg. Encoding the copyright symbol: © (decimal 169)
UTF-8: 0xC2 0xA9
UTF-16: 0x00 0xA9
ISO-8859-1: 0xA9 (this is a single-octet encoding)

That's why stripping the other byte will simply not work to covert
UTF-16 to UTF-8. Stripping the first byte also won't work for any
character above 255 (outside the ISO-8859-1 range).

--
Lachlan Hunt
http://lachy.id.au/
http://GetFirefox.com/ Rediscover the Web
http://GetThunderbird.com/ Reclaim your Inbox

[Martin Trautmann]

On Thu, 03 Mar 2005 08:50:11 +1100, Lachlan Hunt wrote:

It won't work for the latin range, only the US-ASCII range I said.

Thanks for your further explanation. I actually did not have any
critical problems within this range yet. But I felt it was the wrong
move, apart from being incomplete.

Martin