Autodetect UTF-8 vs ISO-8859-1

[Lars]

As part of an error correction mechanism, I'd like to
autodetect ISO-8859-1 vs UTF-8 usage. Where is this
described concisely?

-Lars

[Dean Tiegs]

Lars <la**@nospam.ac.no> writes:

As part of an error correction mechanism, I'd like to autodetect
ISO-8859-1 vs UTF-8 usage. Where is this described concisely?

For an arbitrary text file, it is impossible to distinguish
automatically between the two and be 100 percent sure of choosing
correctly. However, if the file contains no invalid UTF-8 sequences,
it is almost certainly UTF-8. It would be a very unusual ISO-8859-1
file that did not have invalid UTF-8 sequences.

For XML files, it's much simpler: if it is ISO-8859-1, it has to be
declared in the XML declaration.

--
Dean Tiegs, NE¼-20-52-25-W4
“Confortare et esto robustus�
http://telusplanet.net/public/dctiegs/

[Johannes Koch]

Dean Tiegs wrote:

For XML files, it's much simpler: if it is ISO-8859-1, it has to be
declared in the XML declaration.

Or some lower-level protocol, like HTTP Content-Type header.
--
Johannes Koch
In te domine speravi; non confundar in aeternum.
(Te Deum, 4th cent.)