UTF-8 question.

[Richard Connamacher]

I'm new to PostgreSQL, and from the looks of it, it's a great database,
and I'll be using more of it in the future.

I had a quick question if anyone could clear this up. The documentation
for PostgreSQL (version 7.1, the version this server is using) says that
it supports multibyte character encodings like Unicode (which implies
UTF-16 encoding). Later on, the same page says that Unicode is
represented using UTF-8 encoding. UTF-8 is the 8-bit version of Unicode.
The multibyte version of Unicode is UTF-16.

So, which is it? If I create a database using Unicode as the encoding,
will the encoding be UTF-8 (singlebyte) or UTF-16 (multibyte)?

Thanks!
Rich

---------------------------(end of broadcast)---------------------------
TIP 9: the planner will ignore your desire to choose an index scan if your
joining column's datatypes do not match

[Dan Sugalski]

At 8:39 PM -0400 9/16/04, Richard Connamacher wrote:

I'm new to PostgreSQL, and from the looks of it, it's a great database,
and I'll be using more of it in the future.

I had a quick question if anyone could clear this up. The documentation
for PostgreSQL (version 7.1, the version this server is using) says that
it supports multibyte character encodings like Unicode (which implies
UTF-16 encoding).
Don't confuse Unicode, the 'character set' and rules for characters,
represented by a sequence of abstract 32 bit integers, with
UTF-[8|16|32] which is a way to encode those abstract integers into a
stream of bytes someplace.
Later on, the same page says that Unicode is
represented using UTF-8 encoding. UTF-8 is the 8-bit version of Unicode.
The multibyte version of Unicode is UTF-16.

So, which is it? If I create a database using Unicode as the encoding,
will the encoding be UTF-8 (singlebyte) or UTF-16 (multibyte)?

Erm... UTF-8 *is* a multibyte encoding. Up to 6 bytes per code point,
if things get really degenerate. (And, last I checked, means you can
have up to 70 bytes for really degenerate characters, but my memory
might be off (could be 80))

UTF-8, UTF-16, and UTF-32 will all encode Unicode characters just fine.
--
Dan

--------------------------------------it's like this-------------------
Dan Sugalski even samurai
da*@sidhe.org have teddy bears and even
teddy bears get drunk

---------------------------(end of broadcast)---------------------------
TIP 7: don't forget to increase your free space map settings

[Michael Glaesemann]

On Sep 17, 2004, at 9:39 AM, Richard Connamacher wrote:

UTF-8 is the 8-bit version of Unicode.
The multibyte version of Unicode is UTF-16.

UTF-8 encodes characters with varying numbers of bytes, not just 1 byte
per character. IIRC, it's anywhere from 1 to 5 bytes, actually.
PostgreSQL uses UTF-8.

If you can, upgrade. 7.1 is nearing prehistoric. :)

Michael Glaesemann
grzm myrealbox com
---------------------------(end of broadcast)---------------------------
TIP 4: Don't 'kill -9' the postmaster

[Pierre-Frédéric Caillaud]

=> show client_encoding ;
client_encoding
-----------------
UNICODE
(1 ligne)
=> select char_length('a' ), bit_length('a') ;
char_length | bit_length
-------------+------------
1 | 8
(1 ligne)
# that's an accented "e"
=> select char_length('é' ), bit_length('é') ; ;
char_length | bit_length
-------------+------------
1 | 16 <= two bytes
(1 ligne)
pg does not simply store utf-8 data, it also understands it if you set
your encoding correctly (ie. initdb to UNICODE and client_encoding too so
that data doesn't get mangled on the way to the db). It will refuse to eat
illegal UTF8 characters too.
Once you try unicode, all the codepage mess starts to look old...

On Thu, 16 Sep 2004 20:39:48 -0400, Richard Connamacher
<ri*****@indiei mage.com> wrote:

I'm new to PostgreSQL, and from the looks of it, it's a great database,
and I'll be using more of it in the future.

I had a quick question if anyone could clear this up. The documentation
for PostgreSQL (version 7.1, the version this server is using) says that
it supports multibyte character encodings like Unicode (which implies
UTF-16 encoding). Later on, the same page says that Unicode is
represented using UTF-8 encoding. UTF-8 is the 8-bit version of Unicode.
The multibyte version of Unicode is UTF-16.

So, which is it? If I create a database using Unicode as the encoding,
will the encoding be UTF-8 (singlebyte) or UTF-16 (multibyte)?

Thanks!
Rich

---------------------------(end of broadcast)---------------------------
TIP 9: the planner will ignore your desire to choose an index scan if
your
joining column's datatypes do not match

---------------------------(end of broadcast)---------------------------
TIP 7: don't forget to increase your free space map settings