I have a variable that contains a string which may or may not include
unicode characters, I understand that I declare a unicode string like this
u'whatever' , the question is that when that string is read in from
an external source (in this case an id3 tag) how do I make the
interpretter understand that it may contain unicode?

cheers

John Morey wrote:
[color=blue]
> I have a variable that contains a string which may or may not include
> unicode characters, I understand that I declare a unicode string like this
> u'whatever' , the question is that when that string is read in from
> an external source (in this case an id3 tag) how do I make the
> interpretter understand that it may contain unicode?[/color]

Read it as a string, and then decode it with the .decode method. You
specify what encoding it's in.

--
Erik Max Francis && max@alcyone.com && http://www.alcyone.com/max/
San Jose, CA, USA && 37 20 N 121 53 W && AIM erikmaxfrancis
You are in the music / In the man's car next to me
-- Sade

Erik Max Francis wrote:[color=blue]
> Read it as a string, and then decode it with the .decode method. You
> specify what encoding it's in.[/color]

Most probably, the OP is asking what to do with an UTF-8 encoded string.

To decode that, just use:

s.decode("utf-8")

to get a unicode string object.

Example:
[color=blue][color=green][color=darkred]
>>> s[/color][/color][/color]
'\xc3\xa4\xc3\xb6\xc3\xbc'[color=blue][color=green][color=darkred]
>>> print s[/color][/color][/color]
äöü[color=blue][color=green][color=darkred]
>>> print s.decode("utf-8")[/color][/color][/color]
äöü[color=blue][color=green][color=darkred]
>>>[/color][/color][/color]

HTH!

--- Heiko.

Heiko Wundram wrote:
[color=blue]
> Most probably, the OP is asking what to do with an UTF-8 encoded string.
>
> To decode that, just use:
>
> s.decode("utf-8")
>
> to get a unicode string object.[/color]

Flags in the ID3 tag specify the encoding. It is not always UTF-8.

http://www.id3.org/id3v2.4.0-structure.txt

--
Erik Max Francis && max@alcyone.com && http://www.alcyone.com/max/
San Jose, CA, USA && 37 20 N 121 53 W && AIM erikmaxfrancis
You are in the music / In the man's car next to me
-- Sade

Erik Max Francis wrote:[color=blue]
> Flags in the ID3 tag specify the encoding. It is not always UTF-8.
>
> http://www.id3.org/id3v2.4.0-structure.txt[/color]

Okay... Didn't know that, and I'm pretty sure with ID3V1 this was always
utf-8 in case it was unicode (which was a non-standard extension anyway).
Thanks for the info.

--- Heiko.

>> Read it as a string, and then decode it with the .decode method. You[color=blue][color=green]
>> specify what encoding it's in.[/color]
>
> Most probably, the OP is asking what to do with an UTF-8 encoded string.
>
> To decode that, just use:
>
> s.decode("utf-8")[/color]

I prefer:

unicode(s, 'utf-8')

That way it's more clear that what you get is a unicode object.