reload(sys)

[=?iso-8859-9?q?S=F6nmez_Kartal?=]

Hello,

I've had an encoding issue and solved it by
"sys.setdefaultencoding('utf-8')"...

My first try wasn't successful since setdefaultencoding is not named
when I imported sys module. After, I import sys module, I needed to
write "reload(sys)" also.

I wonder why we need to call "reload(sys)" to get setdefaultencoding
named?

Happy coding

[Steven Bethard]

Sönmez Kartal wrote:

I've had an encoding issue and solved it by
"sys.setdefaultencoding('utf-8')"...

My first try wasn't successful since setdefaultencoding is not named
when I imported sys module. After, I import sys module, I needed to
write "reload(sys)" also.

I wonder why we need to call "reload(sys)" to get setdefaultencoding
named?

sys.setdefaultencoding is purposely deleted from the sys module after
it's loaded because you really shouldn't be using it. The reload() call
restores the deleted attribute.

If you'd like a less brittle solution to your encoding issue, explain
what the issue was, and people here can probably help you find a better
solution.

STeVe

[=?iso-8859-1?q?S=F6nmez_Kartal?=]

On 31 A ustos, 04:24, Steven Bethard <[e-mail]:

Sönmez Kartal wrote:

I've had an encoding issue and solved it by
"sys.setdefaultencoding('utf-8')"...

My first try wasn't successful since setdefaultencoding is not named
when I imported sys module. After, I import sys module, I needed to
write "reload(sys)" also.

I wonder why we need to call "reload(sys)" to get setdefaultencoding
named?

sys.setdefaultencoding is purposely deleted from the sys module after
it's loaded because you really shouldn't be using it. The reload() call
restores the deleted attribute.

If you'd like a less brittle solution to your encoding issue, explain
what the issue was, and people here can probably help you find a better
solution.

STeVe

I was using the XMLBuilder(xmlbuilder.py). I'm writing XML files as
"f.write(str(xml))". At execution of that line, it gives error with
description, configure your default encoding... My operating system's
default is utf-8, and Emacs' is utf-8 too. Default of XMLBuilder is
utf-8 too. There were some characters interpreter may couldn't print
in ascii. I have tried to replace those characters like (TM) (R)... I
cannot remember them right now, but if necessary I can find them
easily...

This is the part of xmlbuilder.py which raises the error.

try:
if self.pretty:
# tabs are evil, so we will use two spaces
outstr = self._dom.toprettyxml("
",encoding=self.encoding)
else:
outstr = self._dom.toxml(encoding=self.encoding)
except UnicodeDecodeError:
sys.stderr.write('Decoding Error: You must configure
default encoding\n')
sys.exit()

What I can do instead of "import sys; reload(sys);
sys.setdefaultencoding('utf-8')"?

Happy coding

[Marc 'BlackJack' Rintsch]

On Fri, 31 Aug 2007 12:53:36 +0000, Sönmez Kartal wrote:

On 31 A ustos, 04:24, Steven Bethard <[e-mail]:

>Snmez Kartal wrote:

I've had an encoding issue and solved it by
"sys.setdefaultencoding('utf-8')"...

My first try wasn't successful since setdefaultencoding is not named
when I imported sys module. After, I import sys module, I needed to
write "reload(sys)" also.

I wonder why we need to call "reload(sys)" to get setdefaultencoding
named?

sys.setdefaultencoding is purposely deleted from the sys module after
it's loaded because you really shouldn't be using it. The reload() call
restores the deleted attribute.

If you'd like a less brittle solution to your encoding issue, explain
what the issue was, and people here can probably help you find a better
solution.

I was using the XMLBuilder(xmlbuilder.py). I'm writing XML files as
"f.write(str(xml))". At execution of that line, it gives error with
description, configure your default encoding...

This doesn't help us that much. What is `f` here and what is `xml`?

This is the part of xmlbuilder.py which raises the error.

try:
if self.pretty:
# tabs are evil, so we will use two spaces
outstr = self._dom.toprettyxml("
",encoding=self.encoding)
else:
outstr = self._dom.toxml(encoding=self.encoding)
except UnicodeDecodeError:
sys.stderr.write('Decoding Error: You must configure
default encoding\n')
sys.exit()

So there is an attribute `self.encoding` on that object. Is it set? What
encoding is it? And do you put byte strings with values outside ASCII
into your XML or unicode strings?

Ciao,
Marc 'BlackJack' Rintsch

[Steven Bethard]

Sönmez Kartal wrote:

On 31 A ustos, 04:24, Steven Bethard <[e-mail]:

>Sönmez Kartal wrote:

>>I've had an encoding issue and solved it by
"sys.setdefaultencoding('utf-8')"...
My first try wasn't successful since setdefaultencoding is not named
when I imported sys module. After, I import sys module, I needed to
write "reload(sys)" also.
I wonder why we need to call "reload(sys)" to get setdefaultencoding
named?

sys.setdefaultencoding is purposely deleted from the sys module after
it's loaded because you really shouldn't be using it. The reload() call
restores the deleted attribute.

If you'd like a less brittle solution to your encoding issue, explain
what the issue was, and people here can probably help you find a better
solution.

STeVe

I was using the XMLBuilder(xmlbuilder.py). I'm writing XML files as
"f.write(str(xml))". At execution of that line, it gives error with
description, configure your default encoding...

Could you post the actual traceback you're getting?

STeVe

[=?iso-8859-1?q?S=F6nmez_Kartal?=]

On 31 A ustos, 16:58, Marc 'BlackJack' Rintsch <[e-mail]:

On Fri, 31 Aug 2007 12:53:36 +0000, Sönmez Kartal wrote:

On 31 A ustos, 04:24, Steven Bethard <[e-mail]:

Snmez Kartal wrote:
I've had an encoding issue and solved it by
"sys.setdefaultencoding('utf-8')"...

My first try wasn't successful since setdefaultencoding is not named
when I imported sys module. After, I import sys module, I needed to
write "reload(sys)" also.

I wonder why we need to call "reload(sys)" to get setdefaultencoding
named?

sys.setdefaultencoding is purposely deleted from the sys module after
it's loaded because you really shouldn't be using it. The reload() call
restores the deleted attribute.

If you'd like a less brittle solution to your encoding issue, explain
what the issue was, and people here can probably help you find a better
solution.

I was using the XMLBuilder(xmlbuilder.py). I'm writing XML files as
"f.write(str(xml))". At execution of that line, it gives error with
description, configure your default encoding...

This doesn't help us that much. What is `f` here and what is `xml`?

This is the part of xmlbuilder.py which raises the error.

try:
if self.pretty:
# tabs are evil, so we will use two spaces
outstr = self._dom.toprettyxml("
",encoding=self.encoding)
else:
outstr = self._dom.toxml(encoding=self.encoding)
except UnicodeDecodeError:
sys.stderr.write('Decoding Error: You must configure
default encoding\n')
sys.exit()

So there is an attribute `self.encoding` on that object. Is it set? What
encoding is it? And do you put byte strings with values outside ASCII
into your XML or unicode strings?

Ciao,
Marc 'BlackJack' Rintsch

I should have said that 'f' is a file object and xml is a XMLBuilder
object. Sorry. :-)

self.encoding is 'utf-8' by default.

I have only ® and ™ characters in the XML file and a space character
which Emacs shows as colored '_'. I have replaced those but didn't
work!

Here is the full code of xmlbuilder.py: http://rafb.net/p/9rURi822.html

I don't wanna bother you but if you see there is something not
practical then I'll keep writing about this. :-)

[=?iso-8859-1?q?S=F6nmez_Kartal?=]

On 31 A ustos, 20:09, Steven Bethard <[e-mail]:

Sönmez Kartal wrote:

On 31 A ustos, 04:24, Steven Bethard <[e-mail]:

Sönmez Kartal wrote:
I've had an encoding issue and solved it by
"sys.setdefaultencoding('utf-8')"...
My first try wasn't successful since setdefaultencoding is not named
when I imported sys module. After, I import sys module, I needed to
write "reload(sys)" also.
I wonder why we need to call "reload(sys)" to get setdefaultencoding
named?
sys.setdefaultencoding is purposely deleted from the sys module after
it's loaded because you really shouldn't be using it. The reload() call
restores the deleted attribute.

If you'd like a less brittle solution to your encoding issue, explain
what the issue was, and people here can probably help you find a better
solution.

STeVe

I was using the XMLBuilder(xmlbuilder.py). I'm writing XML files as
"f.write(str(xml))". At execution of that line, it gives error with
description, configure your default encoding...

Could you post the actual traceback you're getting?

STeVe

Steve,
I get this when it happens: "Decoding Error: You must configure
default encoding" which comes from in the code excerpt in
xmlbuilder.py (http://rafb.net/p/9rURi822.html)

[Steven Bethard]

Sönmez Kartal wrote:

I was using the XMLBuilder(xmlbuilder.py). I'm writing XML files as
"f.write(str(xml))". At execution of that line, it gives error with
description, configure your default encoding...

[and later]

>
I get this when it happens: "Decoding Error: You must configure
default encoding" which comes from in the code excerpt in
xmlbuilder.py (http://rafb.net/p/9rURi822.html)

Can you show the code where you populate the XMLBuilder? I'm guessing
you're doing something like::

import xmlbuilder
builder = xmlbuilder.XMLBuilder()
builder.foo = dict(bar='® and ™')
str(builder)

That breaks because the string '® and ™' is not properly encoded. Have
you declared an encoding in your source file? PEP 263 shows you how:

http://www.python.org/dev/peps/pep-0263/

Note that with Python 2.5 the code above gives a SyntaxError without a
proper encoding. You should also probably be prefixing your string
literals containing weird characters with "u" to make them unicode.
Doing both of these in the code above made it work for me.

STeVe

[=?iso-8859-1?q?S=F6nmez_Kartal?=]

On 3 Eylül, 05:40, Steven Bethard <[e-mail]:

Sönmez Kartal wrote:

I was using the XMLBuilder(xmlbuilder.py). I'm writing XML files as
"f.write(str(xml))". At execution of that line, it gives error with
description, configure your default encoding...

[and later]

I get this when it happens: "Decoding Error: You must configure
default encoding" which comes from in the code excerpt in
xmlbuilder.py (http://rafb.net/p/9rURi822.html)

Can you show the code where you populate the XMLBuilder? I'm guessing
you're doing something like::

import xmlbuilder
builder = xmlbuilder.XMLBuilder()
builder.foo = dict(bar='® and ™')
str(builder)

That breaks because the string '® and ™' is not properly encoded. Have
you declared an encoding in your source file? PEP 263 shows you how:

http://www.python.org/dev/peps/pep-0263/

Note that with Python 2.5 the code above gives a SyntaxError without a
proper encoding. You should also probably be prefixing your string
literals containing weird characters with "u" to make them unicode.
Doing both of these in the code above made it work for me.

STeVe

http://rafb.net/p/RfaF8215.html

products in the code is a list of dictionaries which are returned by
makeProduct function.

I'm not typing or pasting those characters into my script. So,
declaring an encoding didn't make it. :-( But, your code excerpt
runned well.

I have tried "f.write(unicode(xml))" for the last line of the script.
No success.

I think we should think about how we can achieve the effect of
"sys.setdefaultencoding('utf-8')"

[Gabriel Genellina]

On 6 sep, 08:13, Sönmez Kartal <[e-mail]:

On 3 Eylül, 05:40, Steven Bethard <[e-mail]:

Sönmez Kartal wrote:

I was using the XMLBuilder(xmlbuilder.py). I'm writing XML files as
"f.write(str(xml))". At execution of that line, it gives error with
description, configure your default encoding...

Can you show the code where you populate the XMLBuilder? I'm guessing
you're doing something like::
import xmlbuilder
builder = xmlbuilder.XMLBuilder()
builder.foo = dict(bar='® and ™')
str(builder)

That breaks because the string '® and ™' is not properly encoded. Have
you declared an encoding in your source file? PEP 263 shows you how:

http://rafb.net/p/RfaF8215.html

products in the code is a list of dictionaries which are returned by
makeProduct function.

I'm not typing or pasting those characters into my script. So,
declaring an encoding didn't make it. :-( But, your code excerpt
runned well.

You should ensure that arguments to makeProduct are either:
- unicode objects
- ASCII strings

If you got them from some other place, decode the strings as soon as
possible into unicode. Read <http://www.amk.ca/python/howto/unicode>
to understand what's happening (and why the XMLBuilder error message
is *not* a good advice)

--
Gabriel Genellina

[Steven Bethard]

Sönmez Kartal wrote:

I was using the XMLBuilder(xmlbuilder.py). I'm writing XML files as
"f.write(str(xml))". At execution of that line, it gives error with
description, configure your default encoding...

[and later]

>
http://rafb.net/p/RfaF8215.html

products in the code is a list of dictionaries which are returned by
makeProduct function.

I'm not typing or pasting those characters into my script. So,
declaring an encoding didn't make it. :-( But, your code excerpt
runned well.

Gabriel Genellina wrote:

You should ensure that arguments to makeProduct are either:
- unicode objects
- ASCII strings

If you got them from some other place, decode the strings as soon as
possible into unicode. Read <http://www.amk.ca/python/howto/unicode>
to understand what's happening

To further illustrate Gabriel's point, here is some code where I read in
some UTF8 text from a file. If you properly decode that text from UTF8,
you don't get any errors. If you forget to decode that text, you'll get
exactly the "default encoding" error you were getting before:

>>f = open('temp.txt', 'w')
f.write(u'® and ™'.encode('utf8'))
f.close()
non_decoded_text = open('temp.txt').read()
decoded_text = non_decoded_text.decode('utf8')
import xmlbuilder
builder = xmlbuilder.XMLBuilder()
builder.foo = dict(bar=non_decoded_text)
str(builder)

Decoding Error: You must configure default encoding

>>builder = xmlbuilder.XMLBuilder()
builder.foo = dict(bar=decoded_text)
str(builder)

'<?xml version="1.0" encoding="utf-8"?><foo><bar>\xc2\xae and
\xc2\x99</bar></foo>'

Note that I didn't have to do anything with the default encoding. I
simply had to decode the text file with the appropriate codec. So,
looking at your code, I'm guessing that you need to figure out where
you're reading in the "name", "url" and "image" values, and make sure
you're properly decoding that text.

STeVe

P.S. If you can find somewhere to file a bug report for XMLBuilder, you
really should. The error instructing you to configure the default
encoding is really just wrong.

[=?iso-8859-1?q?S=F6nmez_Kartal?=]

On 6 Eylül, 19:19, Steven Bethard <[e-mail]:

Sönmez Kartal wrote:

I was using the XMLBuilder(xmlbuilder.py). I'm writing XML files as
"f.write(str(xml))". At execution of that line, it gives error with
description, configure your default encoding...

[and later]

http://rafb.net/p/RfaF8215.html

products in the code is a list of dictionaries which are returned by
makeProduct function.

I'm not typing or pasting those characters into my script. So,
declaring an encoding didn't make it. :-( But, your code excerpt
runned well.

Gabriel Genellina wrote:

You should ensure that arguments to makeProduct are either:
- unicode objects
- ASCII strings

If you got them from some other place, decode the strings as soon as
possible into unicode. Read <http://www.amk.ca/python/howto/unicode>
to understand what's happening

To further illustrate Gabriel's point, here is some code where I read in
some UTF8 text from a file. If you properly decode that text from UTF8,
you don't get any errors. If you forget to decode that text, you'll get
exactly the "default encoding" error you were getting before:

>>f = open('temp.txt', 'w')
>>f.write(u'® and ™'.encode('utf8'))
>>f.close()
>>non_decoded_text = open('temp.txt').read()
>>decoded_text = non_decoded_text.decode('utf8')
>>import xmlbuilder
>>builder = xmlbuilder.XMLBuilder()
>>builder.foo = dict(bar=non_decoded_text)
>>str(builder)

Decoding Error: You must configure default encoding

>>builder = xmlbuilder.XMLBuilder()
>>builder.foo = dict(bar=decoded_text)
>>str(builder)

'<?xml version="1.0" encoding="utf-8"?><foo><bar>\xc2\xae and
\xc2\x99</bar></foo>'

Note that I didn't have to do anything with the default encoding. I
simply had to decode the text file with the appropriate codec. So,
looking at your code, I'm guessing that you need to figure out where
you're reading in the "name", "url" and "image" values, and make sure
you're properly decoding that text.

STeVe

P.S. If you can find somewhere to file a bug report for XMLBuilder, you
really should. The error instructing you to configure the default
encoding is really just wrong.

Thank you Steve for your answers and Gabriel for reminding Unicode
HOWTO to me which I'm going to read tomorrow!

Happy coding