473,385 Members | 1,326 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,385 software developers and data experts.

encode() question

s1 = "hello"
s2 = s1.encode("utf-8")

s1 = "an accented 'e': \xc3\xa9"
s2 = s1.encode("utf-8")

The last line produces the error:

---
Traceback (most recent call last):
File "test1.py", line 6, in ?
s2 = s1.encode("utf-8")
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position
17: ordinal not in range(128)
---

The error is a "decode" error, and as far as I can tell, decoding
happens when you convert a regular string to a unicode string. So, is
there an implicit conversion taking place from s1 to a unicode string
before encode() is called? By what mechanism?

Jul 31 '07 #1
6 2312
En Tue, 31 Jul 2007 13:53:11 -0300, 7stud <bb**********@yahoo.com>
escribió:
s1 = "hello"
s2 = s1.encode("utf-8")

s1 = "an accented 'e': \xc3\xa9"
s2 = s1.encode("utf-8")

The last line produces the error:

---
Traceback (most recent call last):
File "test1.py", line 6, in ?
s2 = s1.encode("utf-8")
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position
17: ordinal not in range(128)
---

The error is a "decode" error, and as far as I can tell, decoding
happens when you convert a regular string to a unicode string. So, is
there an implicit conversion taking place from s1 to a unicode string
before encode() is called? By what mechanism?
Converting from unicode characters into a string of bytes is the "encode"
operation: unicode.encode() -str
Converting from string of bytes to unicode characters is the "decode"
operation: str.decode() -unicode
str.decode and unicode.encode should NOT exist, or at least issue a
warning (IMHO).
When you try to do str.encode, as the encode operation requires an unicode
source, the string is first decoded using the default encoding - and fails.

--
Gabriel Genellina

Jul 31 '07 #2
On Jul 31, 11:18 am, "Gabriel Genellina" <gagsl-...@yahoo.com.ar>
wrote:
str.decode and unicode.encode should NOT exist, or at least issue a
warning (IMHO).
Yes, that sounds like a good idea.

Jul 31 '07 #3
On Tue, 31 Jul 2007 10:45:26 -0700, 7stud wrote:
On Jul 31, 11:18 am, "Gabriel Genellina" <gagsl-...@yahoo.com.ar>
wrote:
>str.decode and unicode.encode should NOT exist, or at least issue a
warning (IMHO).

Yes, that sounds like a good idea.
It sounds like horrible idea as those are the ones that are really needed.
One could argue about `str.encode` and `unicode.decode`. But there are at
least uses for `str.encode` like 'sting-escape', 'hex', 'bz2', 'base64'
etc.

Ciao,
Marc 'BlackJack' Rintsch
Jul 31 '07 #4
>>str.decode and unicode.encode should NOT exist, or at least issue a
>>warning (IMHO).
Yes, that sounds like a good idea.

It sounds like horrible idea as those are the ones that are really needed.
Correct.
One could argue about `str.encode` and `unicode.decode`. But there are at
least uses for `str.encode` like 'sting-escape', 'hex', 'bz2', 'base64'
etc.
Indeed, in Py3k, those will be gone.

Regards,
Martin
Jul 31 '07 #5
En Tue, 31 Jul 2007 16:41:48 -0300, Marc 'BlackJack' Rintsch
<bj****@gmx.netescribió:
On Tue, 31 Jul 2007 10:45:26 -0700, 7stud wrote:
>On Jul 31, 11:18 am, "Gabriel Genellina" <gagsl-...@yahoo.com.ar>
wrote:
>>str.decode and unicode.encode should NOT exist, or at least issue a
warning (IMHO).
Yes, that sounds like a good idea.

It sounds like horrible idea as those are the ones that are really
needed.
Ouch! caffeine levels below critical threshold, I think.

--
Gabriel Genellina

Aug 2 '07 #6
On Tue, Jul 31, 2007 at 09:53:11AM -0700, 7stud wrote:
s1 = "hello"
s2 = s1.encode("utf-8")

s1 = "an accented 'e': \xc3\xa9"
s2 = s1.encode("utf-8")

The last line produces the error:

---
Traceback (most recent call last):
File "test1.py", line 6, in ?
s2 = s1.encode("utf-8")
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position
17: ordinal not in range(128)
---

The error is a "decode" error, and as far as I can tell, decoding
happens when you convert a regular string to a unicode string. So, is
there an implicit conversion taking place from s1 to a unicode string
before encode() is called? By what mechanism?
Yep. You are trying to encode a string. The problem is that strings are
already encoded, so it generally makes no sense to call .encode() on
them.

..encode()ing a string can be handy if you want to convert its encoding.
In such a case, though, Python will first convert the string to Unicode.
To do that, it has to know how the string is encoded. Unless you tell it
otherwise, Python assumes the string is encoded in ascii. You had a byte
in there that was out of ascii's range...thus, the error. Python was
trying to decode the string, assumed it was ascii, but that didn't work.

This is all very confusing; I'd highly recommend reading this bit about
Unicode. It started me down the difficult road of actually understanding
what is going on here.

http://www.joelonsoftware.com/articles/Unicode.html
--
It's another Baseline Boulder Morning.
Aug 6 '07 #7

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

1
by: fartsniff | last post by:
hey all. for a small website, based off of phpnuke. running on a 2.4 down / 512 up cable connection, doing medium traffic. is it worth encoding the site using zend safeguard suite ? overall,...
3
by: Ricky | last post by:
Is there any way to detect if a field has been encoded before I decode a field.
5
by: Scott Matthews | last post by:
I've recently come upon an odd Javascript (and/or browser) behavior, and after hunting around the Web I still can't seem to find an answer. Specifically, I have noticed that the Javascript...
4
by: Newbie | last post by:
How would I modify this form to encode *all* the characters in the 'source' textarea to the '%xx' format & place result code into the 'output' textarea? (cross browser compatable) Any help is...
3
by: Peter | last post by:
Hi, I try to make up a javascript string which contains numeric numbers in any positions. For example, I want to make a string: secretcode, where secretcode.charAt(0)==(-21),...
1
by: ok | last post by:
I think my last question was not clear, so people gave me the reverse answer. I want to put a string in an html file, and human eyes or robots will not be able to read it. For example I want...
13
by: mario | last post by:
Hello! i stumbled on this situation, that is if I decode some string, below just the empty string, using the mcbs encoding, it succeeds, but if I try to encode it back with the same encoding it...
4
by: J Peyret | last post by:
Well, as usual I am confused by unicode encoding errors. I have a string with problematic characters in it which I'd like to put into a postgresql table. That results in a postgresql error so I...
1
by: anonymous | last post by:
1 Objective to write little programs to help me learn German. See code after numbered comments. //Thanks in advance for any direction or suggestions. tk 2 Want keyboard answer input, for...
1
by: CloudSolutions | last post by:
Introduction: For many beginners and individual users, requiring a credit card and email registration may pose a barrier when starting to use cloud servers. However, some cloud server providers now...
0
by: Faith0G | last post by:
I am starting a new it consulting business and it's been a while since I setup a new website. Is wordpress still the best web based software for hosting a 5 page website? The webpages will be...
0
by: ryjfgjl | last post by:
In our work, we often need to import Excel data into databases (such as MySQL, SQL Server, Oracle) for data analysis and processing. Usually, we use database tools like Navicat or the Excel import...
0
by: taylorcarr | last post by:
A Canon printer is a smart device known for being advanced, efficient, and reliable. It is designed for home, office, and hybrid workspace use and can also be used for a variety of purposes. However,...
0
by: aa123db | last post by:
Variable and constants Use var or let for variables and const fror constants. Var foo ='bar'; Let foo ='bar';const baz ='bar'; Functions function $name$ ($parameters$) { } ...
0
by: ryjfgjl | last post by:
If we have dozens or hundreds of excel to import into the database, if we use the excel import function provided by database editors such as navicat, it will be extremely tedious and time-consuming...
0
by: ryjfgjl | last post by:
In our work, we often receive Excel tables with data in the same format. If we want to analyze these data, it can be difficult to analyze them because the data is spread across multiple Excel files...
0
by: emmanuelkatto | last post by:
Hi All, I am Emmanuel katto from Uganda. I want to ask what challenges you've faced while migrating a website to cloud. Please let me know. Thanks! Emmanuel
1
by: Sonnysonu | last post by:
This is the data of csv file 1 2 3 1 2 3 1 2 3 1 2 3 2 3 2 3 3 the lengths should be different i have to store the data by column-wise with in the specific length. suppose the i have to...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.