473,324 Members | 2,002 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,324 software developers and data experts.

what is this UnicodeDecodeError:....?

I have a number of excel files. In each file DATE is represented by
different name. I want to read the date from those different file. Also
the date is in different column in different file.

To identify the date field in different files I have created a file
called _globals where I keep all aliases for DATE in a array called
'alias_DATE'.

Array alias_DATE looks like,

alias_DATE=['TRADEDATE', 'Accounting Date', 'Date de VL','Datum',
'Kurs-datum', 'Date', 'Fecha Datos', 'Calculation Date', 'ClosingDate',
'Pricing Date', 'NAV Date', 'NAVDate', 'NAVDATE', 'ValuationDate',
'Datestamp', 'Fecha de Valoración', 'Kurs-','datum',
"""Kurs-\ndatum""", "Kurs-\ndatum"]

Now I want the index of the column where date is there. I followed the
with followin code.

>>b=xlrd.open_workbook('Santander_051206.xls')
sh=b.sheet_by_index(0)
sh.cell_value(rowx=0, colx=11)
u'Fecha de Valoraci\xf3n'
>>val=sh.cell_value(rowx=0, colx=11)
val
u'Fecha de Valoraci\xf3n'
>>print val
Fecha de Valoración
>>import _globals # the file where I have stored my 'alias_DATE' array
_globals.alias_DATE.index(val)
Traceback (most recent call last):
File "<interactive input>", line 1, in ?
UnicodeDecodeError: 'ascii' codec can't decode byte 0xf3 in position
17: ordinal not in range(128)
>>>
Though I have matching value in the array, why I am getting this error.
Can any one please tell me why is this error, and how to get rid of
this error. Because I have some files which containing some more
special characters.
Thank you in advance.
Sudhir.

Oct 10 '06 #1
7 3006
In <11**********************@k70g2000cwa.googlegroups .com>, kath wrote:
To identify the date field in different files I have created a file
called _globals where I keep all aliases for DATE in a array called
'alias_DATE'.

Array alias_DATE looks like,

alias_DATE=['TRADEDATE', 'Accounting Date', 'Date de VL','Datum',
'Kurs-datum', 'Date', 'Fecha Datos', 'Calculation Date', 'ClosingDate',
'Pricing Date', 'NAV Date', 'NAVDate', 'NAVDATE', 'ValuationDate',
'Datestamp', 'Fecha de Valoración', 'Kurs-','datum',
"""Kurs-\ndatum""", "Kurs-\ndatum"]

Now I want the index of the column where date is there. I followed the
with followin code.

>>>b=xlrd.open_workbook('Santander_051206.xls')
sh=b.sheet_by_index(0)
sh.cell_value(rowx=0, colx=11)
u'Fecha de Valoraci\xf3n'
>>>val=sh.cell_value(rowx=0, colx=11)
val
u'Fecha de Valoraci\xf3n'
>>>print val
Fecha de Valoración
>>>import _globals # the file where I have stored my 'alias_DATE' array
_globals.alias_DATE.index(val)
Traceback (most recent call last):
File "<interactive input>", line 1, in ?
UnicodeDecodeError: 'ascii' codec can't decode byte 0xf3 in position
17: ordinal not in range(128)
>>>>

Though I have matching value in the array, why I am getting this error.
Because you are trying to compare a unicode string `val` with a byte
string in the list. The unicode string will be converted to a byte string
for this comparison with the default encoding: ASCII. But 'ó' is not
contained in ASCII.
Can any one please tell me why is this error, and how to get rid of
this error. Because I have some files which containing some more
special characters.
Either use an unicode string in the list search too or explicitly encode
the unicode string `val` with the appropriate encoding before using it to
search the list.

Ciao,
Marc 'BlackJack' Rintsch
Oct 10 '06 #2
kath wrote:
I have a number of excel files. In each file DATE is represented by
different name. I want to read the date from those different file. Also
the date is in different column in different file.

To identify the date field in different files I have created a file
called _globals where I keep all aliases for DATE in a array called
'alias_DATE'.
It's actually a list. In Python an array is something else; look at the
docs for the array module if you're interested.
>
Array alias_DATE looks like,

alias_DATE=['TRADEDATE', 'Accounting Date', 'Date de VL','Datum',
'Kurs-datum', 'Date', 'Fecha Datos', 'Calculation Date', 'ClosingDate',
'Pricing Date', 'NAV Date', 'NAVDate', 'NAVDATE', 'ValuationDate',
'Datestamp', 'Fecha de Valoración', 'Kurs-','datum',
"""Kurs-\ndatum""", "Kurs-\ndatum"]
Nothing to do with the question you asked, but the last two entries
have the same value; is that intentional?
| >>"""Kurs-\ndatum""" == "Kurs-\ndatum"
| True

>
Now I want the index of the column where date is there. I followed the
with followin code.

>b=xlrd.open_workbook('Santander_051206.xls')
sh=b.sheet_by_index(0)
sh.cell_value(rowx=0, colx=11)
u'Fecha de Valoraci\xf3n'
>val=sh.cell_value(rowx=0, colx=11)
val
u'Fecha de Valoraci\xf3n'
>print val
Fecha de Valoración
>import _globals # the file where I have stored my 'alias_DATE' array
_globals.alias_DATE.index(val)
Traceback (most recent call last):
File "<interactive input>", line 1, in ?
UnicodeDecodeError: 'ascii' codec can't decode byte 0xf3 in position
17: ordinal not in range(128)
>>

Though I have matching value in the array, why I am getting this error.
Can any one please tell me why is this error, and how to get rid of
this error. Because I have some files which containing some more
special characters.
Hello again, Sudhir.

The text string returned by xlrd is a unicode object (u'Fecha de
Valoraci\xf3n'). The text strings in your list are str objects, encoded
in some unspecified encoding. Python is trying to convert the str
object 'Fecha de Valoración' to Unicode, using the (default) ascii
codec to do the conversion, and failing.

One way to handle this is to specify any non-ASCII strings in your
lookup list as unicode, like this:

contents of sudhir.py:
| # -*- coding: cp1252 -*-
| alist = ['Datestamp', u'Fecha de Valoraci\xf3n', 'Kurs-','datum']
| blist = ['Datestamp', u'Fecha de Valoración', 'Kurs-','datum']
| assert alist == blist
| val = u'Fecha de Valoraci\xf3n'
| print 'a', alist.index(val)
| print 'b', blist.index(val)

| OS prompt>sudhir.py
| a 1
| b 1

Note: the encoding "cp1252" is appropriate to my environment, not
necessarily to yours.

You may like to have a look through this:
http://www.amk.ca/python/howto/unicode

HTH,
John

Oct 10 '06 #3

Marc 'BlackJack' Rintsch wrote:
Because you are trying to compare a unicode string `val` with a byte
string in the list. The unicode string will be converted to a byte string
for this comparison with the default encoding: ASCII.
:-)

I presume you must live north of the equator. Down under, it seems to
happen the other way up -- the byte strings are decoded to unicode:

| >>['a', 'exotic1\xff', 'exotic2\xf3'].index(u'\xf3')
| Traceback (most recent call last):
| File "<stdin>", line 1, in ?
| UnicodeDecodeError: 'ascii' codec can't decode byte 0xff in position
7: ordinal not in range(128)

(-:

Oct 10 '06 #4
John Machin wrote:
Marc 'BlackJack' Rintsch wrote:

>>Because you are trying to compare a unicode string `val` with a byte
string in the list. The unicode string will be converted to a byte string
for this comparison with the default encoding: ASCII.


:-)

I presume you must live north of the equator. Down under, it seems to
happen the other way up -- the byte strings are decoded to unicode:

| >>['a', 'exotic1\xff', 'exotic2\xf3'].index(u'\xf3')
| Traceback (most recent call last):
| File "<stdin>", line 1, in ?
| UnicodeDecodeError: 'ascii' codec can't decode byte 0xff in position
7: ordinal not in range(128)

(-:
I see you also use little-endian smileys in the antipodes.

regards
Steve
--
Steve Holden +44 150 684 7255 +1 800 494 3119
Holden Web LLC/Ltd http://www.holdenweb.com
Skype: holdenweb http://holdenweb.blogspot.com
Recent Ramblings http://del.icio.us/steve.holden

Oct 11 '06 #5
In <11**********************@c28g2000cwb.googlegroups .com>, John Machin
wrote:
Marc 'BlackJack' Rintsch wrote:
>Because you are trying to compare a unicode string `val` with a byte
string in the list. The unicode string will be converted to a byte string
for this comparison with the default encoding: ASCII.

:-)

I presume you must live north of the equator. Down under, it seems to
happen the other way up -- the byte strings are decoded to unicode:
(-: Ooops, I stand corrected. :-)

Ciao,
Marc 'BlackJack' Rintsch
Oct 11 '06 #6

Steve Holden wrote:
John Machin wrote:
:-)
[stuff]
(-:
I see you also use little-endian smileys in the antipodes.
I was using it in a bracketing manner similar to the Spanish ¿and ¡
except at the other end of the bracketed text. This admittedly
confusing usage of course overloads the normal :-) While that sort of
caper should be a doddle for a document-level parser, it could present
a problem to parsers with limited buffers (like humans), so it looks
like I should reverse the order.

I wonder what Unicode.org would think of a proposal for 4 new
characters: open/close smiley/grumpy bracket. No weirder than some of
the characters on the roster.

Cheers,
John

Oct 11 '06 #7
John Machin wrote:
kath wrote:
I have a number of excel files. In each file DATE is represented by
different name. I want to read the date from those different file. Also
the date is in different column in different file.

To identify the date field in different files I have created a file
called _globals where I keep all aliases for DATE in a array called
'alias_DATE'.

It's actually a list. In Python an array is something else; look at the
docs for the array module if you're interested.

Array alias_DATE looks like,

alias_DATE=['TRADEDATE', 'Accounting Date', 'Date de VL','Datum',
'Kurs-datum', 'Date', 'Fecha Datos', 'Calculation Date', 'ClosingDate',
'Pricing Date', 'NAV Date', 'NAVDate', 'NAVDATE', 'ValuationDate',
'Datestamp', 'Fecha de Valoración', 'Kurs-','datum',
"""Kurs-\ndatum""", "Kurs-\ndatum"]

Nothing to do with the question you asked, but the last two entries
have the same value; is that intentional?
| >>"""Kurs-\ndatum""" == "Kurs-\ndatum"
| True


Now I want the index of the column where date is there. I followed the
with followin code.

>>b=xlrd.open_workbook('Santander_051206.xls')
>>sh=b.sheet_by_index(0)
>>sh.cell_value(rowx=0, colx=11)
u'Fecha de Valoraci\xf3n'
>>val=sh.cell_value(rowx=0, colx=11)
>>val
u'Fecha de Valoraci\xf3n'
>>print val
Fecha de Valoración
>>import _globals # the file where I have stored my 'alias_DATE' array
>>_globals.alias_DATE.index(val)
Traceback (most recent call last):
File "<interactive input>", line 1, in ?
UnicodeDecodeError: 'ascii' codec can't decode byte 0xf3 in position
17: ordinal not in range(128)
>>>
Though I have matching value in the array, why I am getting this error.
Can any one please tell me why is this error, and how to get rid of
this error. Because I have some files which containing some more
special characters.

Hello again, Sudhir.

The text string returned by xlrd is a unicode object (u'Fecha de
Valoraci\xf3n'). The text strings in your list are str objects, encoded
in some unspecified encoding. Python is trying to convert the str
object 'Fecha de Valoración' to Unicode, using the (default) ascii
codec to do the conversion, and failing.

One way to handle this is to specify any non-ASCII strings in your
lookup list as unicode, like this:

contents of sudhir.py:
| # -*- coding: cp1252 -*-
| alist = ['Datestamp', u'Fecha de Valoraci\xf3n', 'Kurs-','datum']
| blist = ['Datestamp', u'Fecha de Valoración', 'Kurs-','datum']
| assert alist == blist
| val = u'Fecha de Valoraci\xf3n'
| print 'a', alist.index(val)
| print 'b', blist.index(val)

| OS prompt>sudhir.py
| a 1
| b 1

Note: the encoding "cp1252" is appropriate to my environment, not
necessarily to yours.

You may like to have a look through this:
http://www.amk.ca/python/howto/unicode

HTH,
John

Hi.... thanks for your brave reply. The link you gave was the good one.
It had comprehensive information.I enjoyed reading it. Well it cleared
my doubts regarding encoding data, what is Unicode data, how to deal
with unicode data.

Thank you very much..

Regards,
sudhir.

Oct 11 '06 #8

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

16
by: Jim Hefferon | last post by:
Hello, I'm getting an error join-ing strings and wonder if someone can explain why the function is behaving this way? If I .join in a string that contains a high character then I get an ascii...
4
by: Robin Siebler | last post by:
I have no idea what is causing this error, or how to fix it. The full error is: Traceback (most recent call last): File "D:\ScriptRuntime\PS\Automation\Handlers\SCMTestToolResourceToolsBAT.py",...
22
by: jeremito | last post by:
I am writing a class that is intended to be subclassed. What is the proper way to indicate that a sub class must override a method? Thanks, Jeremy
5
by: Ed Jensen | last post by:
I'm really enjoying using the Python interactive interpreter to learn more about the language. It's fantastic you can get method help right in there as well. It saves a lot of time. With that...
1
by: Karl | last post by:
error msg: Mod_python error: "PythonHandler mod_python.publisher" Traceback (most recent call last): File "/usr/lib/python2.3/site-packages/mod_python/apache.py", line 299, in HandlerDispatch...
1
by: Ben | last post by:
Is the following a known bug? $ python -U Python 2.4.4 (#1, Oct 23 2006, 13:58:18) on linux2 Type "help", "copyright", "credits" or "license" for more information. Traceback (most recent...
3
by: Jorgen Bodde | last post by:
Hi All, I am relatively new to python unicode pains and I would like to have some advice. I have this snippet of code: def playFile(cmd, args): argstr = list() for arg in...
0
by: Edwin.Madari | last post by:
if you can print out values of 'filemask', and 'thefile' variables, when it crashes, I can help. thx. Edwin -----Original Message----- From:...
7
by: Gilles Ganault | last post by:
Hello Data that I download from the web seems to be using different code pages at times, and Python doesn't like this. Google returned a way to handle this, but I'm still getting an error:...
0
by: DolphinDB | last post by:
Tired of spending countless mintues downsampling your data? Look no further! In this article, you’ll learn how to efficiently downsample 6.48 billion high-frequency records to 61 million...
0
by: ryjfgjl | last post by:
ExcelToDatabase: batch import excel into database automatically...
0
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 6 Mar 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:15 (7.15PM). In this month's session, we are pleased to welcome back...
0
by: Vimpel783 | last post by:
Hello! Guys, I found this code on the Internet, but I need to modify it a little. It works well, the problem is this: Data is sent from only one cell, in this case B5, but it is necessary that data...
1
by: CloudSolutions | last post by:
Introduction: For many beginners and individual users, requiring a credit card and email registration may pose a barrier when starting to use cloud servers. However, some cloud server providers now...
1
by: Defcon1945 | last post by:
I'm trying to learn Python using Pycharm but import shutil doesn't work
1
by: Shællîpôpï 09 | last post by:
If u are using a keypad phone, how do u turn on JavaScript, to access features like WhatsApp, Facebook, Instagram....
0
by: Faith0G | last post by:
I am starting a new it consulting business and it's been a while since I setup a new website. Is wordpress still the best web based software for hosting a 5 page website? The webpages will be...
0
isladogs
by: isladogs | last post by:
The next Access Europe User Group meeting will be on Wednesday 3 Apr 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM). In this session, we are pleased to welcome former...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.