By using this site, you agree to our updated Privacy Policy and our Terms of Use. Manage your Cookies Settings.
464,667 Members | 1,348 Online
Bytes IT Community
+ Ask a Question
Need help? Post your question and get tips & solutions from a community of 464,667 IT Pros & Developers. It's quick & easy.

UnicodeDecodeError: 'ascii' codec can't decode byte

P: n/a
Hello

It seems like I have Unicode data in a CSV file but Python is using
a different code page, so isn't happy when I'm trying to read and put
this data into an SQLite database with APSW:

========
sql = "INSERT INTO mytable (col1,col2) VALUES (?,?)"
cursor.executemany(sql, records("test.tsv"))
"""
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc9 in position
18: ordinal not in range(128)
"""
========

What should I do so Python doesn't raise this error? Should I convert
data in the CVS file, or is there some function that I should call
before APSW's executemany()?

Thank you.
Jun 27 '08 #1
Share this Question
Share on Google+
2 Replies

P: n/a
Gilles Ganault wrote:
It seems like I have Unicode data in a CSV file but Python is using
a different code page, so isn't happy when I'm trying to read and put
this data into an SQLite database with APSW:
My guess is that you have non-ascii characters in a bytestring.
What should I do so Python doesn't raise this error? Should I convert
data in the CVS file, or is there some function that I should call
before APSW's executemany()?
You cannot have unicode data in a file, only unicode converted to
bytestrings using some encoding. Assuming that encoding is UTF-8 and that
apsw can cope with unicode, try to convert your data to unicode before
feeding it to the database api:
sql = "INSERT INTO mytable (col1,col2) VALUES (?,?)"
rows = ([col.decode("utf-8") for col in row] for row in
records("test.tsv"))
cursor.executemany(sql, rows)

Peter
Jun 27 '08 #2

P: n/a
On Tue, 17 Jun 2008 09:23:28 +0200, Peter Otten <__*******@web.de>
wrote:
Assuming that encoding is UTF-8 and that apsw can cope
with unicode, try to convert your data to unicode before
feeding it to the database api:
>sql = "INSERT INTO mytable (col1,col2) VALUES (?,?)"

rows = ([col.decode("utf-8") for col in row] for row in
records("test.tsv"))
cursor.executemany(sql, rows)
Thanks again.
Jun 27 '08 #3

This discussion thread is closed

Replies have been disabled for this discussion.