Hello
It seems like I have Unicode data in a CSV file but Python is using
a different code page, so isn't happy when I'm trying to read and put
this data into an SQLite database with APSW:
========
sql = "INSERT INTO mytable (col1,col2) VALUES (?,?)"
cursor.executem any(sql, records("test.t sv"))
"""
UnicodeDecodeEr ror: 'ascii' codec can't decode byte 0xc9 in position
18: ordinal not in range(128)
"""
========
What should I do so Python doesn't raise this error? Should I convert
data in the CVS file, or is there some function that I should call
before APSW's executemany()?
Thank you. 2 4851
Gilles Ganault wrote:
It seems like I have Unicode data in a CSV file but Python is using
a different code page, so isn't happy when I'm trying to read and put
this data into an SQLite database with APSW:
My guess is that you have non-ascii characters in a bytestring.
What should I do so Python doesn't raise this error? Should I convert
data in the CVS file, or is there some function that I should call
before APSW's executemany()?
You cannot have unicode data in a file, only unicode converted to
bytestrings using some encoding. Assuming that encoding is UTF-8 and that
apsw can cope with unicode, try to convert your data to unicode before
feeding it to the database api:
sql = "INSERT INTO mytable (col1,col2) VALUES (?,?)"
rows = ([col.decode("utf-8") for col in row] for row in
records("test.t sv"))
cursor.executem any(sql, rows)
Peter
On Tue, 17 Jun 2008 09:23:28 +0200, Peter Otten <__*******@web. de>
wrote:
Assuming that encoding is UTF-8 and that apsw can cope
with unicode, try to convert your data to unicode before
feeding it to the database api:
>sql = "INSERT INTO mytable (col1,col2) VALUES (?,?)"
rows = ([col.decode("utf-8") for col in row] for row in records("test. tsv"))
cursor.executem any(sql, rows)
Thanks again. This thread has been closed and replies have been disabled. Please start a new discussion. Similar topics |
by: Ruslan |
last post by:
Hi, everybody.
In this excerpt of code
enc = 'some_type_of_encoding'
def _encode(v):
if isinstance(v, UnicodeType):
v = v.encode(v)
return v
|
by: Robin Siebler |
last post by:
I have no idea what is causing this error, or how to fix it. The full error is:
Traceback (most recent call last):
File "D:\ScriptRuntime\PS\Automation\Handlers\SCMTestToolResourceToolsBAT.py",
line 60, in Run
PS.Automation.Utility.System.AppendSystemPath(args, context)
File "D:\ScriptRuntime\PS\Automation\Utility\System.py", line 55, in AppendSys
temPath
AppendPathVariable("PATH", appendtext, context)
File...
|
by: ash |
last post by:
hi,
one of the modules in my programs stopped wroking after i upgraded from
python 2.3 to 2.4. I also changed my wxPython to unicode supported one
during the process.
what the module essentially does is search for a stirng pattern form a
list of strings.
this is the function:
def srchqu(self):
for i in range(1,len(qu)):#qu is the list of strings
|
by: Robin Haswell |
last post by:
Okay I'm getting really frustrated with Python's Unicode handling, I'm
trying everything I can think of an I can't escape Unicode(En|De)codeError
no matter what I try.
Could someone explain to me what I'm doing wrong here, so I can hope to
throw light on the myriad of similar problems I'm having? Thanks :-)
Python 2.4.1 (#2, May 6 2005, 11:22:24)
on linux2
Type "help", "copyright", "credits" or "license" for more information.
|
by: kath |
last post by:
I have a number of excel files. In each file DATE is represented by
different name. I want to read the date from those different file. Also
the date is in different column in different file.
To identify the date field in different files I have created a file
called _globals where I keep all aliases for DATE in a array called
'alias_DATE'.
Array alias_DATE looks like,
| |
by: Oleg Parashchenko |
last post by:
Hello,
I'm working on an unicode-aware application. I like to use "print" to
debug programs, but in this case it was nightmare. The most popular
result of "print" was:
UnicodeDecodeError: 'ascii' codec can't decode byte 0xXX in position
0: ordinal not in range(128)
I spent two hours fixing it, and I hope it's done. The solution is one
|
by: Jorgen Bodde |
last post by:
Hi All,
I am relatively new to python unicode pains and I would like to have
some advice. I have this snippet of code:
def playFile(cmd, args):
argstr = list()
for arg in appcfg.options.split():
thefile = args
filemask = u"%file%"
|
by: Jorgen Bodde |
last post by:
Hi Edwin,
Filemask is obvious as it is assigned in the python code itself. It is
"%file%". The idea is that the file clicked is substituted for the
"%file%" by the replace action. The file that needs to be substituted
is a simple file on disk.
Here is a dump of the file and it's characters. I do understand that
it is not in the range of ASCII but how can I make it so that it will
work?
|
by: Gilles Ganault |
last post by:
Hello
I'm getting this error while downloading and parsing web pages:
=====
title = m.group(1)
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe9 in position
48: ordinal not in range(128)
=====
|
by: Hystou |
last post by:
Most computers default to English, but sometimes we require a different language, especially when relocating. Forgot to request a specific language before your computer shipped? No problem! You can effortlessly switch the default language on Windows 10 without reinstalling. I'll walk you through it.
First, let's disable language synchronization. With a Microsoft account, language settings sync across devices. To prevent any complications,...
|
by: Oralloy |
last post by:
Hello folks,
I am unable to find appropriate documentation on the type promotion of bit-fields when using the generalised comparison operator "<=>".
The problem is that using the GNU compilers, it seems that the internal comparison operator "<=>" tries to promote arguments from unsigned to signed.
This is as boiled down as I can make it.
Here is my compilation command:
g++-12 -std=c++20 -Wnarrowing bit_field.cpp
Here is the code in...
| |
by: Hystou |
last post by:
Overview:
Windows 11 and 10 have less user interface control over operating system update behaviour than previous versions of Windows. In Windows 11 and 10, there is no way to turn off the Windows Update option using the Control Panel or Settings app; it automatically checks for updates and installs any it finds, whether you like it or not. For most users, this new feature is actually very convenient. If you want to control the update process,...
|
by: tracyyun |
last post by:
Dear forum friends,
With the development of smart home technology, a variety of wireless communication protocols have appeared on the market, such as Zigbee, Z-Wave, Wi-Fi, Bluetooth, etc. Each protocol has its own unique characteristics and advantages, but as a user who is planning to build a smart home system, I am a bit confused by the choice of these technologies. I'm particularly interested in Zigbee because I've heard it does some...
|
by: conductexam |
last post by:
I have .net C# application in which I am extracting data from word file and save it in database particularly. To store word all data as it is I am converting the whole word file firstly in HTML and then checking html paragraph one by one.
At the time of converting from word file to html my equations which are in the word document file was convert into image.
Globals.ThisAddIn.Application.ActiveDocument.Select();...
|
by: TSSRALBI |
last post by:
Hello
I'm a network technician in training and I need your help.
I am currently learning how to create and manage the different types of VPNs and I have a question about LAN-to-LAN VPNs.
The last exercise I practiced was to create a LAN-to-LAN VPN between two Pfsense firewalls, by using IPSEC protocols.
I succeeded, with both firewalls in the same network. But I'm wondering if it's possible to do the same thing, with 2 Pfsense firewalls...
|
by: 6302768590 |
last post by:
Hai team
i want code for transfer the data from one system to another through IP address by using C# our system has to for every 5mins then we have to update the data what the data is updated we have to send another system
|
by: muto222 |
last post by:
How can i add a mobile payment intergratation into php mysql website.
| |
by: bsmnconsultancy |
last post by:
In today's digital era, a well-designed website is crucial for businesses looking to succeed. Whether you're a small business owner or a large corporation in Toronto, having a strong online presence can significantly impact your brand's success. BSMN Consultancy, a leader in Website Development in Toronto offers valuable insights into creating effective websites that not only look great but also perform exceptionally well. In this comprehensive...
| |