473,387 Members | 1,398 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,387 software developers and data experts.

Unicode -> Python -> DBAPI -> PyPgSQL -> PostgreSQL

I can't seem to find any way to specify the character encoding with the DB
API implementation of PyPgSQL. There is no mention of encoding and Unicode
in the DB API v2.0 spec and the PyPgSQL README.

When I have Unicode strings in Python and store it in a PostgreSQL Unicode
database, will the data automatically be correctly encoded? Or do I need
to specify the UTF-8 client encoding on the database connection somehow?

I'm using the current packages of Debian stable (woody):
Python 2.2
PyPgSQL 2.0
PostgreSQL 7.2 (database created with UNICODE / UTF-8 encoding)

--
René Pijlman
Jul 18 '05 #1
2 3580
Rene Pijlman wrote:
I can't seem to find any way to specify the character encoding with the DB
API implementation of PyPgSQL. There is no mention of encoding and Unicode
in the DB API v2.0 spec and the PyPgSQL README. [...]


See section 2.2.5 in the pyPgSQL README:

pyPgSQL has a few extensions that make it possible to insert Unicode strings
into PostgreSQL and fetch unicode strings instead of byte strings from the
database.

The module-level connect() function has two Unicode-related parameters:

- client_encoding
- unicode_results

*client_encoding* accepts the same parameters as the encode method
of Unicode strings. If you also want to set a policy for encoding
errors, set client_encoding to a tuple, like ("koi8-r", "replace")

Note that you still must make sure that the PostgreSQL client is
using the same encoding as set with the client_encoding parameter.
This is typically done by issuing a "SET CLIENT_ENCODING TO ..."
SQL statement immediately after creating the connection.

If you also want to fetch Unicode strings from the database, set
*unicode_results* to 1.

For example, assuming a database created with *createdb mydb -E UNICODE*
and a
table *TEST(V VARCHAR(50))*:
from pyPgSQL import PgSQL
cx = PgSQL.connect(database="mydb", client_encoding="utf-8", unicode_results=1) cu = cx.cursor()
cu.execute("set client_encoding to unicode")
cu.execute("insert into test(v) values (%s)", (u'\x99sterreich',))
cu.execute("select v from test")
cu.fetchone() [u'\x99sterreich']

-- Gerhard
Jul 18 '05 #2
Gerhard Häring:
Rene Pijlman:
I can't seem to find any way to specify the character encoding with the DB
API implementation of PyPgSQL. There is no mention of encoding and Unicode
in the DB API v2.0 spec and the PyPgSQL README. [...]


See section 2.2.5 in the pyPgSQL README:


Well, its not in this README I found in the Debian package python2.2-pgsql
with pyPgSQL 2.0:
#ident "@(#) $Id: README,v 1.20 2001/11/05 01:18:12 ghaering Exp $"
pyPgSQL - v2.0: Python DB-API 2.0 Compliant Interface Module for
PostgreSQL.

But this tells me I probably need to upgrade to pyPgSQL 2.3 or 2.4:
"Q: I’ve heard of Unicode support for pyPgSQL. What’s the current status?
A: It’s integrated in pyPgSQL 2.3."
http://pypgsql.sourceforge.net/pypgsql-faq.pdf

Thanks a lot Gerhard, you've put me on the right track.

--
René Pijlman
Jul 18 '05 #3

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

5
by: Paradox | last post by:
In general I love Python for text manipulation but at our company we have the need to manipulate large text values stored in either a SQL Server database or text files. This data is stored in a...
2
by: leroybt.rm | last post by:
I don't understand why this does not work: <FILE1> test1.py #Import Packages import string # data=0 data=data+1
1
by: Michael R Seefelt | last post by:
I have written a simple C-program that loads a Python Function that connects to a PostgreSQL database> When I only load and execute the Python function all works OK. If I call the Python function a...
49
IDE
by: Thomas Lindgaard | last post by:
Hello I am probably going to start a war now... but so be it :) I just want to hear what all you guys who eat pythons for breakfast use for python coding. Currently I use Kate, but I would...
5
by: john | last post by:
Hi I am devlopeing a data centric intranetsite with data in mysql database I would like your opinion on this architecture Data will be fetched from database by python & converted into xml The...
1
by: Trond Eivind Glomsrød | last post by:
I am currently writing a python interface to a C++ library. Some of the functions in this library take unicode strings (UTF-8, mostly) as arguments. However, when getting these data I run into...
4
by: news | last post by:
I've just started to test/learn python. I've got Linux > mandrake9 > python & documentation. What I'll initially want to be doing needs file I/O, so I wanted to confirm file I/O early in my...
5
by: hzqij | last post by:
i have a python source code test.py # -*- coding: UTF-8 -*- # s is a unicode string, include chinese s = u'ÕÅÈý' then i run $ python test.py
0
by: susikto | last post by:
hi, I am getting this error when I am trying to connect the PSQL database. PHP Warning: pg_connect() : Unable to connect to PostgreSQL server: FATAL: user &quot;apache&quot; does not exist Can...
0
by: taylorcarr | last post by:
A Canon printer is a smart device known for being advanced, efficient, and reliable. It is designed for home, office, and hybrid workspace use and can also be used for a variety of purposes. However,...
0
by: ryjfgjl | last post by:
If we have dozens or hundreds of excel to import into the database, if we use the excel import function provided by database editors such as navicat, it will be extremely tedious and time-consuming...
0
by: ryjfgjl | last post by:
In our work, we often receive Excel tables with data in the same format. If we want to analyze these data, it can be difficult to analyze them because the data is spread across multiple Excel files...
0
by: emmanuelkatto | last post by:
Hi All, I am Emmanuel katto from Uganda. I want to ask what challenges you've faced while migrating a website to cloud. Please let me know. Thanks! Emmanuel
1
by: nemocccc | last post by:
hello, everyone, I want to develop a software for my android phone for daily needs, any suggestions?
0
by: Hystou | last post by:
There are some requirements for setting up RAID: 1. The motherboard and BIOS support RAID configuration. 2. The motherboard has 2 or more available SATA protocol SSD/HDD slots (including MSATA, M.2...
0
by: Hystou | last post by:
Most computers default to English, but sometimes we require a different language, especially when relocating. Forgot to request a specific language before your computer shipped? No problem! You can...
0
Oralloy
by: Oralloy | last post by:
Hello folks, I am unable to find appropriate documentation on the type promotion of bit-fields when using the generalised comparison operator "<=>". The problem is that using the GNU compilers,...
0
jinu1996
by: jinu1996 | last post by:
In today's digital age, having a compelling online presence is paramount for businesses aiming to thrive in a competitive landscape. At the heart of this digital strategy lies an intricately woven...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.