I started playing with python a few weeks ago after a number of years
of perl programming and I can say that my first impression is,
unsurprisingly, quite positive. ;)
The reason I am writing here is that I can't seem to figure out how to
save/restore python objects into a relational database. The way I
used to do it in perl was to 'freeze' the object before storing it
into the database and 'thaw' it before restoring it. (For those not
familiar with the perl terminology freeze and thaw are method from
perl's persistence module Storable). I found that the python's
corresponding module is Pickle but it doesn't seem to work for me.. I
do roughly the following:
class TestA:
def insert():
i = TestA('asdf')
output = cStringIO.StringIO()
cPickle.dump(i, output, 2)
print "output.getvalue(): %s" % output.getvalue()
jd = libpq.PgQuoteBytea(output.getvalue())
print "jd %s" % jd
return dbpool.runOperation("insert into jobs (effective_job_id,
job_description) values (1, " + jd + ")") 10 3007
and then I try to restore the object with the following code
def success(rv):
print "success"
str = cStringIO.StringIO(libpq.PgUnQuoteBytea(rv[0][0]))
i = cPickle.load(str)
i.toString()
the execution fails just after the print statement, and I am not quite
sure why is that.
I would love to find out what people are using when they need to do
something similar --
perhaps I am trying to do it the perl way, while there is an elegant
python solution.
thanks
On 23 mai, 23:14, nayden <nay...@gmail.comwrote:
and then I try to restore the object with the following code
def success(rv):
print "success"
str = cStringIO.StringIO(libpq.PgUnQuoteBytea(rv[0][0]))
i = cPickle.load(str)
i.toString()
the execution fails just after the print statement, and I am not quite
sure why is that.
Please reread the doc for pickle.dump, pickle.dumps, pickle.load and
pickle.loads. You just don't need StringIO here, just use the 's
versions of the functions.
I would love to find out what people are using when they need to do
something similar
perhaps I am trying to do it the perl way, while there is an elegant
python solution.
I don't know if you'd label it 'elegant', but as far as I'm concerned,
storing serialized objects as blobs in a relational database is mostly
non-sense. If I use a relational database, it's because it is a
*relational* database. If you want an OODB, then we have the ZODB,
Durus and a couple others.
On May 24, 7:14 am, nayden <nay...@gmail.comwrote:
the execution fails just after the print statement, and I am not quite
sure why is that.
It's often helpful to include the traceback, or at the very least the
last 3-4 lines of it, as it helps everyone work out the issue you're
having.
If you're not sure which line in a function is causing the issue, try
commenting out all but the first, run-and-test, re-add the next, run-
and-test etc
But the error is most likely this line:
i = cPickle.load(str)
cPickle.load unpickles from a file, but here you're handing it a
string. You want cPickle.loads.
At the interpreter, you can always quickly check these out by looking
up the docstring via 'help(cPickle.loads)' (or 'cPickle.loads?' if
you're using iPythhon).
- alex23
On Fri, May 23, 2008 at 5:07 PM, nayden <na****@gmail.comwrote:
The reason I am writing here is that I can't seem to figure out how to
save/restore python objects into a relational database.
Here's a basic version using the sqlite bindings included with Python 2.5:
import sqlite3, pickle
thing = {'date': '2008-05-21',
'event': 'Something happened this day'}
conn = sqlite3.connect(':memory:')
c = conn.cursor()
c.execute('''create table stuff (idx integer, data text)''')
c.execute('''insert into stuff values (?,?)''', (1, pickle.dumps(thing)))
c.execute('''select data from stuff where idx=1''')
row = c.fetchone()
# sqlite3 stores text fields as unicode, so we need to encode to ascii
# before we unpickle
pickle_str = row[0].encode('ascii')
thing2 = pickle.loads(pickle_str)
print thing
print thing2
Maybe that will set you on the right track.
--
Jerry
On May 23, 7:00*pm, alex23 <wuwe...@gmail.comwrote:
On May 24, 7:14 am, nayden <nay...@gmail.comwrote:
the execution fails just after the print statement, and I am not quite
sure why is that.
It's often helpful to include the traceback, or at the very least the
last 3-4 lines of it, as it helps everyone work out the issue you're
having.
If you're not sure which line in a function is causing the issue, try
commenting out all but the first, run-and-test, re-add the next, run-
and-test etc
But the error is most likely this line:
* * i = cPickle.load(str)
cPickle.load unpickles from a file, but here you're handing it a
string. You want cPickle.loads.
At the interpreter, you can always quickly check these out by looking
up the docstring via 'help(cPickle.loads)' (or 'cPickle.loads?' if
you're using iPythhon).
- alex23
It's not a string it's a cStringIO.StringIO, even though his variable
name is confusing.
nayden: 'str' is a built-in variable that is the string type. Try this
for different values of x:
type(x) is str
str(x)
When you override it, it may be confusing down the line.
I'd suggest installing pychecker, which will help you catch errors
like this: http://pychecker.sourceforge.net/
"br*****************@gmail.com" <br*****************@gmail.comwrites:
I don't know if you'd label it 'elegant', but as far as I'm
concerned, storing serialized objects as blobs in a relational
database is mostly non-sense. If I use a relational database, it's
because it is a *relational* database. If you want an OODB, then we
have the ZODB, Durus and a couple others.
.... not to forget object-relational mappers like SQLAlchemy, SQLObject... br*****************@gmail.com pisze:
I don't know if you'd label it 'elegant', but as far as I'm concerned,
storing serialized objects as blobs in a relational database is mostly
non-sense. If I use a relational database, it's because it is a
*relational* database. If you want an OODB, then we have the ZODB,
Durus and a couple others.
It is sometimes convenient to store objects in mature relational
database backend (reliability, stability, support, tools,
replication, etc.). See latst efforts with RelStorage backend
for ZODB ( http://wiki.zope.org/ZODB/RelStorage) - it stores
pickled Python objects in Oracle, PostgreSQL or MySQL)
Ville M. Vainio a écrit :
"br*****************@gmail.com" <br*****************@gmail.comwrites:
>I don't know if you'd label it 'elegant', but as far as I'm concerned, storing serialized objects as blobs in a relational database is mostly non-sense. If I use a relational database, it's because it is a *relational* database. If you want an OODB, then we have the ZODB, Durus and a couple others.
... not to forget object-relational mappers like SQLAlchemy, SQLObject...
Which are more IMHO another way to "bridge" RBDMS with the programming
language than a way to persist objects.
Thanks everyone for the helpful suggestions.
On 24 mai, 13:01, Piotr Chamera <piotr_cham...@poczta.onet.plwrote:
bruno.desthuilli...@gmail.com pisze:
I don't know if you'd label it 'elegant', but as far as I'm concerned,
storing serialized objects as blobs in a relational database is mostly
non-sense. If I use a relational database, it's because it is a
*relational* database. If you want an OODB, then we have the ZODB,
Durus and a couple others.
It is sometimes convenient to store objects in mature relational
database backend (reliability, stability, support, tools,
replication, etc.). See latst efforts with RelStorage backend
for ZODB (http://wiki.zope.org/ZODB/RelStorage) - it stores
pickled Python objects in Oracle, PostgreSQL or MySQL)
You mean a SQL database backend here - the conveniences that you
mention have nothing to do with being relational or not. And that's my
point: pickling objects, you loose of the convenience of a
*relational* database. This thread has been closed and replies have been disabled. Please start a new discussion. Similar topics
by: Rolf Hemmerling |
last post by:
Hello !
Beginner's question:
What ist the easiest way to store and save objects in a file generated
by a C++ program, by using the "standard C++ library" and/or "Standard
Template Library (...
|
by: ViperDK \(Daniel K.\) |
last post by:
i've a database where relations are hold in a special way which the project
leaders think of as "performant and uncomplicated" but which is very
questionable to me:...
|
by: Tom |
last post by:
Hi,
I'm designing a website where all the articles/news will be stored in
XML. (I will use PHP to parse the XML.) Should I store the XML in
separate .xml files or should I store the XML in a...
|
by: Radi Radichev |
last post by:
Hi!
I'm making a database application and i heard from a friend that it is more
proffecional and easy to do this with bussines objects. Can anyone tell me
where i can find more info on bussines...
|
by: Mudcat |
last post by:
Hi,
I am trying to build a tool that analyzes stock data. Therefore I am
going to download and store quite a vast amount of it. Just for a
general number - assuming there are about 7000 listed...
|
by: nacho222 |
last post by:
I'm currently in the middle of writing a persistence framework, and I
have to make a design decission. The framework will take care of all
the saving and restoring objects, and also the...
|
by: Rhino |
last post by:
Is it possible to store Java objects in DB2 V8.2 for Windows/Unix/Linux via
JDBC?
Specifically, if I have a 4-dimensional boolean array, i.e. boolean,
can I store it directly in a column of a...
|
by: Penelope Dramas |
last post by:
Hello,
I'm in a front of very serious .net redesign/rewrite of an old VB6
application.
I had been asked to make it .NET 2.0 and would like to ask couple of
questions regarding data access as...
|
by: Lint Radley |
last post by:
Hi Everyone,
I need an opinion here on storing data for a program I am working on the
processes DICOM images. Essentially, my program stores 25-45 (it varies
depending on the user) ranges of...
|
by: isladogs |
last post by:
The next Access Europe meeting will be on Wednesday 7 Feb 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:30 (7.30PM).
In this month's session, the creator of the excellent VBE...
|
by: MeoLessi9 |
last post by:
I have VirtualBox installed on Windows 11 and now I would like to install Kali on a virtual machine. However, on the official website, I see two options: "Installer images" and "Virtual machines"....
|
by: DolphinDB |
last post by:
The formulas of 101 quantitative trading alphas used by WorldQuant were presented in the paper 101 Formulaic Alphas. However, some formulas are complex, leading to challenges in calculation.
Take...
|
by: DolphinDB |
last post by:
Tired of spending countless mintues downsampling your data? Look no further!
In this article, you’ll learn how to efficiently downsample 6.48 billion high-frequency records to 61 million...
|
by: Aftab Ahmad |
last post by:
Hello Experts!
I have written a code in MS Access for a cmd called "WhatsApp Message" to open WhatsApp using that very code but the problem is that it gives a popup message everytime I clicked on...
|
by: Aftab Ahmad |
last post by:
So, I have written a code for a cmd called "Send WhatsApp Message" to open and send WhatsApp messaage. The code is given below.
Dim IE As Object
Set IE =...
|
by: ryjfgjl |
last post by:
ExcelToDatabase: batch import excel into database automatically...
|
by: Vimpel783 |
last post by:
Hello!
Guys, I found this code on the Internet, but I need to modify it a little. It works well, the problem is this: Data is sent from only one cell, in this case B5, but it is necessary that data...
|
by: ArrayDB |
last post by:
The error message I've encountered is; ERROR:root:Error generating model response: exception: access violation writing 0x0000000000005140, which seems to be indicative of an access violation...
| |