473,288 Members | 1,705 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,288 software developers and data experts.

Storing objects in relational database

I started playing with python a few weeks ago after a number of years
of perl programming and I can say that my first impression is,
unsurprisingly, quite positive. ;)
The reason I am writing here is that I can't seem to figure out how to
save/restore python objects into a relational database. The way I
used to do it in perl was to 'freeze' the object before storing it
into the database and 'thaw' it before restoring it. (For those not
familiar with the perl terminology freeze and thaw are method from
perl's persistence module Storable). I found that the python's
corresponding module is Pickle but it doesn't seem to work for me.. I
do roughly the following:
class TestA:
def insert():
i = TestA('asdf')
output = cStringIO.StringIO()
cPickle.dump(i, output, 2)
print "output.getvalue(): %s" % output.getvalue()
jd = libpq.PgQuoteBytea(output.getvalue())
print "jd %s" % jd
return dbpool.runOperation("insert into jobs (effective_job_id,
job_description) values (1, " + jd + ")")
Jun 27 '08 #1
10 3007
and then I try to restore the object with the following code

def success(rv):
print "success"
str = cStringIO.StringIO(libpq.PgUnQuoteBytea(rv[0][0]))
i = cPickle.load(str)
i.toString()

the execution fails just after the print statement, and I am not quite
sure why is that.

I would love to find out what people are using when they need to do
something similar --
perhaps I am trying to do it the perl way, while there is an elegant
python solution.

thanks
Jun 27 '08 #2
On 23 mai, 23:14, nayden <nay...@gmail.comwrote:
and then I try to restore the object with the following code

def success(rv):
print "success"
str = cStringIO.StringIO(libpq.PgUnQuoteBytea(rv[0][0]))
i = cPickle.load(str)
i.toString()

the execution fails just after the print statement, and I am not quite
sure why is that.
Please reread the doc for pickle.dump, pickle.dumps, pickle.load and
pickle.loads. You just don't need StringIO here, just use the 's
versions of the functions.
I would love to find out what people are using when they need to do
something similar
perhaps I am trying to do it the perl way, while there is an elegant
python solution.
I don't know if you'd label it 'elegant', but as far as I'm concerned,
storing serialized objects as blobs in a relational database is mostly
non-sense. If I use a relational database, it's because it is a
*relational* database. If you want an OODB, then we have the ZODB,
Durus and a couple others.
Jun 27 '08 #3
On May 24, 7:14 am, nayden <nay...@gmail.comwrote:
the execution fails just after the print statement, and I am not quite
sure why is that.
It's often helpful to include the traceback, or at the very least the
last 3-4 lines of it, as it helps everyone work out the issue you're
having.

If you're not sure which line in a function is causing the issue, try
commenting out all but the first, run-and-test, re-add the next, run-
and-test etc

But the error is most likely this line:
i = cPickle.load(str)
cPickle.load unpickles from a file, but here you're handing it a
string. You want cPickle.loads.

At the interpreter, you can always quickly check these out by looking
up the docstring via 'help(cPickle.loads)' (or 'cPickle.loads?' if
you're using iPythhon).

- alex23
Jun 27 '08 #4
On Fri, May 23, 2008 at 5:07 PM, nayden <na****@gmail.comwrote:
The reason I am writing here is that I can't seem to figure out how to
save/restore python objects into a relational database.
Here's a basic version using the sqlite bindings included with Python 2.5:

import sqlite3, pickle

thing = {'date': '2008-05-21',
'event': 'Something happened this day'}

conn = sqlite3.connect(':memory:')
c = conn.cursor()
c.execute('''create table stuff (idx integer, data text)''')
c.execute('''insert into stuff values (?,?)''', (1, pickle.dumps(thing)))
c.execute('''select data from stuff where idx=1''')
row = c.fetchone()
# sqlite3 stores text fields as unicode, so we need to encode to ascii
# before we unpickle
pickle_str = row[0].encode('ascii')
thing2 = pickle.loads(pickle_str)

print thing
print thing2

Maybe that will set you on the right track.

--
Jerry
Jun 27 '08 #5
On May 23, 7:00*pm, alex23 <wuwe...@gmail.comwrote:
On May 24, 7:14 am, nayden <nay...@gmail.comwrote:
the execution fails just after the print statement, and I am not quite
sure why is that.

It's often helpful to include the traceback, or at the very least the
last 3-4 lines of it, as it helps everyone work out the issue you're
having.

If you're not sure which line in a function is causing the issue, try
commenting out all but the first, run-and-test, re-add the next, run-
and-test etc

But the error is most likely this line:
* * i = cPickle.load(str)

cPickle.load unpickles from a file, but here you're handing it a
string. You want cPickle.loads.

At the interpreter, you can always quickly check these out by looking
up the docstring via 'help(cPickle.loads)' (or 'cPickle.loads?' if
you're using iPythhon).

- alex23
It's not a string it's a cStringIO.StringIO, even though his variable
name is confusing.

nayden: 'str' is a built-in variable that is the string type. Try this
for different values of x:
type(x) is str
str(x)
When you override it, it may be confusing down the line.

I'd suggest installing pychecker, which will help you catch errors
like this:
http://pychecker.sourceforge.net/
Jun 27 '08 #6
"br*****************@gmail.com" <br*****************@gmail.comwrites:

I don't know if you'd label it 'elegant', but as far as I'm
concerned, storing serialized objects as blobs in a relational
database is mostly non-sense. If I use a relational database, it's
because it is a *relational* database. If you want an OODB, then we
have the ZODB, Durus and a couple others.
.... not to forget object-relational mappers like SQLAlchemy, SQLObject...
Jun 27 '08 #7
br*****************@gmail.com pisze:
I don't know if you'd label it 'elegant', but as far as I'm concerned,
storing serialized objects as blobs in a relational database is mostly
non-sense. If I use a relational database, it's because it is a
*relational* database. If you want an OODB, then we have the ZODB,
Durus and a couple others.
It is sometimes convenient to store objects in mature relational
database backend (reliability, stability, support, tools,
replication, etc.). See latst efforts with RelStorage backend
for ZODB (http://wiki.zope.org/ZODB/RelStorage) - it stores
pickled Python objects in Oracle, PostgreSQL or MySQL)
Jun 27 '08 #8
Ville M. Vainio a écrit :
"br*****************@gmail.com" <br*****************@gmail.comwrites:

>I don't know if you'd label it 'elegant', but as far as I'm
concerned, storing serialized objects as blobs in a relational
database is mostly non-sense. If I use a relational database, it's
because it is a *relational* database. If you want an OODB, then we
have the ZODB, Durus and a couple others.

... not to forget object-relational mappers like SQLAlchemy, SQLObject...
Which are more IMHO another way to "bridge" RBDMS with the programming
language than a way to persist objects.
Jun 27 '08 #9
Thanks everyone for the helpful suggestions.
Jun 27 '08 #10
On 24 mai, 13:01, Piotr Chamera <piotr_cham...@poczta.onet.plwrote:
bruno.desthuilli...@gmail.com pisze:
I don't know if you'd label it 'elegant', but as far as I'm concerned,
storing serialized objects as blobs in a relational database is mostly
non-sense. If I use a relational database, it's because it is a
*relational* database. If you want an OODB, then we have the ZODB,
Durus and a couple others.

It is sometimes convenient to store objects in mature relational
database backend (reliability, stability, support, tools,
replication, etc.). See latst efforts with RelStorage backend
for ZODB (http://wiki.zope.org/ZODB/RelStorage) - it stores
pickled Python objects in Oracle, PostgreSQL or MySQL)
You mean a SQL database backend here - the conveniences that you
mention have nothing to do with being relational or not. And that's my
point: pickling objects, you loose of the convenience of a
*relational* database.
Jun 27 '08 #11

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

7
by: Rolf Hemmerling | last post by:
Hello ! Beginner's question: What ist the easiest way to store and save objects in a file generated by a C++ program, by using the "standard C++ library" and/or "Standard Template Library (...
1
by: ViperDK \(Daniel K.\) | last post by:
i've a database where relations are hold in a special way which the project leaders think of as "performant and uncomplicated" but which is very questionable to me:...
2
by: Tom | last post by:
Hi, I'm designing a website where all the articles/news will be stored in XML. (I will use PHP to parse the XML.) Should I store the XML in separate .xml files or should I store the XML in a...
38
by: Radi Radichev | last post by:
Hi! I'm making a database application and i heard from a friend that it is more proffecional and easy to do this with bussines objects. Can anyone tell me where i can find more info on bussines...
6
by: Mudcat | last post by:
Hi, I am trying to build a tool that analyzes stock data. Therefore I am going to download and store quite a vast amount of it. Just for a general number - assuming there are about 7000 listed...
2
by: nacho222 | last post by:
I'm currently in the middle of writing a persistence framework, and I have to make a design decission. The framework will take care of all the saving and restoring objects, and also the...
13
by: Rhino | last post by:
Is it possible to store Java objects in DB2 V8.2 for Windows/Unix/Linux via JDBC? Specifically, if I have a 4-dimensional boolean array, i.e. boolean, can I store it directly in a column of a...
25
by: Penelope Dramas | last post by:
Hello, I'm in a front of very serious .net redesign/rewrite of an old VB6 application. I had been asked to make it .NET 2.0 and would like to ask couple of questions regarding data access as...
6
by: Lint Radley | last post by:
Hi Everyone, I need an opinion here on storing data for a program I am working on the processes DICOM images. Essentially, my program stores 25-45 (it varies depending on the user) ranges of...
2
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 7 Feb 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:30 (7.30PM). In this month's session, the creator of the excellent VBE...
0
by: MeoLessi9 | last post by:
I have VirtualBox installed on Windows 11 and now I would like to install Kali on a virtual machine. However, on the official website, I see two options: "Installer images" and "Virtual machines"....
0
by: DolphinDB | last post by:
The formulas of 101 quantitative trading alphas used by WorldQuant were presented in the paper 101 Formulaic Alphas. However, some formulas are complex, leading to challenges in calculation. Take...
0
by: DolphinDB | last post by:
Tired of spending countless mintues downsampling your data? Look no further! In this article, you’ll learn how to efficiently downsample 6.48 billion high-frequency records to 61 million...
0
by: Aftab Ahmad | last post by:
Hello Experts! I have written a code in MS Access for a cmd called "WhatsApp Message" to open WhatsApp using that very code but the problem is that it gives a popup message everytime I clicked on...
0
by: Aftab Ahmad | last post by:
So, I have written a code for a cmd called "Send WhatsApp Message" to open and send WhatsApp messaage. The code is given below. Dim IE As Object Set IE =...
0
by: ryjfgjl | last post by:
ExcelToDatabase: batch import excel into database automatically...
0
by: Vimpel783 | last post by:
Hello! Guys, I found this code on the Internet, but I need to modify it a little. It works well, the problem is this: Data is sent from only one cell, in this case B5, but it is necessary that data...
0
by: ArrayDB | last post by:
The error message I've encountered is; ERROR:root:Error generating model response: exception: access violation writing 0x0000000000005140, which seems to be indicative of an access violation...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.