473,696 Members | 1,722 Online
Bytes | Software Development & Data Engineering Community
+ Post

Home Posts Topics Members FAQ

Storing objects in relational database

I started playing with python a few weeks ago after a number of years
of perl programming and I can say that my first impression is,
unsurprisingly, quite positive. ;)
The reason I am writing here is that I can't seem to figure out how to
save/restore python objects into a relational database. The way I
used to do it in perl was to 'freeze' the object before storing it
into the database and 'thaw' it before restoring it. (For those not
familiar with the perl terminology freeze and thaw are method from
perl's persistence module Storable). I found that the python's
corresponding module is Pickle but it doesn't seem to work for me.. I
do roughly the following:
class TestA:
def insert():
i = TestA('asdf')
output = cStringIO.Strin gIO()
cPickle.dump(i, output, 2)
print "output.getvalu e(): %s" % output.getvalue ()
jd = libpq.PgQuoteBy tea(output.getv alue())
print "jd %s" % jd
return dbpool.runOpera tion("insert into jobs (effective_job_ id,
job_description ) values (1, " + jd + ")")
Jun 27 '08 #1
10 3045
and then I try to restore the object with the following code

def success(rv):
print "success"
str = cStringIO.Strin gIO(libpq.PgUnQ uoteBytea(rv[0][0]))
i = cPickle.load(st r)
i.toString()

the execution fails just after the print statement, and I am not quite
sure why is that.

I would love to find out what people are using when they need to do
something similar --
perhaps I am trying to do it the perl way, while there is an elegant
python solution.

thanks
Jun 27 '08 #2
On 23 mai, 23:14, nayden <nay...@gmail.c omwrote:
and then I try to restore the object with the following code

def success(rv):
print "success"
str = cStringIO.Strin gIO(libpq.PgUnQ uoteBytea(rv[0][0]))
i = cPickle.load(st r)
i.toString()

the execution fails just after the print statement, and I am not quite
sure why is that.
Please reread the doc for pickle.dump, pickle.dumps, pickle.load and
pickle.loads. You just don't need StringIO here, just use the 's
versions of the functions.
I would love to find out what people are using when they need to do
something similar
perhaps I am trying to do it the perl way, while there is an elegant
python solution.
I don't know if you'd label it 'elegant', but as far as I'm concerned,
storing serialized objects as blobs in a relational database is mostly
non-sense. If I use a relational database, it's because it is a
*relational* database. If you want an OODB, then we have the ZODB,
Durus and a couple others.
Jun 27 '08 #3
On May 24, 7:14 am, nayden <nay...@gmail.c omwrote:
the execution fails just after the print statement, and I am not quite
sure why is that.
It's often helpful to include the traceback, or at the very least the
last 3-4 lines of it, as it helps everyone work out the issue you're
having.

If you're not sure which line in a function is causing the issue, try
commenting out all but the first, run-and-test, re-add the next, run-
and-test etc

But the error is most likely this line:
i = cPickle.load(st r)
cPickle.load unpickles from a file, but here you're handing it a
string. You want cPickle.loads.

At the interpreter, you can always quickly check these out by looking
up the docstring via 'help(cPickle.l oads)' (or 'cPickle.loads? ' if
you're using iPythhon).

- alex23
Jun 27 '08 #4
On Fri, May 23, 2008 at 5:07 PM, nayden <na****@gmail.c omwrote:
The reason I am writing here is that I can't seem to figure out how to
save/restore python objects into a relational database.
Here's a basic version using the sqlite bindings included with Python 2.5:

import sqlite3, pickle

thing = {'date': '2008-05-21',
'event': 'Something happened this day'}

conn = sqlite3.connect (':memory:')
c = conn.cursor()
c.execute('''cr eate table stuff (idx integer, data text)''')
c.execute('''in sert into stuff values (?,?)''', (1, pickle.dumps(th ing)))
c.execute('''se lect data from stuff where idx=1''')
row = c.fetchone()
# sqlite3 stores text fields as unicode, so we need to encode to ascii
# before we unpickle
pickle_str = row[0].encode('ascii' )
thing2 = pickle.loads(pi ckle_str)

print thing
print thing2

Maybe that will set you on the right track.

--
Jerry
Jun 27 '08 #5
On May 23, 7:00*pm, alex23 <wuwe...@gmail. comwrote:
On May 24, 7:14 am, nayden <nay...@gmail.c omwrote:
the execution fails just after the print statement, and I am not quite
sure why is that.

It's often helpful to include the traceback, or at the very least the
last 3-4 lines of it, as it helps everyone work out the issue you're
having.

If you're not sure which line in a function is causing the issue, try
commenting out all but the first, run-and-test, re-add the next, run-
and-test etc

But the error is most likely this line:
* * i = cPickle.load(st r)

cPickle.load unpickles from a file, but here you're handing it a
string. You want cPickle.loads.

At the interpreter, you can always quickly check these out by looking
up the docstring via 'help(cPickle.l oads)' (or 'cPickle.loads? ' if
you're using iPythhon).

- alex23
It's not a string it's a cStringIO.Strin gIO, even though his variable
name is confusing.

nayden: 'str' is a built-in variable that is the string type. Try this
for different values of x:
type(x) is str
str(x)
When you override it, it may be confusing down the line.

I'd suggest installing pychecker, which will help you catch errors
like this:
http://pychecker.sourceforge.net/
Jun 27 '08 #6
"br************ *****@gmail.com " <br************ *****@gmail.com writes:

I don't know if you'd label it 'elegant', but as far as I'm
concerned, storing serialized objects as blobs in a relational
database is mostly non-sense. If I use a relational database, it's
because it is a *relational* database. If you want an OODB, then we
have the ZODB, Durus and a couple others.
.... not to forget object-relational mappers like SQLAlchemy, SQLObject...
Jun 27 '08 #7
br************* ****@gmail.com pisze:
I don't know if you'd label it 'elegant', but as far as I'm concerned,
storing serialized objects as blobs in a relational database is mostly
non-sense. If I use a relational database, it's because it is a
*relational* database. If you want an OODB, then we have the ZODB,
Durus and a couple others.
It is sometimes convenient to store objects in mature relational
database backend (reliability, stability, support, tools,
replication, etc.). See latst efforts with RelStorage backend
for ZODB (http://wiki.zope.org/ZODB/RelStorage) - it stores
pickled Python objects in Oracle, PostgreSQL or MySQL)
Jun 27 '08 #8
Ville M. Vainio a écrit :
"br************ *****@gmail.com " <br************ *****@gmail.com writes:

>I don't know if you'd label it 'elegant', but as far as I'm
concerned, storing serialized objects as blobs in a relational
database is mostly non-sense. If I use a relational database, it's
because it is a *relational* database. If you want an OODB, then we
have the ZODB, Durus and a couple others.

... not to forget object-relational mappers like SQLAlchemy, SQLObject...
Which are more IMHO another way to "bridge" RBDMS with the programming
language than a way to persist objects.
Jun 27 '08 #9
Thanks everyone for the helpful suggestions.
Jun 27 '08 #10

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

7
11512
by: Rolf Hemmerling | last post by:
Hello ! Beginner's question: What ist the easiest way to store and save objects in a file generated by a C++ program, by using the "standard C++ library" and/or "Standard Template Library ( STL )" ? So I would like to generate some objects ( of different classes ) with a C++ program and would like to make it permanent / persistent, so that
1
1658
by: ViperDK \(Daniel K.\) | last post by:
i've a database where relations are hold in a special way which the project leaders think of as "performant and uncomplicated" but which is very questionable to me: ------------------------------------------------ Table Fields and Table Fields object stuff.... and (varchar with 0-20 ids usually)
2
1564
by: Tom | last post by:
Hi, I'm designing a website where all the articles/news will be stored in XML. (I will use PHP to parse the XML.) Should I store the XML in separate .xml files or should I store the XML in a MySQL database? Which would be faster? Best regards, Tom
38
3720
by: Radi Radichev | last post by:
Hi! I'm making a database application and i heard from a friend that it is more proffecional and easy to do this with bussines objects. Can anyone tell me where i can find more info on bussines objects and how to implement them with c#? I would appreciate any help. Thanks.
6
2495
by: Mudcat | last post by:
Hi, I am trying to build a tool that analyzes stock data. Therefore I am going to download and store quite a vast amount of it. Just for a general number - assuming there are about 7000 listed stocks on the two major markets plus some extras, 255 tradying days a year for 20 years, that is about 36 million entries. Obviously a database is a logical choice for that. However I've never used one, nor do I know what benefits I would get...
2
1846
by: nacho222 | last post by:
I'm currently in the middle of writing a persistence framework, and I have to make a design decission. The framework will take care of all the saving and restoring objects, and also the relationships between them. Obviously, I need a way to identify objects uniquely. Now, one way would be to use an autogenerated id from a autoincremental field in the DB. But the problem is that I need to know the ID of the object as soon as it is created,...
13
3687
by: Rhino | last post by:
Is it possible to store Java objects in DB2 V8.2 for Windows/Unix/Linux via JDBC? Specifically, if I have a 4-dimensional boolean array, i.e. boolean, can I store it directly in a column of a DB2 table? If so, how do I do it? It would be VERY convenient if I could store this boolean array directly in a column of a DB2 table but I'm not at all clear on whether this is possible, even after reading the documentation in the Information...
25
2770
by: Penelope Dramas | last post by:
Hello, I'm in a front of very serious .net redesign/rewrite of an old VB6 application. I had been asked to make it .NET 2.0 and would like to ask couple of questions regarding data access as this application is heavily data-centric around MSDE database. Would it be better to use custom business objects or extend
6
1296
by: Lint Radley | last post by:
Hi Everyone, I need an opinion here on storing data for a program I am working on the processes DICOM images. Essentially, my program stores 25-45 (it varies depending on the user) ranges of pixel values to search the image for. Currently, I am using a .MDF database that requires SQL Express to be installed. For only 25-45 records it seems like overkill to me. That being said, I love the ability to easily update records, delete, add,...
0
8666
marktang
by: marktang | last post by:
ONU (Optical Network Unit) is one of the key components for providing high-speed Internet services. Its primary function is to act as an endpoint device located at the user's premises. However, people are often confused as to whether an ONU can Work As a Router. In this blog post, we’ll explore What is ONU, What Is Router, ONU & Router’s main usage, and What is the difference between ONU and Router. Let’s take a closer look ! Part I. Meaning of...
0
9010
jinu1996
by: jinu1996 | last post by:
In today's digital age, having a compelling online presence is paramount for businesses aiming to thrive in a competitive landscape. At the heart of this digital strategy lies an intricately woven tapestry of website design and digital marketing. It's not merely about having a website; it's about crafting an immersive digital experience that captivates audiences and drives business growth. The Art of Business Website Design Your website is...
0
8853
tracyyun
by: tracyyun | last post by:
Dear forum friends, With the development of smart home technology, a variety of wireless communication protocols have appeared on the market, such as Zigbee, Z-Wave, Wi-Fi, Bluetooth, etc. Each protocol has its own unique characteristics and advantages, but as a user who is planning to build a smart home system, I am a bit confused by the choice of these technologies. I'm particularly interested in Zigbee because I've heard it does some...
0
7703
agi2029
by: agi2029 | last post by:
Let's talk about the concept of autonomous AI software engineers and no-code agents. These AIs are designed to manage the entire lifecycle of a software development project—planning, coding, testing, and deployment—without human intervention. Imagine an AI that can take a project description, break it down, write the code, debug it, and then launch it, all on its own.... Now, this would greatly impact the work of software developers. The idea...
0
5857
by: conductexam | last post by:
I have .net C# application in which I am extracting data from word file and save it in database particularly. To store word all data as it is I am converting the whole word file firstly in HTML and then checking html paragraph one by one. At the time of converting from word file to html my equations which are in the word document file was convert into image. Globals.ThisAddIn.Application.ActiveDocument.Select();...
0
4356
by: TSSRALBI | last post by:
Hello I'm a network technician in training and I need your help. I am currently learning how to create and manage the different types of VPNs and I have a question about LAN-to-LAN VPNs. The last exercise I practiced was to create a LAN-to-LAN VPN between two Pfsense firewalls, by using IPSEC protocols. I succeeded, with both firewalls in the same network. But I'm wondering if it's possible to do the same thing, with 2 Pfsense firewalls...
0
4611
by: adsilva | last post by:
A Windows Forms form does not have the event Unload, like VB6. What one acts like?
1
3033
by: 6302768590 | last post by:
Hai team i want code for transfer the data from one system to another through IP address by using C# our system has to for every 5mins then we have to update the data what the data is updated we have to send another system
2
2319
muto222
by: muto222 | last post by:
How can i add a mobile payment intergratation into php mysql website.

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.