473,385 Members | 1,645 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,385 software developers and data experts.

ZODB: single database, multiple connections

Hello all

I am using Python 2.3 and ZODB (without the rest of Zope) with the
following pattern:

* One process which writes stuff to a ZODB instance (call it test.db)
* Another process which reads stuff from the above ZODB instance
test.db

What I find is that when the first process writes, the second doesn't
pick up the changes. I am sure this must be because I am using ZODB
wrongly, but I can't find any examples that show how to use ZODB in
this way, and I can't find any API docs for FileStorage, Connection,
etc. Reading the source code (from C:\python23\lib\site-packages) has
not thrown up anything useful.

Here's my test code:

A simple database class:

class Database(object):
PersistentObject = persistent.Persistent
PersistentDict = BTrees.OOBTree.OOBTree

def __init__(self, filename, read_only = False):
self.storage = FileStorage.FileStorage(filename, read_only =
read_only)
self.db = DB(self.storage)
self.connection = self.db.open()
self.dbroot = self.connection.root()
Write:
db = Database("test.db")
db.data = db.get_dictionary('data')

sz = len(db.data.keys())

class Datum(Persistent):
def __init__(self, value):
self.value = value

if __name__ == '__main__':
# insert 10 things
for i in range(0, 10):
val = i + sz
d = Datum(val)
db.data[val] = d
transaction.commit()

Read:

db = Database("test.db", read_only = True)

data = db.get_dictionary('data')

If I have a Python shell open and run the above two lines, if I run the
write process repeatedly, the above "data" object never contains any of
the newly added items. To pick them up I have to totally recreate the
"db" object.

I must be doing something wrongly, but I can't figure out what.

Any suggestions?

Thanks,

PC

Oct 30 '06 #1
2 2853
[Petra Chong]
I am using Python 2.3 and ZODB (without the rest of Zope) with the
following pattern:

* One process which writes stuff to a ZODB instance (call it test.db)
* Another process which reads stuff from the above ZODB instance
test.db

What I find is that when the first process writes, the second doesn't
pick up the changes. I am sure this must be because I am using ZODB
wrongly, but I can't find any examples that show how to use ZODB in
this way, and I can't find any API docs for FileStorage, Connection,
etc. Reading the source code (from C:\python23\lib\site-packages) has
not thrown up anything useful.

Here's my test code:

A simple database class:

...

Write:

...

Read:

db = Database("test.db", read_only = True)

data = db.get_dictionary('data')

If I have a Python shell open and run the above two lines, if I run the
write process repeatedly, the above "data" object never contains any of
the newly added items. To pick them up I have to totally recreate the
"db" object.
You say that like it's hard to do ;-)

It's a decent way to proceed. ZODB is a database, and has
transactional semantics: you never see new object state on the read
side because you're /in/ a transaction, and a transaction guarantees
to give you a consistent view of the data. The view would be
inconsistent if it showed you state committed by different
transactions on the write side while you're still in the same
transaction on the read side.
I must be doing something wrongly, but I can't figure out what.
Seems to be a conceptual problem more than anything else.
Any suggestions?
You already know that tossing your connection and opening a new
connection will give you a newer view of the database, and it's
unclear why you don't think that's good enough. Other approaches
amount to telling ZODB (on the read side) that you're done with the
current transaction. For example, try doing

transaction.abort()

on the read side when you're ready to see newer object state.

BTW, a better place to ask about ZODB is the zodb-dev list:

http://mail.zope.org/mailman/listinfo/zodb-dev

It's not just for developers /of/ ZODB. Note that you need to
subscribe to it in order to post to it (that's a heavyweight anti-spam
gimmick common to all Zope lists).
Oct 30 '06 #2
If I have a Python shell open and run the above two lines, if I run the
write process repeatedly, the above "data" object never contains any of
the newly added items. To pick them up I have to totally recreate the
"db" object.

You say that like it's hard to do ;-)
It isn't, but this was the problem:

It took 15 seconds to open the database, so I thought that I shouldn't
be recreating the database and should be refreshing it. There is
nothing in the docs that says that the right behaviour is to recreate
it.

However, I then put some debug statements around it (I know, I know, I
could have used profiling) to find out exactly what was taking 15
seconds. Turned out there was some rubbish in the db that shouldn't
have been there. I got rid of it, and now it doesn't take 15 seconds to
recreate the database.

So- I am now recreating the db, and my problem is solved.
>
BTW, a better place to ask about ZODB is the zodb-dev list:

http://mail.zope.org/mailman/listinfo/zodb-dev

It's not just for developers /of/ ZODB. Note that you need to
subscribe to it in order to post to it (that's a heavyweight anti-spam
gimmick common to all Zope lists).
Thanks for that- it's been hard to track down information on ZODB.

Regards,

PC

Oct 30 '06 #3

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

16
by: noah | last post by:
Does PHP have a feature to associate Cookie sessions with a persistent database connection that will allow a single transaction across multiple HTTP requests? Here is how I imagine my process: I...
49
by: Paul Rubin | last post by:
I've started a few threads before on object persistence in medium to high end server apps. This one is about low end apps, for example, a simple cgi on a personal web site that might get a dozen...
1
by: DJTB | last post by:
zodb-dev@zope.org] Hi, I'm having problems storing large amounts of objects in a ZODB. After committing changes to the database, elements are not cleared from memory. Since the number of...
10
by: ls | last post by:
Hi All, I looking for help with ZODB data export to text file. I have file Data.fs (file becomes from Plone CMS) and I have to display content structure, data like intro, body of article, etc...
9
by: Abhishek Srivastava | last post by:
Hello All, In IIS 6.0 We have a concept of worker processes and application pools. As I understand it, we can have multiple worker process per appliction pool. Each worker process is dedicated...
20
by: Keith G. Murphy | last post by:
I'm trying to get a feel for what most people are doing or consider best practice. Given a mod_perl application talking to a PostgreSQL database on the same host, where different users are...
35
by: keerthyragavendran | last post by:
hi i'm downloading a single file using multiple threads... how can i specify a particular range of bytes alone from a single large file... for example say if i need only bytes ranging from...
4
by: Andrus | last post by:
For winforms application with multiple related forms it is reasonable to create Linq database context object in start of application. Context object is released only when application exits. So...
3
by: Jean-Paul Calderone | last post by:
On Thu, 14 Aug 2008 16:15:11 +0200, "Diez B. Roggisch" <deets@nospam.web.dewrote: ZODB has transactions. That's probably the best way to make this code safe for concurrent use. A threading.Lock...
1
by: CloudSolutions | last post by:
Introduction: For many beginners and individual users, requiring a credit card and email registration may pose a barrier when starting to use cloud servers. However, some cloud server providers now...
0
by: Faith0G | last post by:
I am starting a new it consulting business and it's been a while since I setup a new website. Is wordpress still the best web based software for hosting a 5 page website? The webpages will be...
0
by: ryjfgjl | last post by:
In our work, we often need to import Excel data into databases (such as MySQL, SQL Server, Oracle) for data analysis and processing. Usually, we use database tools like Navicat or the Excel import...
0
by: taylorcarr | last post by:
A Canon printer is a smart device known for being advanced, efficient, and reliable. It is designed for home, office, and hybrid workspace use and can also be used for a variety of purposes. However,...
0
by: Charles Arthur | last post by:
How do i turn on java script on a villaon, callus and itel keypad mobile phone
0
BarryA
by: BarryA | last post by:
What are the essential steps and strategies outlined in the Data Structures and Algorithms (DSA) roadmap for aspiring data scientists? How can individuals effectively utilize this roadmap to progress...
1
by: nemocccc | last post by:
hello, everyone, I want to develop a software for my android phone for daily needs, any suggestions?
1
by: Sonnysonu | last post by:
This is the data of csv file 1 2 3 1 2 3 1 2 3 1 2 3 2 3 2 3 3 the lengths should be different i have to store the data by column-wise with in the specific length. suppose the i have to...
0
by: Hystou | last post by:
There are some requirements for setting up RAID: 1. The motherboard and BIOS support RAID configuration. 2. The motherboard has 2 or more available SATA protocol SSD/HDD slots (including MSATA, M.2...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.