By using this site, you agree to our updated Privacy Policy and our Terms of Use. Manage your Cookies Settings.
443,907 Members | 2,039 Online
Bytes IT Community
+ Ask a Question
Need help? Post your question and get tips & solutions from a community of 443,907 IT Pros & Developers. It's quick & easy.

DBM scalability

P: n/a
I'm trying to create a dbm database with around 4.5 million entries but the existing dbm modules
(dbhash, gdbm) don't seem to cut it. What happens is that the more entries are added, the more time
per new entry is required, so the complexity seems to be much worse than linear. Is this to be
expected and if so, should I expect better performance (i.e. linear or almost linear) from a real
database, e.g. sqlite ?

George
Oct 21 '05 #1
Share this Question
Share on Google+
2 Replies


P: n/a
"George Sakkis" <gs*****@rutgers.edu> writes:
I'm trying to create a dbm database with around 4.5 million entries
but the existing dbm modules (dbhash, gdbm) don't seem to cut
it. What happens is that the more entries are added, the more time
per new entry is required, so the complexity seems to be much worse
than linear. Is this to be expected


No, not expected. See if you're using something like db.keys() which
tries to read all the keys from the db into memory, or anything like that.
Oct 21 '05 #2

P: n/a
"Paul Rubin" <http://ph****@NOSPAM.invalid> wrote:
"George Sakkis" <gs*****@rutgers.edu> writes:
I'm trying to create a dbm database with around 4.5 million entries
but the existing dbm modules (dbhash, gdbm) don't seem to cut
it. What happens is that the more entries are added, the more time
per new entry is required, so the complexity seems to be much worse
than linear. Is this to be expected


No, not expected. See if you're using something like db.keys() which
tries to read all the keys from the db into memory, or anything like that.


It turns out it doesn't have to do with python or the dbm modules. The same program on a different
box and platform runs linearly, so I guess it has to do with the OS and/or the hard disk
configuration.

George
Oct 22 '05 #3

This discussion thread is closed

Replies have been disabled for this discussion.