473,387 Members | 1,574 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,387 software developers and data experts.

disk based dictionaries


Hi,

I want to store dictionaries on disk. I had a look at a few modules
like bsddb, shelve etc. However would it be possible for me to do the
following

hash[1] = [1, 2, 3] where the key is an int and not a string

bsddb requires that both the key,value are string.
shelve does support values being object but not the keys. Is there any
module which support keys which are not strings

Also how do i use disk based hashes for multidimensional hashes such as
below

#!/usr/bin/python

dict={}
dict['key1'] = {}
dict[('key1')][('key2')] = 'value'

key1=dict['key1']
print key1['key2']

I have read of mxBeeDict but was unable to get it work properly. I am
not sure if it supports what i need as i was unable to get any
documentation about it. Is the module used widely ?

Below is how i am using the module

bdict = BeeDict('/tmp/beedict')

bdict[1] = 1
print bdict.keys()

bdict.commit()
bdict.close()

bdict1 = BeeDict('/tmp/beedict')
print bdict1.keys()
print bdict1.values()
Would it be that using disk based dictionaries once opened are as fast
as in memory dictionaries ?

Thanks in advance,

Best Regards,
Shivram U


Confidentiality Notice

The information contained in this electronic message and any attachments to this message are intended
for the exclusive use of the addressee(s) and may contain confidential or privileged information. If
you are not the intended recipient, please notify the sender at Wipro or Ma*******@wipro.com immediately
and destroy all copies of this message and any attachments.
Jul 18 '05 #1
3 2357
You may also want to take a look at ZODB (Zope database).
It handles the pickling, storage and retrieval of all
Python objects (including dictionaries) very well. And yes
you can use ZODB without using Zope proper.

http://www.zope.org/Products/StandaloneZODB

http://zope.org/Members/adytumsoluti...LoveZODB_PartI

http://www.h7.dion.ne.jp/~harm/ZODB-Tutorial.py
Larry Bates
Shivram U wrote:
Hi,

I want to store dictionaries on disk. I had a look at a few modules
like bsddb, shelve etc. However would it be possible for me to do the
following

hash[1] = [1, 2, 3] where the key is an int and not a string

bsddb requires that both the key,value are string.
shelve does support values being object but not the keys. Is there any
module which support keys which are not strings

Also how do i use disk based hashes for multidimensional hashes such as
below

#!/usr/bin/python

dict={}
dict['key1'] = {}
dict[('key1')][('key2')] = 'value'

key1=dict['key1']
print key1['key2']

I have read of mxBeeDict but was unable to get it work properly. I am
not sure if it supports what i need as i was unable to get any
documentation about it. Is the module used widely ?

Below is how i am using the module

bdict = BeeDict('/tmp/beedict')

bdict[1] = 1
print bdict.keys()

bdict.commit()
bdict.close()

bdict1 = BeeDict('/tmp/beedict')
print bdict1.keys()
print bdict1.values()
Would it be that using disk based dictionaries once opened are as fast
as in memory dictionaries ?

Thanks in advance,

Best Regards,
Shivram U


Confidentiality Notice

The information contained in this electronic message and any attachments to this message are intended
for the exclusive use of the addressee(s) and may contain confidential or privileged information. If
you are not the intended recipient, please notify the sender at Wipro or Ma*******@wipro.com immediately
and destroy all copies of this message and any attachments.

Jul 18 '05 #2
You may also want to take a look at ZODB (Zope database).
It handles the pickling, storage and retrieval of all
Python objects (including dictionaries) very well. And yes
you can use ZODB without using Zope proper.

http://www.zope.org/Products/StandaloneZODB

http://zope.org/Members/adytumsoluti...LoveZODB_PartI

http://www.h7.dion.ne.jp/~harm/ZODB-Tutorial.py
Larry Bates
Shivram U wrote:
Hi,

I want to store dictionaries on disk. I had a look at a few modules
like bsddb, shelve etc. However would it be possible for me to do the
following

hash[1] = [1, 2, 3] where the key is an int and not a string

bsddb requires that both the key,value are string.
shelve does support values being object but not the keys. Is there any
module which support keys which are not strings

Also how do i use disk based hashes for multidimensional hashes such as
below

#!/usr/bin/python

dict={}
dict['key1'] = {}
dict[('key1')][('key2')] = 'value'

key1=dict['key1']
print key1['key2']

I have read of mxBeeDict but was unable to get it work properly. I am
not sure if it supports what i need as i was unable to get any
documentation about it. Is the module used widely ?

Below is how i am using the module

bdict = BeeDict('/tmp/beedict')

bdict[1] = 1
print bdict.keys()

bdict.commit()
bdict.close()

bdict1 = BeeDict('/tmp/beedict')
print bdict1.keys()
print bdict1.values()
Would it be that using disk based dictionaries once opened are as fast
as in memory dictionaries ?

Thanks in advance,

Best Regards,
Shivram U


Confidentiality Notice

The information contained in this electronic message and any attachments to this message are intended
for the exclusive use of the addressee(s) and may contain confidential or privileged information. If
you are not the intended recipient, please notify the sender at Wipro or Ma*******@wipro.com immediately
and destroy all copies of this message and any attachments.

Jul 18 '05 #3
I'd like to second this suggestion. While there are a few things you
need to be aware of when writing your code (mostly taken care of in
the latest release) it's a mostly trivial code change. (For me it was
replacing a few dictionaries with PersistentMap objects and changing
the base class of a few objects to Persistant from object.

FWIW, I'm using ZODB to help track EDI transactions for a help desk
application. Right now my database hovers in the 100MB range with
several ten of thousands of objects. I also use it for single object
temp storage, so I feel it works well from both the small and mid-size
scale. (It probably works fine for large projects as well, I just
don't have one right now...)

Chris
On Thu, 02 Dec 2004 18:53:44 -0600, Larry Bates <lb****@syscononline.com> wrote:
You may also want to take a look at ZODB (Zope database).
It handles the pickling, storage and retrieval of all
Python objects (including dictionaries) very well. And yes
you can use ZODB without using Zope proper.

http://www.zope.org/Products/StandaloneZODB

http://zope.org/Members/adytumsoluti...LoveZODB_PartI

http://www.h7.dion.ne.jp/~harm/ZODB-Tutorial.py
Larry Bates


Shivram U wrote:
Hi,

I want to store dictionaries on disk. I had a look at a few modules
like bsddb, shelve etc. However would it be possible for me to do the
following

hash[1] = [1, 2, 3] where the key is an int and not a string

bsddb requires that both the key,value are string.
shelve does support values being object but not the keys. Is there any
module which support keys which are not strings

Also how do i use disk based hashes for multidimensional hashes such as
below

#!/usr/bin/python

dict={}
dict['key1'] = {}
dict[('key1')][('key2')] = 'value'

key1=dict['key1']
print key1['key2']

I have read of mxBeeDict but was unable to get it work properly. I am
not sure if it supports what i need as i was unable to get any
documentation about it. Is the module used widely ?

Below is how i am using the module

bdict = BeeDict('/tmp/beedict')

bdict[1] = 1
print bdict.keys()

bdict.commit()
bdict.close()

bdict1 = BeeDict('/tmp/beedict')
print bdict1.keys()
print bdict1.values()
Would it be that using disk based dictionaries once opened are as fast
as in memory dictionaries ?

Thanks in advance,

Best Regards,
Shivram U


Confidentiality Notice

The information contained in this electronic message and any attachments to this message are intended
for the exclusive use of the addressee(s) and may contain confidential or privileged information. If
you are not the intended recipient, please notify the sender at Wipro or Ma*******@wipro.com immediately
and destroy all copies of this message and any attachments.

--
http://mail.python.org/mailman/listinfo/python-list

--
"It is our responsibilities, not ourselves, that we should take
seriously." -- Peter Ustinov
Jul 18 '05 #4

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

6
by: Chris | last post by:
I have a set of routines, the first of which reads lots and lots of data from disparate regions of disk. This read routine takes 40 minutes on a P3-866 (with IDE drives). This routine populates...
8
by: robin | last post by:
I need to do a search through about 50 million records, each of which are less than 100 bytes wide. A database is actually too slow for this, so I thought of optimising the data and putting it all...
8
by: Foodbank | last post by:
Hi, Has anyone ever hashed a text file to disk before? I'm trying to convert an in-memory hash to disk hash and I can't find any relevant information to help me. Also, I'd like to use lseek,...
3
by: Bruce | last post by:
I am building a WinForms app that uses Web Services access to a server for most of its data input/output, but I also need to persist some of its data to the local disk (basically as a cache of some...
57
by: Chris Foote | last post by:
Hi all. I have the need to store a large (10M) number of keys in a hash table, based on a tuple of (long_integer, integer). The standard python dictionary works well for small numbers of keys,...
1
by: dopey483 | last post by:
I am manipulating lots of log files (about 500,000 files and about 30Gb in total) to get them into a little SQL db. Part of this process is "normalisation" and creating tables of common data. I am...
1
by: qouify | last post by:
Hello, here is my problem. I have a dictionnary of words (e.g., type char) stored sequentially (but not sorted) in a file F and I want to insert in it some words stored in memory in a balanced...
11
by: GG | last post by:
Anybody knows of any collection where is not stored in memory but using hard disk instead? Thanks *** Sent via Developersdex http://www.developersdex.com ***
0
by: M.-A. Lemburg | last post by:
On 2008-07-31 02:29, python@bdurham.com wrote: If you don't have a problem with taking a small performance hit, then I'd suggest to have a look at mxBeeBase, which is an on-disk dictionary...
0
by: Charles Arthur | last post by:
How do i turn on java script on a villaon, callus and itel keypad mobile phone
0
by: ryjfgjl | last post by:
If we have dozens or hundreds of excel to import into the database, if we use the excel import function provided by database editors such as navicat, it will be extremely tedious and time-consuming...
0
by: ryjfgjl | last post by:
In our work, we often receive Excel tables with data in the same format. If we want to analyze these data, it can be difficult to analyze them because the data is spread across multiple Excel files...
0
by: emmanuelkatto | last post by:
Hi All, I am Emmanuel katto from Uganda. I want to ask what challenges you've faced while migrating a website to cloud. Please let me know. Thanks! Emmanuel
0
BarryA
by: BarryA | last post by:
What are the essential steps and strategies outlined in the Data Structures and Algorithms (DSA) roadmap for aspiring data scientists? How can individuals effectively utilize this roadmap to progress...
1
by: nemocccc | last post by:
hello, everyone, I want to develop a software for my android phone for daily needs, any suggestions?
1
by: Sonnysonu | last post by:
This is the data of csv file 1 2 3 1 2 3 1 2 3 1 2 3 2 3 2 3 3 the lengths should be different i have to store the data by column-wise with in the specific length. suppose the i have to...
0
by: Hystou | last post by:
Most computers default to English, but sometimes we require a different language, especially when relocating. Forgot to request a specific language before your computer shipped? No problem! You can...
0
jinu1996
by: jinu1996 | last post by:
In today's digital age, having a compelling online presence is paramount for businesses aiming to thrive in a competitive landscape. At the heart of this digital strategy lies an intricately woven...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.