I am trying to make some of the in-memory dictionaries in my code persistent using bsddb 4.3. Here is an example dictionary in my code for which I would like to create a persistent version: -
# Example dictionary
-
patterns = {pat1: {t1:None},
-
pat2: {t1:None, t2:None},
-
pat3: {t2:None}}
-
# Each key is an instance of the class T
-
# Each value is a dictionary, where keys are instances of the class T, and values are None. I could have used set instead of dictionary here
-
-
# Sample instance data
-
t1 = T('John','age','35')
-
t2 = T('John','created','www.blogger.com')
-
pat1 = T('John','age',None)
-
pat2 = T('John',None,None)
-
pat3 = T('John','created',None)
-
-
class T(tuple):
-
def __new__(cls,*args):
-
a,b,c= tuple(args)
-
return tuple.__new__(cls,(a,b,c))
-
I plan to convert each key in the dictionary into a string using cPickle before I can store the key in the persistent hash.
I am looking for a way to store the values in the dictionary. Since each value is a dictionary, I am not sure how exactly to pickle it. Each value in the hash table is likely to get updated quite often; hence I need an efficient way to pickle and unpickle the values. Any ideas? Please help.
Thanks,
D.
5 2513
I am trying to make some of the in-memory dictionaries in my code persistent using bsddb 4.3. Here is an example dictionary in my code for which I would like to create a persistent version: -
# Example dictionary
-
patterns = {pat1: {t1:None},
-
pat2: {t1:None, t2:None},
-
pat3: {t2:None}}
-
# Each key is an instance of the class T
-
# Each value is a dictionary, where keys are instances of the class T, and values are None. I could have used set instead of dictionary here
-
-
# Sample instance data
-
t1 = T('John','age','35')
-
t2 = T('John','created','www.blogger.com')
-
pat1 = T('John','age',None)
-
pat2 = T('John',None,None)
-
pat3 = T('John','created',None)
-
-
class T(tuple):
-
def __new__(cls,*args):
-
a,b,c= tuple(args)
-
return tuple.__new__(cls,(a,b,c))
-
I plan to convert each key in the dictionary into a string using cPickle before I can store the key in the persistent hash.
I am looking for a way to store the values in the dictionary. Since each value is a dictionary, I am not sure how exactly to pickle it. Each value in the hash table is likely to get updated quite often; hence I need an efficient way to pickle and unpickle the values. Any ideas? Please help.
Thanks,
D.
I'd say go straight to the DB. I've posted dictionary to sql query converters in the Articles section.
bvdet 2,851
Expert Mod 2GB
I am trying to make some of the in-memory dictionaries in my code persistent using bsddb 4.3. Here is an example dictionary in my code for which I would like to create a persistent version: -
# Example dictionary
-
patterns = {pat1: {t1:None},
-
pat2: {t1:None, t2:None},
-
pat3: {t2:None}}
-
# Each key is an instance of the class T
-
# Each value is a dictionary, where keys are instances of the class T, and values are None. I could have used set instead of dictionary here
-
-
# Sample instance data
-
t1 = T('John','age','35')
-
t2 = T('John','created','www.blogger.com')
-
pat1 = T('John','age',None)
-
pat2 = T('John',None,None)
-
pat3 = T('John','created',None)
-
-
class T(tuple):
-
def __new__(cls,*args):
-
a,b,c= tuple(args)
-
return tuple.__new__(cls,(a,b,c))
-
I plan to convert each key in the dictionary into a string using cPickle before I can store the key in the persistent hash.
I am looking for a way to store the values in the dictionary. Since each value is a dictionary, I am not sure how exactly to pickle it. Each value in the hash table is likely to get updated quite often; hence I need an efficient way to pickle and unpickle the values. Any ideas? Please help.
Thanks,
D.
Have you tried this: - import cPickle
-
fn = r'your_file'
-
f = open(fn, "w")
-
cPickle.Pickler(f).dump(patterns)
-
f.close()
Unpickle test: - import cPickle
-
-
class T(tuple):
-
def __new__(cls,*args):
-
a,b,c= tuple(args)
-
return tuple.__new__(cls,(a,b,c))
-
-
if __name__ == '__main__':
-
-
fn = r'H:\TEMP\temsys\pickle_dict.txt'
-
f = open(fn, "r")
-
dd = cPickle.Unpickler(f).load()
-
f.close()
-
-
for key, value in dd.items():
-
print '%s = %s' % (key, value)
>>> ('John', None, None) = {('John', 'age', '35'): None, ('John', 'created', 'www.blogger.com'): None}
('John', 'created', None) = {('John', 'created', 'www.blogger.com'): None}
('John', 'age', None) = {('John', 'age', '35'): None}
>>> type(dd.keys()[0])
<class '__main__.T'>
>>> type(dd.values()[0])
<type 'dict'>
>>> print dd.values()[0]
{('John', 'age', '35'): None, ('John', 'created', 'www.blogger.com'): None}
>>>
I'd say go straight to the DB. I've posted dictionary to sql query converters in the Articles section.
Thanks, your response was useful!
Could you please point me to the article you wrote, I could not find it.
Have you tried this: - import cPickle
-
fn = r'your_file'
-
f = open(fn, "w")
-
cPickle.Pickler(f).dump(patterns)
-
f.close()
Unpickle test: - import cPickle
-
-
class T(tuple):
-
def __new__(cls,*args):
-
a,b,c= tuple(args)
-
return tuple.__new__(cls,(a,b,c))
-
-
if __name__ == '__main__':
-
-
fn = r'H:\TEMP\temsys\pickle_dict.txt'
-
f = open(fn, "r")
-
dd = cPickle.Unpickler(f).load()
-
f.close()
-
-
for key, value in dd.items():
-
print '%s = %s' % (key, value)
>>> ('John', None, None) = {('John', 'age', '35'): None, ('John', 'created', 'www.blogger.com'): None}
('John', 'created', None) = {('John', 'created', 'www.blogger.com'): None}
('John', 'age', None) = {('John', 'age', '35'): None}
>>> type(dd.keys()[0])
<class '__main__.T'>
>>> type(dd.values()[0])
<type 'dict'>
>>> print dd.values()[0]
{('John', 'age', '35'): None, ('John', 'created', 'www.blogger.com'): None}
>>>
Thanks for your response! I did try this out.
Thanks, your response was useful!
Could you please point me to the article you wrote, I could not find it.
They are helper functions in this post.
Sign in to post your reply or Sign up for a free account.
Similar topics
by: Michael Hohn |
last post by:
Hi,
under python 2.2, the pickle/unpickle sequence incorrectly restores
a larger data structure I have.
Under Python 2.3, these structures now give an explicit exception from...
|
by: A.B., Khalid |
last post by:
I wonder if someone can explain what is wrong here. I am pickling a
list of dictionaries (see code attached) and unpickling it back using
the HIGHEST_PROTOCOL of pickle and cPickle. I am getting an...
|
by: Mike P. |
last post by:
Hi all,
I'm working on a simulation (can be considered a game) in Python where I
want to be able to dump the simulation state to a file and be able to load
it up later. I have used the standard...
|
by: Jim Lewis |
last post by:
Pickling an instance of a class, gives "can't pickle instancemethod
objects". What does this mean? How do I find the class method creating
the problem?
|
by: crystalattice |
last post by:
I'm creating an RPG for experience and practice. I've finished a
character creation module and I'm trying to figure out how to get the
file I/O to work.
I've read through the python newsgroup...
|
by: Chris |
last post by:
Why can pickle serialize references to functions, but not methods?
Pickling a function serializes the function name, but pickling a
staticmethod, classmethod, or instancemethod generates an...
|
by: andrewfelch |
last post by:
Hello all,
I'm using the metaclass trick for automatic reloading of class member
functions, found at: http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/160164
My problem is that if I
1)...
|
by: fizilla |
last post by:
Hello all!
I have the following weird problem and since I am new to Python I somehow cannot figure out an elegant solution. The problem reduces to the following question:
How to pickle a...
|
by: IceMan85 |
last post by:
Hi to all, I have spent the whole morning trying, with no success to pickle an object that I have created.
The error that I get is : Can't pickle 'SRE_Match' object: <_sre.SRE_Match object at...
|
by: DolphinDB |
last post by:
Tired of spending countless mintues downsampling your data? Look no further!
In this article, you’ll learn how to efficiently downsample 6.48 billion high-frequency records to 61 million...
|
by: ryjfgjl |
last post by:
ExcelToDatabase: batch import excel into database automatically...
|
by: isladogs |
last post by:
The next Access Europe meeting will be on Wednesday 6 Mar 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:15 (7.15PM).
In this month's session, we are pleased to welcome back...
|
by: Vimpel783 |
last post by:
Hello!
Guys, I found this code on the Internet, but I need to modify it a little. It works well, the problem is this: Data is sent from only one cell, in this case B5, but it is necessary that data...
|
by: PapaRatzi |
last post by:
Hello,
I am teaching myself MS Access forms design and Visual Basic. I've created a table to capture a list of Top 30 singles and forms to capture new entries. The final step is a form (unbound)...
|
by: CloudSolutions |
last post by:
Introduction:
For many beginners and individual users, requiring a credit card and email registration may pose a barrier when starting to use cloud servers. However, some cloud server providers now...
|
by: af34tf |
last post by:
Hi Guys, I have a domain whose name is BytesLimited.com, and I want to sell it. Does anyone know about platforms that allow me to list my domain in auction for free. Thank you
|
by: Faith0G |
last post by:
I am starting a new it consulting business and it's been a while since I setup a new website. Is wordpress still the best web based software for hosting a 5 page website? The webpages will be...
|
by: isladogs |
last post by:
The next Access Europe User Group meeting will be on Wednesday 3 Apr 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM).
In this session, we are pleased to welcome former...
| |