By using this site, you agree to our updated Privacy Policy and our Terms of Use. Manage your Cookies Settings.
444,002 Members | 1,050 Online
Bytes IT Community
+ Ask a Question
Need help? Post your question and get tips & solutions from a community of 444,002 IT Pros & Developers. It's quick & easy.

How to pickle dictionaries?

P: 7
I am trying to make some of the in-memory dictionaries in my code persistent using bsddb 4.3. Here is an example dictionary in my code for which I would like to create a persistent version:

Expand|Select|Wrap|Line Numbers
  1. # Example dictionary
  2. patterns = {pat1: {t1:None},
  3.                  pat2: {t1:None, t2:None},
  4.                  pat3: {t2:None}}
  5. # Each key is an instance of the class T
  6. # Each value is a dictionary, where keys are instances of the class T, and values are None. I could have used set instead of dictionary here
  7.  
  8. # Sample instance data
  9. t1 = T('John','age','35')
  10. t2 = T('John','created','www.blogger.com')
  11. pat1 = T('John','age',None)
  12. pat2 = T('John',None,None)
  13. pat3 = T('John','created',None)
  14.  
  15. class T(tuple):
  16.    def __new__(cls,*args):
  17.       a,b,c= tuple(args)
  18.       return tuple.__new__(cls,(a,b,c))
  19.  
I plan to convert each key in the dictionary into a string using cPickle before I can store the key in the persistent hash.

I am looking for a way to store the values in the dictionary. Since each value is a dictionary, I am not sure how exactly to pickle it. Each value in the hash table is likely to get updated quite often; hence I need an efficient way to pickle and unpickle the values. Any ideas? Please help.

Thanks,
D.
May 8 '07 #1
Share this Question
Share on Google+
5 Replies


bartonc
Expert 5K+
P: 6,596
I am trying to make some of the in-memory dictionaries in my code persistent using bsddb 4.3. Here is an example dictionary in my code for which I would like to create a persistent version:

Expand|Select|Wrap|Line Numbers
  1. # Example dictionary
  2. patterns = {pat1: {t1:None},
  3.                  pat2: {t1:None, t2:None},
  4.                  pat3: {t2:None}}
  5. # Each key is an instance of the class T
  6. # Each value is a dictionary, where keys are instances of the class T, and values are None. I could have used set instead of dictionary here
  7.  
  8. # Sample instance data
  9. t1 = T('John','age','35')
  10. t2 = T('John','created','www.blogger.com')
  11. pat1 = T('John','age',None)
  12. pat2 = T('John',None,None)
  13. pat3 = T('John','created',None)
  14.  
  15. class T(tuple):
  16.    def __new__(cls,*args):
  17.       a,b,c= tuple(args)
  18.       return tuple.__new__(cls,(a,b,c))
  19.  
I plan to convert each key in the dictionary into a string using cPickle before I can store the key in the persistent hash.

I am looking for a way to store the values in the dictionary. Since each value is a dictionary, I am not sure how exactly to pickle it. Each value in the hash table is likely to get updated quite often; hence I need an efficient way to pickle and unpickle the values. Any ideas? Please help.

Thanks,
D.
I'd say go straight to the DB. I've posted dictionary to sql query converters in the Articles section.
May 8 '07 #2

bvdet
Expert Mod 2.5K+
P: 2,851
I am trying to make some of the in-memory dictionaries in my code persistent using bsddb 4.3. Here is an example dictionary in my code for which I would like to create a persistent version:

Expand|Select|Wrap|Line Numbers
  1. # Example dictionary
  2. patterns = {pat1: {t1:None},
  3.                  pat2: {t1:None, t2:None},
  4.                  pat3: {t2:None}}
  5. # Each key is an instance of the class T
  6. # Each value is a dictionary, where keys are instances of the class T, and values are None. I could have used set instead of dictionary here
  7.  
  8. # Sample instance data
  9. t1 = T('John','age','35')
  10. t2 = T('John','created','www.blogger.com')
  11. pat1 = T('John','age',None)
  12. pat2 = T('John',None,None)
  13. pat3 = T('John','created',None)
  14.  
  15. class T(tuple):
  16.    def __new__(cls,*args):
  17.       a,b,c= tuple(args)
  18.       return tuple.__new__(cls,(a,b,c))
  19.  
I plan to convert each key in the dictionary into a string using cPickle before I can store the key in the persistent hash.

I am looking for a way to store the values in the dictionary. Since each value is a dictionary, I am not sure how exactly to pickle it. Each value in the hash table is likely to get updated quite often; hence I need an efficient way to pickle and unpickle the values. Any ideas? Please help.

Thanks,
D.
Have you tried this:
Expand|Select|Wrap|Line Numbers
  1. import cPickle
  2. fn = r'your_file'
  3. f = open(fn, "w")
  4. cPickle.Pickler(f).dump(patterns)
  5. f.close()
Unpickle test:
Expand|Select|Wrap|Line Numbers
  1. import cPickle
  2.  
  3. class T(tuple):
  4.    def __new__(cls,*args):
  5.       a,b,c= tuple(args)
  6.       return tuple.__new__(cls,(a,b,c))
  7.  
  8. if __name__ == '__main__':    
  9.  
  10.     fn = r'H:\TEMP\temsys\pickle_dict.txt'
  11.     f = open(fn, "r")
  12.     dd = cPickle.Unpickler(f).load()
  13.     f.close()
  14.  
  15.     for key, value in dd.items():
  16.         print '%s = %s' % (key, value)
>>> ('John', None, None) = {('John', 'age', '35'): None, ('John', 'created', 'www.blogger.com'): None}
('John', 'created', None) = {('John', 'created', 'www.blogger.com'): None}
('John', 'age', None) = {('John', 'age', '35'): None}
>>> type(dd.keys()[0])
<class '__main__.T'>
>>> type(dd.values()[0])
<type 'dict'>
>>> print dd.values()[0]
{('John', 'age', '35'): None, ('John', 'created', 'www.blogger.com'): None}
>>>
May 8 '07 #3

P: 7
I'd say go straight to the DB. I've posted dictionary to sql query converters in the Articles section.
Thanks, your response was useful!

Could you please point me to the article you wrote, I could not find it.
May 9 '07 #4

P: 7
Have you tried this:
Expand|Select|Wrap|Line Numbers
  1. import cPickle
  2. fn = r'your_file'
  3. f = open(fn, "w")
  4. cPickle.Pickler(f).dump(patterns)
  5. f.close()
Unpickle test:
Expand|Select|Wrap|Line Numbers
  1. import cPickle
  2.  
  3. class T(tuple):
  4.    def __new__(cls,*args):
  5.       a,b,c= tuple(args)
  6.       return tuple.__new__(cls,(a,b,c))
  7.  
  8. if __name__ == '__main__':    
  9.  
  10.     fn = r'H:\TEMP\temsys\pickle_dict.txt'
  11.     f = open(fn, "r")
  12.     dd = cPickle.Unpickler(f).load()
  13.     f.close()
  14.  
  15.     for key, value in dd.items():
  16.         print '%s = %s' % (key, value)
>>> ('John', None, None) = {('John', 'age', '35'): None, ('John', 'created', 'www.blogger.com'): None}
('John', 'created', None) = {('John', 'created', 'www.blogger.com'): None}
('John', 'age', None) = {('John', 'age', '35'): None}
>>> type(dd.keys()[0])
<class '__main__.T'>
>>> type(dd.values()[0])
<type 'dict'>
>>> print dd.values()[0]
{('John', 'age', '35'): None, ('John', 'created', 'www.blogger.com'): None}
>>>
Thanks for your response! I did try this out.
May 9 '07 #5

bartonc
Expert 5K+
P: 6,596
Thanks, your response was useful!

Could you please point me to the article you wrote, I could not find it.
They are helper functions in this post.
May 9 '07 #6

Post your reply

Sign in to post your reply or Sign up for a free account.