473,324 Members | 2,246 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,324 software developers and data experts.

How to pickle dictionaries?

I am trying to make some of the in-memory dictionaries in my code persistent using bsddb 4.3. Here is an example dictionary in my code for which I would like to create a persistent version:

Expand|Select|Wrap|Line Numbers
  1. # Example dictionary
  2. patterns = {pat1: {t1:None},
  3.                  pat2: {t1:None, t2:None},
  4.                  pat3: {t2:None}}
  5. # Each key is an instance of the class T
  6. # Each value is a dictionary, where keys are instances of the class T, and values are None. I could have used set instead of dictionary here
  7.  
  8. # Sample instance data
  9. t1 = T('John','age','35')
  10. t2 = T('John','created','www.blogger.com')
  11. pat1 = T('John','age',None)
  12. pat2 = T('John',None,None)
  13. pat3 = T('John','created',None)
  14.  
  15. class T(tuple):
  16.    def __new__(cls,*args):
  17.       a,b,c= tuple(args)
  18.       return tuple.__new__(cls,(a,b,c))
  19.  
I plan to convert each key in the dictionary into a string using cPickle before I can store the key in the persistent hash.

I am looking for a way to store the values in the dictionary. Since each value is a dictionary, I am not sure how exactly to pickle it. Each value in the hash table is likely to get updated quite often; hence I need an efficient way to pickle and unpickle the values. Any ideas? Please help.

Thanks,
D.
May 8 '07 #1
5 2513
bartonc
6,596 Expert 4TB
I am trying to make some of the in-memory dictionaries in my code persistent using bsddb 4.3. Here is an example dictionary in my code for which I would like to create a persistent version:

Expand|Select|Wrap|Line Numbers
  1. # Example dictionary
  2. patterns = {pat1: {t1:None},
  3.                  pat2: {t1:None, t2:None},
  4.                  pat3: {t2:None}}
  5. # Each key is an instance of the class T
  6. # Each value is a dictionary, where keys are instances of the class T, and values are None. I could have used set instead of dictionary here
  7.  
  8. # Sample instance data
  9. t1 = T('John','age','35')
  10. t2 = T('John','created','www.blogger.com')
  11. pat1 = T('John','age',None)
  12. pat2 = T('John',None,None)
  13. pat3 = T('John','created',None)
  14.  
  15. class T(tuple):
  16.    def __new__(cls,*args):
  17.       a,b,c= tuple(args)
  18.       return tuple.__new__(cls,(a,b,c))
  19.  
I plan to convert each key in the dictionary into a string using cPickle before I can store the key in the persistent hash.

I am looking for a way to store the values in the dictionary. Since each value is a dictionary, I am not sure how exactly to pickle it. Each value in the hash table is likely to get updated quite often; hence I need an efficient way to pickle and unpickle the values. Any ideas? Please help.

Thanks,
D.
I'd say go straight to the DB. I've posted dictionary to sql query converters in the Articles section.
May 8 '07 #2
bvdet
2,851 Expert Mod 2GB
I am trying to make some of the in-memory dictionaries in my code persistent using bsddb 4.3. Here is an example dictionary in my code for which I would like to create a persistent version:

Expand|Select|Wrap|Line Numbers
  1. # Example dictionary
  2. patterns = {pat1: {t1:None},
  3.                  pat2: {t1:None, t2:None},
  4.                  pat3: {t2:None}}
  5. # Each key is an instance of the class T
  6. # Each value is a dictionary, where keys are instances of the class T, and values are None. I could have used set instead of dictionary here
  7.  
  8. # Sample instance data
  9. t1 = T('John','age','35')
  10. t2 = T('John','created','www.blogger.com')
  11. pat1 = T('John','age',None)
  12. pat2 = T('John',None,None)
  13. pat3 = T('John','created',None)
  14.  
  15. class T(tuple):
  16.    def __new__(cls,*args):
  17.       a,b,c= tuple(args)
  18.       return tuple.__new__(cls,(a,b,c))
  19.  
I plan to convert each key in the dictionary into a string using cPickle before I can store the key in the persistent hash.

I am looking for a way to store the values in the dictionary. Since each value is a dictionary, I am not sure how exactly to pickle it. Each value in the hash table is likely to get updated quite often; hence I need an efficient way to pickle and unpickle the values. Any ideas? Please help.

Thanks,
D.
Have you tried this:
Expand|Select|Wrap|Line Numbers
  1. import cPickle
  2. fn = r'your_file'
  3. f = open(fn, "w")
  4. cPickle.Pickler(f).dump(patterns)
  5. f.close()
Unpickle test:
Expand|Select|Wrap|Line Numbers
  1. import cPickle
  2.  
  3. class T(tuple):
  4.    def __new__(cls,*args):
  5.       a,b,c= tuple(args)
  6.       return tuple.__new__(cls,(a,b,c))
  7.  
  8. if __name__ == '__main__':    
  9.  
  10.     fn = r'H:\TEMP\temsys\pickle_dict.txt'
  11.     f = open(fn, "r")
  12.     dd = cPickle.Unpickler(f).load()
  13.     f.close()
  14.  
  15.     for key, value in dd.items():
  16.         print '%s = %s' % (key, value)
>>> ('John', None, None) = {('John', 'age', '35'): None, ('John', 'created', 'www.blogger.com'): None}
('John', 'created', None) = {('John', 'created', 'www.blogger.com'): None}
('John', 'age', None) = {('John', 'age', '35'): None}
>>> type(dd.keys()[0])
<class '__main__.T'>
>>> type(dd.values()[0])
<type 'dict'>
>>> print dd.values()[0]
{('John', 'age', '35'): None, ('John', 'created', 'www.blogger.com'): None}
>>>
May 8 '07 #3
I'd say go straight to the DB. I've posted dictionary to sql query converters in the Articles section.
Thanks, your response was useful!

Could you please point me to the article you wrote, I could not find it.
May 9 '07 #4
Have you tried this:
Expand|Select|Wrap|Line Numbers
  1. import cPickle
  2. fn = r'your_file'
  3. f = open(fn, "w")
  4. cPickle.Pickler(f).dump(patterns)
  5. f.close()
Unpickle test:
Expand|Select|Wrap|Line Numbers
  1. import cPickle
  2.  
  3. class T(tuple):
  4.    def __new__(cls,*args):
  5.       a,b,c= tuple(args)
  6.       return tuple.__new__(cls,(a,b,c))
  7.  
  8. if __name__ == '__main__':    
  9.  
  10.     fn = r'H:\TEMP\temsys\pickle_dict.txt'
  11.     f = open(fn, "r")
  12.     dd = cPickle.Unpickler(f).load()
  13.     f.close()
  14.  
  15.     for key, value in dd.items():
  16.         print '%s = %s' % (key, value)
>>> ('John', None, None) = {('John', 'age', '35'): None, ('John', 'created', 'www.blogger.com'): None}
('John', 'created', None) = {('John', 'created', 'www.blogger.com'): None}
('John', 'age', None) = {('John', 'age', '35'): None}
>>> type(dd.keys()[0])
<class '__main__.T'>
>>> type(dd.values()[0])
<type 'dict'>
>>> print dd.values()[0]
{('John', 'age', '35'): None, ('John', 'created', 'www.blogger.com'): None}
>>>
Thanks for your response! I did try this out.
May 9 '07 #5
bartonc
6,596 Expert 4TB
Thanks, your response was useful!

Could you please point me to the article you wrote, I could not find it.
They are helper functions in this post.
May 9 '07 #6

Sign in to post your reply or Sign up for a free account.

Similar topics

3
by: Michael Hohn | last post by:
Hi, under python 2.2, the pickle/unpickle sequence incorrectly restores a larger data structure I have. Under Python 2.3, these structures now give an explicit exception from...
1
by: A.B., Khalid | last post by:
I wonder if someone can explain what is wrong here. I am pickling a list of dictionaries (see code attached) and unpickling it back using the HIGHEST_PROTOCOL of pickle and cPickle. I am getting an...
0
by: Mike P. | last post by:
Hi all, I'm working on a simulation (can be considered a game) in Python where I want to be able to dump the simulation state to a file and be able to load it up later. I have used the standard...
6
by: Jim Lewis | last post by:
Pickling an instance of a class, gives "can't pickle instancemethod objects". What does this mean? How do I find the class method creating the problem?
10
by: crystalattice | last post by:
I'm creating an RPG for experience and practice. I've finished a character creation module and I'm trying to figure out how to get the file I/O to work. I've read through the python newsgroup...
5
by: Chris | last post by:
Why can pickle serialize references to functions, but not methods? Pickling a function serializes the function name, but pickling a staticmethod, classmethod, or instancemethod generates an...
9
by: andrewfelch | last post by:
Hello all, I'm using the metaclass trick for automatic reloading of class member functions, found at: http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/160164 My problem is that if I 1)...
3
by: fizilla | last post by:
Hello all! I have the following weird problem and since I am new to Python I somehow cannot figure out an elegant solution. The problem reduces to the following question: How to pickle a...
1
by: IceMan85 | last post by:
Hi to all, I have spent the whole morning trying, with no success to pickle an object that I have created. The error that I get is : Can't pickle 'SRE_Match' object: <_sre.SRE_Match object at...
0
by: DolphinDB | last post by:
Tired of spending countless mintues downsampling your data? Look no further! In this article, you’ll learn how to efficiently downsample 6.48 billion high-frequency records to 61 million...
0
by: ryjfgjl | last post by:
ExcelToDatabase: batch import excel into database automatically...
0
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 6 Mar 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:15 (7.15PM). In this month's session, we are pleased to welcome back...
0
by: Vimpel783 | last post by:
Hello! Guys, I found this code on the Internet, but I need to modify it a little. It works well, the problem is this: Data is sent from only one cell, in this case B5, but it is necessary that data...
1
by: PapaRatzi | last post by:
Hello, I am teaching myself MS Access forms design and Visual Basic. I've created a table to capture a list of Top 30 singles and forms to capture new entries. The final step is a form (unbound)...
1
by: CloudSolutions | last post by:
Introduction: For many beginners and individual users, requiring a credit card and email registration may pose a barrier when starting to use cloud servers. However, some cloud server providers now...
0
by: af34tf | last post by:
Hi Guys, I have a domain whose name is BytesLimited.com, and I want to sell it. Does anyone know about platforms that allow me to list my domain in auction for free. Thank you
0
by: Faith0G | last post by:
I am starting a new it consulting business and it's been a while since I setup a new website. Is wordpress still the best web based software for hosting a 5 page website? The webpages will be...
0
isladogs
by: isladogs | last post by:
The next Access Europe User Group meeting will be on Wednesday 3 Apr 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM). In this session, we are pleased to welcome former...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.