473,320 Members | 2,112 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,320 software developers and data experts.

Know a good pickle howto?

I need to know how to use pickle (and its brethren) correctly. Does anyone know of a good tutorial? The python manual is a bit too dense on this subject.
Jan 14 '07 #1
8 5147
ghostdog74
511 Expert 256MB
how about this ?here
Jan 14 '07 #2
bartonc
6,596 Expert 4TB
I need to know how to use pickle (and its brethren) correctly. Does anyone know of a good tutorial? The python manual is a bit too dense on this subject.
It's really quite simple:
Expand|Select|Wrap|Line Numbers
  1. >>> import cPickle
  2. >>> strList = ['asf','dadf','adf','adf',]
  3. >>> pickleFile = open("pickleTest.dat", "w")
  4. >>> cPickle.dump(strList, pickleFile)
  5. >>> pickleFile.close()
  6. >>> pickleFile = open("pickleTest.dat")
  7. >>> unpickledList = cPickle.load(pickleFile)
  8. >>> print unpickledList
  9. ['asf', 'dadf', 'adf', 'adf']
  10. >>> pickleFile.close()
  11. >>> 
Jan 14 '07 #3
bartonc
6,596 Expert 4TB
how about this ?here
Great Post! I'll put that link into the new sticky (actually the parent page). Thanks gd
Jan 14 '07 #4
ghostdog74
511 Expert 256MB
No prob :)
Jan 14 '07 #5
bartonc
6,596 Expert 4TB
No prob :)
two posts to go...
Jan 14 '07 #6
bvdet
2,851 Expert Mod 2GB
I need to know how to use pickle (and its brethren) correctly. Does anyone know of a good tutorial? The python manual is a bit too dense on this subject.
We use Pickler and Unpickler in several of our applications to save and restore a dictionary or list of dictionaries.
Library functions:
Expand|Select|Wrap|Line Numbers
  1. import os
  2. import macrolib.pickle
  3. from param import Warning
  4.  
  5. def import_data(file_name):
  6.     try:
  7.         f = open(file_name, "r")
  8.     except IOError, e:
  9.         # unable to open file
  10.         Warning("Default file import error. Reverting to original defaults...")
  11.         return None
  12.  
  13.     # file open is successful
  14.     try:
  15.  
  16.         dd = macrolib.pickle.Unpickler(f).load()
  17.         f.close()
  18.         # test for a dictionary or list of dictionaries
  19.         if isinstance(dd, dict):
  20.             return dd
  21.         elif isinstance(dd, list):
  22.             for i, item in enumerate(dd):
  23.                 if not isinstance(item, dict):
  24.                     Warning("**INVALID** List item %s is not a dictionary." % (i))
  25.                     return None
  26.             # always return one dictionary
  27.             ddr = {}
  28.             for d in dd: ddr.update(d)
  29.             return ddr
  30.         else:
  31.             # dd is not a dictionary or list of dictionaries
  32.             Warning("Invalid imported data type.\nData must be a dictionary or list of dictionaries.")
  33.             return None
  34.  
  35.     except:
  36.         # file is not compatible with Unpickler - close file and warn user
  37.         f.close()
  38.         Warning("Invalid defaults file. Reverting to original defaults...")
  39.         return None
  40.  
  41. def export_data(file_name, dd):
  42.     if check_Defaults_dir(os.path.dirname(file_name)):
  43.         # directory exists or was created
  44.         try:
  45.             f = open(file_name, "w")
  46.             macrolib.pickle.Pickler(f).dump(dd)
  47.             f.close()
  48.             return True
  49.         except:
  50.             Warning("Default values file export error.")
  51.             return False
  52.  
  53.     else:
  54.         # directory did not exist and the user chose 'no' to create directory
  55.         Warning("The default values file export was aborted")
  56.         return False
Typical function calls:
Expand|Select|Wrap|Line Numbers
  1.     ## auto import defaults data if enabled
  2.     if enable_default_import_export == "Enable":
  3.         dd0 = import_data(os.path.join(default_file_path, def_file))
  4.         if dd0:
  5.             for key, value in dd0.items():
  6.                 exec "%s = %s" % (key, repr(value)) in None
  7.             else:
  8.                 Warning("Invalid data - Reverting to original defaults")
  9.     ## auto export default values to disk if enabled
  10.     if enable_default_import_export == "Enable":
  11.         export_data(os.path.join(default_file_path, def_file), dd_list)
Module pickle works with many data types. From Python Essential Reference:
"Multiple calls to the dump() and load() methods are allowed, provided that the sequence of load() calls used to restore a collection of previously stored objects matches the sequence of dump() calls during the pickling process."
HTH :-),
BV
Jan 14 '07 #7
I figured out how to do what I wanted to do (before you guys posted :p). But I'll have a look at these.
Jan 15 '07 #8
bartonc
6,596 Expert 4TB
I figured out how to do what I wanted to do (before you guys posted :p). But I'll have a look at these.
Did you notice that the docs say that cPickle can be 1000 times faster than the old pickle module?
Jan 15 '07 #9

Sign in to post your reply or Sign up for a free account.

Similar topics

1
by: Simon Burton | last post by:
Hi, I am pickling big graphs of data and running into this problem: File "/usr/lib/python2.2/pickle.py", line 225, in save f(self, object) File "/usr/lib/python2.2/pickle.py", line 414, in...
3
by: Michael Hohn | last post by:
Hi, under python 2.2, the pickle/unpickle sequence incorrectly restores a larger data structure I have. Under Python 2.3, these structures now give an explicit exception from...
28
by: Grant Edwards | last post by:
I finally figured out why one of my apps sometimes fails under Win32 when it always works fine under Linux: Under Win32, the pickle module only works with a subset of floating point values. In...
4
by: Shi Mu | last post by:
I got a sample code and tested it but really can not understand the use of pickle and dump: >>> import pickle >>> f = open("try.txt", "w") >>> pickle.dump(3.14, f) >>> pickle.dump(, f) >>>...
6
by: Jim Lewis | last post by:
Pickling an instance of a class, gives "can't pickle instancemethod objects". What does this mean? How do I find the class method creating the problem?
5
by: Chris | last post by:
Why can pickle serialize references to functions, but not methods? Pickling a function serializes the function name, but pickling a staticmethod, classmethod, or instancemethod generates an...
2
by: Nagu | last post by:
I am trying to save a dictionary of size 65000X50 to a local file and I get the memory error problem. How do I go about resolving this? Is there way to partition the pickle object and combine...
0
by: Nagu | last post by:
I am trying to save a dictionary of size 65000X50 to a local file and I get the memory error problem. How do I go about resolving this? Is there way to partition the pickle object and combine...
1
by: IceMan85 | last post by:
Hi to all, I have spent the whole morning trying, with no success to pickle an object that I have created. The error that I get is : Can't pickle 'SRE_Match' object: <_sre.SRE_Match object at...
0
by: DolphinDB | last post by:
The formulas of 101 quantitative trading alphas used by WorldQuant were presented in the paper 101 Formulaic Alphas. However, some formulas are complex, leading to challenges in calculation. Take...
0
by: ryjfgjl | last post by:
ExcelToDatabase: batch import excel into database automatically...
0
by: Vimpel783 | last post by:
Hello! Guys, I found this code on the Internet, but I need to modify it a little. It works well, the problem is this: Data is sent from only one cell, in this case B5, but it is necessary that data...
0
by: jfyes | last post by:
As a hardware engineer, after seeing that CEIWEI recently released a new tool for Modbus RTU Over TCP/UDP filtering and monitoring, I actively went to its official website to take a look. It turned...
0
by: ArrayDB | last post by:
The error message I've encountered is; ERROR:root:Error generating model response: exception: access violation writing 0x0000000000005140, which seems to be indicative of an access violation...
1
by: PapaRatzi | last post by:
Hello, I am teaching myself MS Access forms design and Visual Basic. I've created a table to capture a list of Top 30 singles and forms to capture new entries. The final step is a form (unbound)...
0
by: CloudSolutions | last post by:
Introduction: For many beginners and individual users, requiring a credit card and email registration may pose a barrier when starting to use cloud servers. However, some cloud server providers now...
0
by: Shællîpôpï 09 | last post by:
If u are using a keypad phone, how do u turn on JavaScript, to access features like WhatsApp, Facebook, Instagram....
0
by: Faith0G | last post by:
I am starting a new it consulting business and it's been a while since I setup a new website. Is wordpress still the best web based software for hosting a 5 page website? The webpages will be...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.