473,695 Members | 3,090 Online
Bytes | Software Development & Data Engineering Community
+ Post

Home Posts Topics Members FAQ

cPickle problems

This is going to be a pretty vague message because it involves a large
block of code I'd rather avoid posting. Basically, I've been pickling
a dictionary of instances of a class I've created (which contains
references to other instances of other classes). At some point in the
last few weeks, pickling has stopped working with the following error:

Traceback (most recent call last):
File "./generateTools.p y", line 50, in ?
proj.findVHDLfi les("vhd")
File "/cygdrive/c/fpga/tools/FPGAProject.py" , line 630, in
findVHDLfiles
cpickle.dump(fi lenameToClassDi c,pickleFile,PI CKLE_FILE_PROTO COL)
File "/usr/lib/python2.4/copy_reg.py", line 83, in _reduce_ex
dict = getstate()
TypeError: 'str' object is not callable
Does anyone have any idea why this might be failing in this way? It's
driving me crazy. If need be I can probably provide the code to the
class, though it's about 1000 lines, most of which is PyParsing code.
I appreciate any help you guys can provide.
Jeff

Nov 17 '06 #1
8 3248
Oh, and I suppose I should provide some version information:

$ python
Python 2.4.3 (#1, May 18 2006, 07:40:45)
[GCC 3.3.3 (cygwin special)] on cygwin
Jeff Poole wrote:
This is going to be a pretty vague message because it involves a large
block of code I'd rather avoid posting. Basically, I've been pickling
a dictionary of instances of a class I've created (which contains
references to other instances of other classes). At some point in the
last few weeks, pickling has stopped working with the following error:

Traceback (most recent call last):
File "./generateTools.p y", line 50, in ?
proj.findVHDLfi les("vhd")
File "/cygdrive/c/fpga/tools/FPGAProject.py" , line 630, in
findVHDLfiles
cpickle.dump(fi lenameToClassDi c,pickleFile,PI CKLE_FILE_PROTO COL)
File "/usr/lib/python2.4/copy_reg.py", line 83, in _reduce_ex
dict = getstate()
TypeError: 'str' object is not callable
Does anyone have any idea why this might be failing in this way? It's
driving me crazy. If need be I can probably provide the code to the
class, though it's about 1000 lines, most of which is PyParsing code.
I appreciate any help you guys can provide.
Jeff
Nov 17 '06 #2
To clutter this up with yet another message, this is what happens if I
use pickle instead of cPickle. Same error, but it has more of a stack
trace so someone more advanced than myself might be able to pick out
what is going amiss.

Traceback (most recent call last):
File "./generateTools.p y", line 50, in ?
proj.findVHDLfi les("vhd")
File "/cygdrive/c/fpga/tools/FPGAProject.py" , line 631, in
findVHDLfiles
cpickle.dump(fi lenameToClassDi c,pickleFile,PI CKLE_FILE_PROTO COL)
File "/usr/lib/python2.4/pickle.py", line 1382, in dump
Pickler(file, protocol, bin).dump(obj)
File "/usr/lib/python2.4/pickle.py", line 231, in dump
self.save(obj)
File "/usr/lib/python2.4/pickle.py", line 293, in save
f(self, obj) # Call unbound method with explicit self
File "/usr/lib/python2.4/pickle.py", line 663, in save_dict
self._batch_set items(obj.iteri tems())
File "/usr/lib/python2.4/pickle.py", line 677, in _batch_setitems
save(v)
File "/usr/lib/python2.4/pickle.py", line 338, in save
self.save_reduc e(obj=obj, *rv)
File "/usr/lib/python2.4/pickle.py", line 433, in save_reduce
save(state)
File "/usr/lib/python2.4/pickle.py", line 293, in save
f(self, obj) # Call unbound method with explicit self
File "/usr/lib/python2.4/pickle.py", line 663, in save_dict
self._batch_set items(obj.iteri tems())
File "/usr/lib/python2.4/pickle.py", line 677, in _batch_setitems
save(v)
File "/usr/lib/python2.4/pickle.py", line 293, in save
f(self, obj) # Call unbound method with explicit self
File "/usr/lib/python2.4/pickle.py", line 614, in save_list
self._batch_app ends(iter(obj))
File "/usr/lib/python2.4/pickle.py", line 629, in _batch_appends
save(x)
File "/usr/lib/python2.4/pickle.py", line 293, in save
f(self, obj) # Call unbound method with explicit self
File "/usr/lib/python2.4/pickle.py", line 576, in save_tuple
save(element)
File "/usr/lib/python2.4/pickle.py", line 338, in save
self.save_reduc e(obj=obj, *rv)
File "/usr/lib/python2.4/pickle.py", line 433, in save_reduce
save(state)
File "/usr/lib/python2.4/pickle.py", line 293, in save
f(self, obj) # Call unbound method with explicit self
File "/usr/lib/python2.4/pickle.py", line 663, in save_dict
self._batch_set items(obj.iteri tems())
File "/usr/lib/python2.4/pickle.py", line 677, in _batch_setitems
save(v)
File "/usr/lib/python2.4/pickle.py", line 313, in save
rv = reduce(self.pro to)
File "/usr/lib/python2.4/copy_reg.py", line 83, in _reduce_ex
dict = getstate()
TypeError: 'str' object is not callable
Jeff Poole wrote:
Oh, and I suppose I should provide some version information:

$ python
Python 2.4.3 (#1, May 18 2006, 07:40:45)
[GCC 3.3.3 (cygwin special)] on cygwin
Jeff Poole wrote:
This is going to be a pretty vague message because it involves a large
block of code I'd rather avoid posting. Basically, I've been pickling
a dictionary of instances of a class I've created (which contains
references to other instances of other classes). At some point in the
last few weeks, pickling has stopped working with the following error:

Traceback (most recent call last):
File "./generateTools.p y", line 50, in ?
proj.findVHDLfi les("vhd")
File "/cygdrive/c/fpga/tools/FPGAProject.py" , line 630, in
findVHDLfiles
cpickle.dump(fi lenameToClassDi c,pickleFile,PI CKLE_FILE_PROTO COL)
File "/usr/lib/python2.4/copy_reg.py", line 83, in _reduce_ex
dict = getstate()
TypeError: 'str' object is not callable
Does anyone have any idea why this might be failing in this way? It's
driving me crazy. If need be I can probably provide the code to the
class, though it's about 1000 lines, most of which is PyParsing code.
I appreciate any help you guys can provide.
Jeff
Nov 17 '06 #3
At Thursday 16/11/2006 21:48, Jeff Poole wrote:
File "/usr/lib/python2.4/pickle.py", line 313, in save
rv = reduce(self.pro to)
File "/usr/lib/python2.4/copy_reg.py", line 83, in _reduce_ex
dict = getstate()
TypeError: 'str' object is not callable
Someway, self.__getstate __ is a string, not a method...
Since it fails inside a python module, you could print some debug
information to see what happens, like repr(self), type(self), repr(getstate). ..

--
Gabriel Genellina
Softlab SRL

_______________ _______________ _______________ _____
Correo Yahoo!
Espacio para todos tus mensajes, antivirus y antispam ¡gratis!
¡Abrí tu cuenta ya! - http://correo.yahoo.com.ar
Nov 17 '06 #4
Good idea. Well, I did that, and I found out that the object causing
problems is a ParseResults object (a class from PyParsing) and that the
__getstate__ member is in fact an empty string (''). I'm not sure
where this leaves me... The PyParsing code clearly never creates such
a member and my code never creates it. In fact, searching all the code
involved (not including what is in /usr/lib/python2.4 ), shows no use
of __getstate__ at all (or even the string "getstate") .

Ok, I figured it out. ParseResults has the following member:

def __getattr__( self, name ):
if name not in self.__slots__:
if self.__tokdict. has_key( name ):
if name not in self.__accumNam es:
return self.__tokdict[name][-1][0]
else:
return ParseResults([ v[0] for v in
self.__tokdict[name] ])
else:
return ""
return None

So when something tries to retrieve .__getstate__, it returns an empty
string. If I tell it to raise an AttributeExcept ion instead, then I
get this message:

TypeError: a class that defines __slots__ without defining __getstate__
cannot be pickled

I think I need to talk to the author of PyParsing about this one...
Thanks for your help!
Jeff

Someway, self.__getstate __ is a string, not a method...
Since it fails inside a python module, you could print some debug
information to see what happens, like repr(self), type(self), repr(getstate). ..

--
Gabriel Genellina
Softlab SRL

_______________ _______________ _______________ _____
Correo Yahoo!
Espacio para todos tus mensajes, antivirus y antispam ¡gratis!
¡Abrí tu cuenta ya! - http://correo.yahoo.com.ar
Nov 17 '06 #5

Jeff Poole wrote:
Good idea. Well, I did that, and I found out that the object causing
problems is a ParseResults object (a class from PyParsing) and that the
__getstate__ member is in fact an empty string (''). I'm not sure
where this leaves me... The PyParsing code clearly never creates such
Sounds like ParseResults is not intended to be pickable.

-Mike

Nov 17 '06 #6
"Jeff Poole" <ko****@gmail.c omwrote in message
news:11******** **************@ k70g2000cwa.goo glegroups.com.. .
So when something tries to retrieve .__getstate__, it returns an empty
string. If I tell it to raise an AttributeExcept ion instead, then I
get this message:

TypeError: a class that defines __slots__ without defining __getstate__
cannot be pickled

I think I need to talk to the author of PyParsing about this one...
Thanks for your help!
Jeff
Jeff -

Sorry not to reply sooner, I have been out of town with limited access to
c.l.py. Also, I didn't really pick up on "cPickle problems" as a
pyparsing-related post. :)

I would be happy to make ParseResults picklable, I didn't realize that this
getattr behavior was defeating that capability. I have had people
successfully use pickle on pyparsing grammars, but have not had anyone
report any pickle experience with ParseResults objects, either pro or con.

I have a test version of pyparsing with pickling support added, and it
passes my simple unit tests. I could send it to you if you would test it
against your program.

-- Paul


Nov 19 '06 #7
Jeff -
>
Sorry not to reply sooner, I have been out of town with limited access to
c.l.py. Also, I didn't really pick up on "cPickle problems" as a
pyparsing-related post. :)

I would be happy to make ParseResults picklable, I didn't realize that this
getattr behavior was defeating that capability. I have had people
successfully use pickle on pyparsing grammars, but have not had anyone
report any pickle experience with ParseResults objects, either pro or con.

I have a test version of pyparsing with pickling support added, and it
passes my simple unit tests. I could send it to you if you would test it
against your program.

-- Paul
Paul,

No problem, I think we've both been busy with other things. I was
actually going to post something on the PyParsing wikispaces since I
wasn't sure if you checked here. I wasn't actually intentionally
pickling any ParseResults objects, but I realized that in my parse
actions, I was making a copy of the "t" parameter, which is actually a
ParseResults object. I switched my code to use t.asList() and things
are ok now. If you'd like me to test your version, I'd be glad to,
otherwise I can wait until whenever that change makes it's way into a
release.

Jeff

Nov 20 '06 #8
Jeff Poole wrote:
Jeff -

Sorry not to reply sooner, I have been out of town with limited access to
c.l.py. Also, I didn't really pick up on "cPickle problems" as a
pyparsing-related post. :)
:-)
Choice of keywords is important -- for example, "Excel" always grabs my
attention. In this case, the keyword the OP should have used is not
"cPickle" but "re"
:-)

[snip]

-- Paul

Paul,

No problem, I think we've both been busy with other things. I was
actually going to post something on the PyParsing wikispaces since I
wasn't sure if you checked here.
:-)
I suggest that the OP take his googler back to the shop he bought it
from; it is severely damaged.
:-)

HTH,
John

Nov 20 '06 #9

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

13
3679
by: Drochom | last post by:
Hello, I have a huge problem with loading very simple structure into memory it is a list of tuples, it has 6MB and consists of 100000 elements >import cPickle >plik = open("mealy","r") >mealy = cPickle.load(plik) >plik.close()
0
1191
by: Richard Kessler | last post by:
I am attempting a GUI using BOA Constructor. I have some simple code to pickle an object, but for some reason when I use cPickle it hangs the system, but pickle works just fine. I do not have a clue why cPickle will not work. The code is myObj = MyObject() myObj.SetSomeProperties f = open("my file name",'w')
0
1110
by: Simon Dahlbacka | last post by:
I'm trying to cache some information using cPickle and I thought that using cPickle.HIGHEST_PROTOCOL would give me an efficient representation. However, when trying to load the information back I just keep getting a broken pickle with EOF error from cPicle.load(file('myfile.pickle','r')) if I am not using cPickle.HIGHEST_PROTOCOL or -1 or 2 it seems to
2
2134
by: sh | last post by:
Hi guys, Well, I have a (maybe dumb) question. I want to write my own little blog using Python (as a fairly small but doable project for myself to learn more deaply Python in a web context). I don't want so far to use a database as a backend, I'd prefer use XML which is enough for a small amount of data the blog would have to deal with.
5
2230
by: Alex Polite | last post by:
I need to put recursive data structures on disc and found out that cPickle doesn't like recursion. What are my options? alex -- Alex Polite http://polite.se
5
9451
by: Marcus Lowland | last post by:
Hello, I'm fairly new to python and have read about and wanted to begin experimenting with cpickle. As I understand, this should be a native module in the python library. I have python 2.3 and now just installed 2.4, but am not able to import or find cpickle.py in any directory of the install, or in the previous version (pickle.py is present and imports correctly). Is there a seperate module package that must be downloaded and installed......
4
2826
by: Mingus Tsai | last post by:
Hello- please help with unpickling problem: I am using Python version 2.3.4 with IDLE version 1.0.3 on a Windows XPhome system. My problem is with using cPickle to deserialize my pickled arrays of datetime.datetime instances. The following is the code I have written: import cPickle, datetime import Numeric
1
1369
by: Carl J. Van Arsdall | last post by:
Hey everyone, cPickle is raising an ImportError that I just don't quite understand. Before I paste the code, let me explain the application. Basically the part of the application that failed is a function that loads a list of objects from a file using cPickle. This list is a queue of requests. I've done some research and it looks like cPickle tries to load some modules as some kind of test. From what I can tell the module that cPickle...
5
1759
by: Victor Kryukov | last post by:
Hello list, The following behavior is completely unexpected. Is it a bug or a by- design feature? Regards, Victor. -----------------
0
8640
marktang
by: marktang | last post by:
ONU (Optical Network Unit) is one of the key components for providing high-speed Internet services. Its primary function is to act as an endpoint device located at the user's premises. However, people are often confused as to whether an ONU can Work As a Router. In this blog post, we’ll explore What is ONU, What Is Router, ONU & Router’s main usage, and What is the difference between ONU and Router. Let’s take a closer look ! Part I. Meaning of...
0
8582
by: Hystou | last post by:
Most computers default to English, but sometimes we require a different language, especially when relocating. Forgot to request a specific language before your computer shipped? No problem! You can effortlessly switch the default language on Windows 10 without reinstalling. I'll walk you through it. First, let's disable language synchronization. With a Microsoft account, language settings sync across devices. To prevent any complications,...
0
9001
jinu1996
by: jinu1996 | last post by:
In today's digital age, having a compelling online presence is paramount for businesses aiming to thrive in a competitive landscape. At the heart of this digital strategy lies an intricately woven tapestry of website design and digital marketing. It's not merely about having a website; it's about crafting an immersive digital experience that captivates audiences and drives business growth. The Art of Business Website Design Your website is...
1
8860
by: Hystou | last post by:
Overview: Windows 11 and 10 have less user interface control over operating system update behaviour than previous versions of Windows. In Windows 11 and 10, there is no way to turn off the Windows Update option using the Control Panel or Settings app; it automatically checks for updates and installs any it finds, whether you like it or not. For most users, this new feature is actually very convenient. If you want to control the update process,...
0
7672
agi2029
by: agi2029 | last post by:
Let's talk about the concept of autonomous AI software engineers and no-code agents. These AIs are designed to manage the entire lifecycle of a software development project—planning, coding, testing, and deployment—without human intervention. Imagine an AI that can take a project description, break it down, write the code, debug it, and then launch it, all on its own.... Now, this would greatly impact the work of software developers. The idea...
0
5841
by: conductexam | last post by:
I have .net C# application in which I am extracting data from word file and save it in database particularly. To store word all data as it is I am converting the whole word file firstly in HTML and then checking html paragraph one by one. At the time of converting from word file to html my equations which are in the word document file was convert into image. Globals.ThisAddIn.Application.ActiveDocument.Select();...
0
4348
by: TSSRALBI | last post by:
Hello I'm a network technician in training and I need your help. I am currently learning how to create and manage the different types of VPNs and I have a question about LAN-to-LAN VPNs. The last exercise I practiced was to create a LAN-to-LAN VPN between two Pfsense firewalls, by using IPSEC protocols. I succeeded, with both firewalls in the same network. But I'm wondering if it's possible to do the same thing, with 2 Pfsense firewalls...
1
3018
by: 6302768590 | last post by:
Hai team i want code for transfer the data from one system to another through IP address by using C# our system has to for every 5mins then we have to update the data what the data is updated we have to send another system
2
2281
muto222
by: muto222 | last post by:
How can i add a mobile payment intergratation into php mysql website.

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.