By using this site, you agree to our updated Privacy Policy and our Terms of Use. Manage your Cookies Settings.
444,089 Members | 2,260 Online
Bytes IT Community
+ Ask a Question
Need help? Post your question and get tips & solutions from a community of 444,089 IT Pros & Developers. It's quick & easy.

cPickle problems

P: n/a
This is going to be a pretty vague message because it involves a large
block of code I'd rather avoid posting. Basically, I've been pickling
a dictionary of instances of a class I've created (which contains
references to other instances of other classes). At some point in the
last few weeks, pickling has stopped working with the following error:

Traceback (most recent call last):
File "./generateTools.py", line 50, in ?
proj.findVHDLfiles("vhd")
File "/cygdrive/c/fpga/tools/FPGAProject.py", line 630, in
findVHDLfiles
cpickle.dump(filenameToClassDic,pickleFile,PICKLE_ FILE_PROTOCOL)
File "/usr/lib/python2.4/copy_reg.py", line 83, in _reduce_ex
dict = getstate()
TypeError: 'str' object is not callable
Does anyone have any idea why this might be failing in this way? It's
driving me crazy. If need be I can probably provide the code to the
class, though it's about 1000 lines, most of which is PyParsing code.
I appreciate any help you guys can provide.
Jeff

Nov 17 '06 #1
Share this Question
Share on Google+
8 Replies


P: n/a
Oh, and I suppose I should provide some version information:

$ python
Python 2.4.3 (#1, May 18 2006, 07:40:45)
[GCC 3.3.3 (cygwin special)] on cygwin
Jeff Poole wrote:
This is going to be a pretty vague message because it involves a large
block of code I'd rather avoid posting. Basically, I've been pickling
a dictionary of instances of a class I've created (which contains
references to other instances of other classes). At some point in the
last few weeks, pickling has stopped working with the following error:

Traceback (most recent call last):
File "./generateTools.py", line 50, in ?
proj.findVHDLfiles("vhd")
File "/cygdrive/c/fpga/tools/FPGAProject.py", line 630, in
findVHDLfiles
cpickle.dump(filenameToClassDic,pickleFile,PICKLE_ FILE_PROTOCOL)
File "/usr/lib/python2.4/copy_reg.py", line 83, in _reduce_ex
dict = getstate()
TypeError: 'str' object is not callable
Does anyone have any idea why this might be failing in this way? It's
driving me crazy. If need be I can probably provide the code to the
class, though it's about 1000 lines, most of which is PyParsing code.
I appreciate any help you guys can provide.
Jeff
Nov 17 '06 #2

P: n/a
To clutter this up with yet another message, this is what happens if I
use pickle instead of cPickle. Same error, but it has more of a stack
trace so someone more advanced than myself might be able to pick out
what is going amiss.

Traceback (most recent call last):
File "./generateTools.py", line 50, in ?
proj.findVHDLfiles("vhd")
File "/cygdrive/c/fpga/tools/FPGAProject.py", line 631, in
findVHDLfiles
cpickle.dump(filenameToClassDic,pickleFile,PICKLE_ FILE_PROTOCOL)
File "/usr/lib/python2.4/pickle.py", line 1382, in dump
Pickler(file, protocol, bin).dump(obj)
File "/usr/lib/python2.4/pickle.py", line 231, in dump
self.save(obj)
File "/usr/lib/python2.4/pickle.py", line 293, in save
f(self, obj) # Call unbound method with explicit self
File "/usr/lib/python2.4/pickle.py", line 663, in save_dict
self._batch_setitems(obj.iteritems())
File "/usr/lib/python2.4/pickle.py", line 677, in _batch_setitems
save(v)
File "/usr/lib/python2.4/pickle.py", line 338, in save
self.save_reduce(obj=obj, *rv)
File "/usr/lib/python2.4/pickle.py", line 433, in save_reduce
save(state)
File "/usr/lib/python2.4/pickle.py", line 293, in save
f(self, obj) # Call unbound method with explicit self
File "/usr/lib/python2.4/pickle.py", line 663, in save_dict
self._batch_setitems(obj.iteritems())
File "/usr/lib/python2.4/pickle.py", line 677, in _batch_setitems
save(v)
File "/usr/lib/python2.4/pickle.py", line 293, in save
f(self, obj) # Call unbound method with explicit self
File "/usr/lib/python2.4/pickle.py", line 614, in save_list
self._batch_appends(iter(obj))
File "/usr/lib/python2.4/pickle.py", line 629, in _batch_appends
save(x)
File "/usr/lib/python2.4/pickle.py", line 293, in save
f(self, obj) # Call unbound method with explicit self
File "/usr/lib/python2.4/pickle.py", line 576, in save_tuple
save(element)
File "/usr/lib/python2.4/pickle.py", line 338, in save
self.save_reduce(obj=obj, *rv)
File "/usr/lib/python2.4/pickle.py", line 433, in save_reduce
save(state)
File "/usr/lib/python2.4/pickle.py", line 293, in save
f(self, obj) # Call unbound method with explicit self
File "/usr/lib/python2.4/pickle.py", line 663, in save_dict
self._batch_setitems(obj.iteritems())
File "/usr/lib/python2.4/pickle.py", line 677, in _batch_setitems
save(v)
File "/usr/lib/python2.4/pickle.py", line 313, in save
rv = reduce(self.proto)
File "/usr/lib/python2.4/copy_reg.py", line 83, in _reduce_ex
dict = getstate()
TypeError: 'str' object is not callable
Jeff Poole wrote:
Oh, and I suppose I should provide some version information:

$ python
Python 2.4.3 (#1, May 18 2006, 07:40:45)
[GCC 3.3.3 (cygwin special)] on cygwin
Jeff Poole wrote:
This is going to be a pretty vague message because it involves a large
block of code I'd rather avoid posting. Basically, I've been pickling
a dictionary of instances of a class I've created (which contains
references to other instances of other classes). At some point in the
last few weeks, pickling has stopped working with the following error:

Traceback (most recent call last):
File "./generateTools.py", line 50, in ?
proj.findVHDLfiles("vhd")
File "/cygdrive/c/fpga/tools/FPGAProject.py", line 630, in
findVHDLfiles
cpickle.dump(filenameToClassDic,pickleFile,PICKLE_ FILE_PROTOCOL)
File "/usr/lib/python2.4/copy_reg.py", line 83, in _reduce_ex
dict = getstate()
TypeError: 'str' object is not callable
Does anyone have any idea why this might be failing in this way? It's
driving me crazy. If need be I can probably provide the code to the
class, though it's about 1000 lines, most of which is PyParsing code.
I appreciate any help you guys can provide.
Jeff
Nov 17 '06 #3

P: n/a
At Thursday 16/11/2006 21:48, Jeff Poole wrote:
File "/usr/lib/python2.4/pickle.py", line 313, in save
rv = reduce(self.proto)
File "/usr/lib/python2.4/copy_reg.py", line 83, in _reduce_ex
dict = getstate()
TypeError: 'str' object is not callable
Someway, self.__getstate__ is a string, not a method...
Since it fails inside a python module, you could print some debug
information to see what happens, like repr(self), type(self), repr(getstate)...

--
Gabriel Genellina
Softlab SRL

__________________________________________________
Correo Yahoo!
Espacio para todos tus mensajes, antivirus y antispam ˇgratis!
ˇAbrí tu cuenta ya! - http://correo.yahoo.com.ar
Nov 17 '06 #4

P: n/a
Good idea. Well, I did that, and I found out that the object causing
problems is a ParseResults object (a class from PyParsing) and that the
__getstate__ member is in fact an empty string (''). I'm not sure
where this leaves me... The PyParsing code clearly never creates such
a member and my code never creates it. In fact, searching all the code
involved (not including what is in /usr/lib/python2.4 ), shows no use
of __getstate__ at all (or even the string "getstate").

Ok, I figured it out. ParseResults has the following member:

def __getattr__( self, name ):
if name not in self.__slots__:
if self.__tokdict.has_key( name ):
if name not in self.__accumNames:
return self.__tokdict[name][-1][0]
else:
return ParseResults([ v[0] for v in
self.__tokdict[name] ])
else:
return ""
return None

So when something tries to retrieve .__getstate__, it returns an empty
string. If I tell it to raise an AttributeException instead, then I
get this message:

TypeError: a class that defines __slots__ without defining __getstate__
cannot be pickled

I think I need to talk to the author of PyParsing about this one...
Thanks for your help!
Jeff

Someway, self.__getstate__ is a string, not a method...
Since it fails inside a python module, you could print some debug
information to see what happens, like repr(self), type(self), repr(getstate)...

--
Gabriel Genellina
Softlab SRL

__________________________________________________
Correo Yahoo!
Espacio para todos tus mensajes, antivirus y antispam ˇgratis!
ˇAbrí tu cuenta ya! - http://correo.yahoo.com.ar
Nov 17 '06 #5

P: n/a

Jeff Poole wrote:
Good idea. Well, I did that, and I found out that the object causing
problems is a ParseResults object (a class from PyParsing) and that the
__getstate__ member is in fact an empty string (''). I'm not sure
where this leaves me... The PyParsing code clearly never creates such
Sounds like ParseResults is not intended to be pickable.

-Mike

Nov 17 '06 #6

P: n/a
"Jeff Poole" <ko****@gmail.comwrote in message
news:11**********************@k70g2000cwa.googlegr oups.com...
So when something tries to retrieve .__getstate__, it returns an empty
string. If I tell it to raise an AttributeException instead, then I
get this message:

TypeError: a class that defines __slots__ without defining __getstate__
cannot be pickled

I think I need to talk to the author of PyParsing about this one...
Thanks for your help!
Jeff
Jeff -

Sorry not to reply sooner, I have been out of town with limited access to
c.l.py. Also, I didn't really pick up on "cPickle problems" as a
pyparsing-related post. :)

I would be happy to make ParseResults picklable, I didn't realize that this
getattr behavior was defeating that capability. I have had people
successfully use pickle on pyparsing grammars, but have not had anyone
report any pickle experience with ParseResults objects, either pro or con.

I have a test version of pyparsing with pickling support added, and it
passes my simple unit tests. I could send it to you if you would test it
against your program.

-- Paul


Nov 19 '06 #7

P: n/a
Jeff -
>
Sorry not to reply sooner, I have been out of town with limited access to
c.l.py. Also, I didn't really pick up on "cPickle problems" as a
pyparsing-related post. :)

I would be happy to make ParseResults picklable, I didn't realize that this
getattr behavior was defeating that capability. I have had people
successfully use pickle on pyparsing grammars, but have not had anyone
report any pickle experience with ParseResults objects, either pro or con.

I have a test version of pyparsing with pickling support added, and it
passes my simple unit tests. I could send it to you if you would test it
against your program.

-- Paul
Paul,

No problem, I think we've both been busy with other things. I was
actually going to post something on the PyParsing wikispaces since I
wasn't sure if you checked here. I wasn't actually intentionally
pickling any ParseResults objects, but I realized that in my parse
actions, I was making a copy of the "t" parameter, which is actually a
ParseResults object. I switched my code to use t.asList() and things
are ok now. If you'd like me to test your version, I'd be glad to,
otherwise I can wait until whenever that change makes it's way into a
release.

Jeff

Nov 20 '06 #8

P: n/a
Jeff Poole wrote:
Jeff -

Sorry not to reply sooner, I have been out of town with limited access to
c.l.py. Also, I didn't really pick up on "cPickle problems" as a
pyparsing-related post. :)
:-)
Choice of keywords is important -- for example, "Excel" always grabs my
attention. In this case, the keyword the OP should have used is not
"cPickle" but "re"
:-)

[snip]

-- Paul

Paul,

No problem, I think we've both been busy with other things. I was
actually going to post something on the PyParsing wikispaces since I
wasn't sure if you checked here.
:-)
I suggest that the OP take his googler back to the shop he bought it
from; it is severely damaged.
:-)

HTH,
John

Nov 20 '06 #9

This discussion thread is closed

Replies have been disabled for this discussion.