467,146 Members | 1,124 Online
Bytes | Developer Community
Ask Question

Home New Posts Topics Members FAQ

Post your question to a community of 467,146 developers. It's quick & easy.

Still the __new__ hell ...

Sorry to put here too many questions about __init__ __new__
stuff but I always found a new problem when using them.
I have searched for simple __new__ docs on how to do the
basic things but find none.

After trying the solutions people gently posted here
(thanks) I run into new trouble when I go with further
development.

Here is the new situation.

As suggested in a previous post, I used __new__ to
subclass date class but now cPickle/pickle loads
does not work.

from datetime import date
import cPickle,string

class MyDate(date):
def __new__(cls,year,month=None,day=None):
if type(year) is str:
year,month,day=map(int,string.split(year,'-'))
if year<100:
year+=2000
return date.__new__(cls,year,month,day)

class C1(object):
def __init__(self):
self.x=MyDate("2007-3-15")

def f(self):
print self.x

c1=C1()

d=cPickle.dumps(c1)
c2=cPickle.loads(d)
c2.f()

1. year is passed to __new__ as a string but with invalid
contents!

2. If I had a __new__ in class C1, cPickle.loads invokes it
with only one parameter. This forces to allow __new__ accept
only 1 parameter (cls) which I don't want for the normal
situation.

Thanks for your patience.
Paulo
Mar 17 '07 #1
  • viewed: 1520
Share:
18 Replies
On Mar 17, 9:31 pm, Paulo da Silva <psdasil...@esotericaX.ptXwrote:
Sorry to put here too many questions about __init__ __new__
stuff but I always found a new problem when using them.
I have searched for simple __new__ docs on how to do the
basic things but find none.

After trying the solutions people gently posted here
(thanks) I run into new trouble when I go with further
development.

Here is the new situation.

As suggested in a previous post, I used __new__ to
subclass date class but now cPickle/pickle loads
does not work.

from datetime import date
import cPickle,string

class MyDate(date):
def __new__(cls,year,month=None,day=None):
if type(year) is str:
year,month,day=map(int,string.split(year,'-'))
if year<100:
year+=2000
return date.__new__(cls,year,month,day)

class C1(object):
def __init__(self):
self.x=MyDate("2007-3-15")

def f(self):
print self.x

c1=C1()

d=cPickle.dumps(c1)
c2=cPickle.loads(d)
c2.f()
I haven't tried your code but I think that you may need to define a
__reduce__ method in your MyDate class in order to give a clue to the
python as to how to pickle its instances. For more details see:

http://docs.python.org/lib/node321.html

Something like:

class MyDate(date):
...
def __reduce__(self):
return type(self), (self.year, self.month, self.day)

might solve your problem.

HTH

--
Arnaud

Mar 17 '07 #2
On Mar 17, 4:31 pm, Paulo da Silva <psdasil...@esotericaX.ptXwrote:
Sorry to put here too many questions about __init__ __new__
stuff but I always found a new problem when using them.
I have searched for simple __new__ docs on how to do the
basic things but find none.

After trying the solutions people gently posted here
(thanks) I run into new trouble when I go with further
development.

Here is the new situation.

As suggested in a previous post, I used __new__ to
subclass date class but now cPickle/pickle loads
does not work.

from datetime import date
import cPickle,string

class MyDate(date):
def __new__(cls,year,month=None,day=None):
if type(year) is str:
year,month,day=map(int,string.split(year,'-'))
if year<100:
year+=2000
return date.__new__(cls,year,month,day)

class C1(object):
def __init__(self):
self.x=MyDate("2007-3-15")

def f(self):
print self.x

c1=C1()

d=cPickle.dumps(c1)
c2=cPickle.loads(d)
c2.f()

1. year is passed to __new__ as a string but with invalid
contents!

2. If I had a __new__ in class C1, cPickle.loads invokes it
with only one parameter. This forces to allow __new__ accept
only 1 parameter (cls) which I don't want for the normal
situation.

Thanks for your patience.
Paulo
Any special reason you have to use __new__ for this factory method?
This version works, without the problems with __new__:

def getDate(*args):
if isinstance(args[0],basestring):
year,month,day = map(int,string.split(args[0],'-'))
else:
year,month,day = args
if year < 100:
year += 2000
return date(year,month,day)

class C1(object):
def __init__(self):
#self.x=MyDate("2007-3-15")
self.x = getDate("2007-3-15")

def f(self):
print self.x
-- Paul

Mar 17 '07 #3
Arnaud Delobelle escreveu:
On Mar 17, 9:31 pm, Paulo da Silva <psdasil...@esotericaX.ptXwrote:
....
>I used __new__ to
subclass date class but now cPickle/pickle loads
does not work.

from datetime import date
import cPickle,string

class MyDate(date):
def __new__(cls,year,month=None,day=None):
if type(year) is str:
year,month,day=map(int,string.split(year,'-'))
if year<100:
year+=2000
return date.__new__(cls,year,month,day)

class C1(object):
def __init__(self):
self.x=MyDate("2007-3-15")

def f(self):
print self.x

c1=C1()

d=cPickle.dumps(c1)
c2=cPickle.loads(d)
c2.f()

I haven't tried your code but I think that you may need to define a
__reduce__ method in your MyDate class in order to give a clue to the
python as to how to pickle its instances. For more details see:

http://docs.python.org/lib/node321.html

Something like:

class MyDate(date):
...
def __reduce__(self):
return type(self), (self.year, self.month, self.day)

might solve your problem.

HTH

--
Arnaud
Thanks. This works exactly the way you wrote.
Yet I am misunderstanding something. Can't pickle "see" that being
MyDate derived from date it also has to look at variables from date?
When do I need to do this? I am using pickle with a lot more complex
classes without this problem.

Thank you
Paulo
Mar 18 '07 #4
On Mar 17, 11:48 pm, Paulo da Silva <psdasil...@esotericaX.ptXwrote:
Arnaud Delobelle escreveu:On Mar 17, 9:31 pm, Paulo da Silva <psdasil...@esotericaX.ptXwrote:
[snip]
Thanks. This works exactly the way you wrote.
Yet I am misunderstanding something. Can't pickle "see" that being
MyDate derived from date it also has to look at variables from date?
When do I need to do this? I am using pickle with a lot more complex
classes without this problem.
Without the MyDate.__reduce__ method, Python uses the
datetime.date.__reduce__ method to pickle your MyDate instances, then
it uses MyDate.__new__ to unpickle them. But your MyDate.__new__
method does not understand the format of a pickled date. You could
also change the MyDate.__new__ method so that it does understand it,
but it wouldn't be that easy as you want it to accept a string, and
this is the format that dates are pickled in.

--
Arnaud

Mar 18 '07 #5
Arnaud Delobelle escreveu:
On Mar 17, 11:48 pm, Paulo da Silva <psdasil...@esotericaX.ptXwrote:
>Arnaud Delobelle escreveu:On Mar 17, 9:31 pm, Paulo da Silva <psdasil...@esotericaX.ptXwrote:

[snip]
>Thanks. This works exactly the way you wrote.
Yet I am misunderstanding something. Can't pickle "see" that being
MyDate derived from date it also has to look at variables from date?
When do I need to do this? I am using pickle with a lot more complex
classes without this problem.

Without the MyDate.__reduce__ method, Python uses the
datetime.date.__reduce__ method to pickle your MyDate instances, then
it uses MyDate.__new__ to unpickle them. But your MyDate.__new__
method does not understand the format of a pickled date. You could
also change the MyDate.__new__ method so that it does understand it,
but it wouldn't be that easy as you want it to accept a string, and
this is the format that dates are pickled in.

--
Arnaud
I see now. I need to read a little further about this stuff as soon as
I get some time to do it.

Thanks Arnaud.
Mar 18 '07 #6
Paulo da Silva a écrit :
(snip)

Not an answer to your question, just a couple advices:
from datetime import date
import cPickle,string
The string module is mostly deprecated. You should use str type methods
whenever possible (cf below)
class MyDate(date):
def __new__(cls,year,month=None,day=None):
if type(year) is str:
And what if it's a unicode string ?
The correct idiom here is:
if isinstance(year, basestring):
year,month,day=map(int,string.split(year,'-'))
year, month, day = map(int, year.split('-'))
if year < 100:
year += 2000
return date.__new__(cls,year,month,day)
(snip)
Mar 19 '07 #7
Bruno Desthuilliers escreveu:
Paulo da Silva a écrit :
....
>class MyDate(date):
def __new__(cls,year,month=None,day=None):
if type(year) is str:

And what if it's a unicode string ?
The correct idiom here is:
if isinstance(year, basestring):

Thanks.
If I do type(year) I get either int or str (may be unicode for unicode
strings) but never anything like basestring. As a relatively inexperient
in python, how could I know that a 'string' is an instance of
basestring? x=u"xxxxxx"; help(x) says this is unicode based on
basestring but help does not "work" for x="xxxxxxxx".
May be the python tutorial should be upgraded to include these new
concepts. Also covering the basics of __init__/__new__ (why have both?)
would be nice.

Paulo
Mar 19 '07 #8
Paulo da Silva wrote:
As a relatively inexperient
in python, how could I know that a 'string' is an instance of
basestring?
isinstance(x, basestring)

This works because basestring is defined as the
tuple (str, unicode) and isinstance accepts a
tuple of types as well as just a single type.

--
Greg
Mar 20 '07 #9
greg wrote:
Paulo da Silva wrote:
>As a relatively inexperient
in python, how could I know that a 'string' is an instance of
basestring?

isinstance(x, basestring)

This works because basestring is defined as the
tuple (str, unicode) and isinstance accepts a
tuple of types as well as just a single type.
The idea is right, but the detail is completely wrong.

basestring is a *type*.
>>basestring
<type 'basestring'>

It's the base class of which both str and unicode are subclasses.

regards
Steve
--
Steve Holden +44 150 684 7255 +1 800 494 3119
Holden Web LLC/Ltd http://www.holdenweb.com
Skype: holdenweb http://del.icio.us/steve.holden
Recent Ramblings http://holdenweb.blogspot.com

Mar 20 '07 #10
Steve Holden <st***@holdenweb.comwrote:
greg wrote:
Paulo da Silva wrote:
As a relatively inexperient
in python, how could I know that a 'string' is an instance of
basestring?
isinstance(x, basestring)

This works because basestring is defined as the
tuple (str, unicode) and isinstance accepts a
tuple of types as well as just a single type.
The idea is right, but the detail is completely wrong.

basestring is a *type*.
>>basestring
<type 'basestring'>

It's the base class of which both str and unicode are subclasses.
I believe it used to be a tuple back in Python 2.2 (sorry, don't have a
Python 2.2 installation to check this right now).
Alex
Mar 20 '07 #11
Steve Holden wrote:
>>basestring
<type 'basestring'>
You're right, I'm not sure what made me think it
was a tuple. Maybe because people used to write
isinstance(x, (str, unicode)) before basestring
existed.

--
Greg
Mar 20 '07 #12
In article <1h***************************@mac.com>,
Alex Martelli <al***@mac.comwrote:
>Steve Holden <st***@holdenweb.comwrote:
>>
basestring is a *type*.
> >>basestring
<type 'basestring'>

It's the base class of which both str and unicode are subclasses.

I believe it used to be a tuple back in Python 2.2 (sorry, don't have a
Python 2.2 installation to check this right now).
Python 2.2.2 (#1, Feb 24 2003, 19:13:11)
[GCC 3.2.2 20030222 (Red Hat Linux 3.2.2-4)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>basestring
Traceback (most recent call last):
File "<stdin>", line 1, in ?
NameError: name 'basestring' is not defined
--
Aahz (aa**@pythoncraft.com) <* http://www.pythoncraft.com/

"Typing is cheap. Thinking is expensive." --Roy Smith
Mar 20 '07 #13
En Tue, 20 Mar 2007 10:16:30 -0300, Aahz <aa**@pythoncraft.comescribió:
In article <1h***************************@mac.com>,
Alex Martelli <al***@mac.comwrote:
>Steve Holden <st***@holdenweb.comwrote:
>>>
basestring is a *type*.

I believe it used to be a tuple back in Python 2.2 (sorry, don't have a
Python 2.2 installation to check this right now).

Python 2.2.2 (#1, Feb 24 2003, 19:13:11)
[GCC 3.2.2 20030222 (Red Hat Linux 3.2.2-4)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>>basestring
Traceback (most recent call last):
File "<stdin>", line 1, in ?
NameError: name 'basestring' is not defined
On 2.2 it appeared types.StringTypes == (str,unicode)

--
Gabriel Genellina

Mar 20 '07 #14
Aahz <aa**@pythoncraft.comwrote:
In article <1h***************************@mac.com>,
Alex Martelli <al***@mac.comwrote:
Steve Holden <st***@holdenweb.comwrote:
>
basestring is a *type*.

>>basestring
<type 'basestring'>

It's the base class of which both str and unicode are subclasses.
I believe it used to be a tuple back in Python 2.2 (sorry, don't have a
Python 2.2 installation to check this right now).

Python 2.2.2 (#1, Feb 24 2003, 19:13:11)
[GCC 3.2.2 20030222 (Red Hat Linux 3.2.2-4)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>basestring
Traceback (most recent call last):
File "<stdin>", line 1, in ?
NameError: name 'basestring' is not defined
Thanks for double checking on my vague and apparently incorrect
historical memory! Obviously I must have been thinking of some _trick_
whereby one bound basestring to the pair to make isinstance on
basestring work in 2.2 much as it did later, rather than an intrinsic
2.2 feature.
Alex
Mar 20 '07 #15
Paulo da Silva a écrit :
Bruno Desthuilliers escreveu:
>Paulo da Silva a écrit :
...
>>class MyDate(date):
def __new__(cls,year,month=None,day=None):
if type(year) is str:
And what if it's a unicode string ?
The correct idiom here is:
if isinstance(year, basestring):


Thanks.
If I do type(year) I get either int or str (may be unicode for unicode
strings) but never anything like basestring.
It's an abstract class.
As a relatively inexperient
in python, how could I know that a 'string' is an instance of
basestring?
By reading this newsgroup ?-)
x=u"xxxxxx"; help(x) says this is unicode based on
basestring but help does not "work" for x="xxxxxxxx".
help(str)
May be the python tutorial should be upgraded to include these new
concepts. Also covering the basics of __init__/__new__ (why have both?)
It's a long story. But it's a GoodThing(tm) IMVHO.
Mar 21 '07 #16
Paulo da Silva <ps********@esotericaX.ptXwrote:
As a relatively inexperient
in python, how could I know that a 'string' is an instance of
basestring?
I think a good tip for anyone learning Python (or upgrading from an earlier
version) is to do:

dir(__builtins__)

at the interactive prompt and read through the list it produces working out
what each builtin does. Some of them you can ignore(1), some you can file
away as interesting to know they exist but you don't need to know the
details today(2), and others are indispensible(3).

Do that and you'll find basestring (category 2 until you needed to ask your
question) between apply (category 1) and bool (category 3).
Mar 21 '07 #17
En Wed, 21 Mar 2007 07:51:32 -0300, Bruno Desthuilliers
<br********************@wtf.websiteburo.oops.comes cribió:
Paulo da Silva a écrit :
>As a relatively inexperient
in python, how could I know that a 'string' is an instance of
basestring?

By reading this newsgroup ?-)
Or asking Python itself:
>>type("any string").mro()
[<type 'str'>, <type 'basestring'>, <type 'object'>]

--
Gabriel Genellina

Mar 23 '07 #18
Gabriel Genellina escreveu:
En Wed, 21 Mar 2007 07:51:32 -0300, Bruno Desthuilliers
<br********************@wtf.websiteburo.oops.comes cribió:
>Paulo da Silva a écrit :
>>As a relatively inexperient
in python, how could I know that a 'string' is an instance of
basestring?

By reading this newsgroup ?-)

Or asking Python itself:
>>>type("any string").mro()
[<type 'str'>, <type 'basestring'>, <type 'object'>]

--Gabriel Genellina
Good! ;-)
Paulo
Mar 23 '07 #19

This discussion thread is closed

Replies have been disabled for this discussion.

Similar topics

3 posts views Thread by H Jansen | last post: by
9 posts views Thread by Felix Wiemann | last post: by
5 posts views Thread by could ildg | last post: by
5 posts views Thread by Ken Schutte | last post: by
1 post views Thread by Frank Benkstein | last post: by
5 posts views Thread by Sandra-24 | last post: by
4 posts views Thread by Steven D'Aprano | last post: by
3 posts views Thread by Torsten Mohr | last post: by
3 posts views Thread by macaronikazoo | last post: by
By using this site, you agree to our Privacy Policy and Terms of Use.