Still the new hell ...

Paulo da Silva

Sorry to put here too many questions about __init__ __new__
stuff but I always found a new problem when using them.
I have searched for simple __new__ docs on how to do the
basic things but find none.

After trying the solutions people gently posted here
(thanks) I run into new trouble when I go with further
development.

Here is the new situation.

As suggested in a previous post, I used __new__ to
subclass date class but now cPickle/pickle loads
does not work.

from datetime import date
import cPickle,string

class MyDate(date):
def __new__(cls,year,month=None,day=None):
if type(year) is str:
year,month,day=map(int,string.split(year,'-'))
if year<100:
year+=2000
return date.__new__(cls,year,month,day)

class C1(object):
def __init__(self):
self.x=MyDate("2007-3-15")

def f(self):
print self.x

c1=C1()

d=cPickle.dumps(c1)
c2=cPickle.loads(d)
c2.f()

1. year is passed to __new__ as a string but with invalid
contents!

2. If I had a __new__ in class C1, cPickle.loads invokes it
with only one parameter. This forces to allow __new__ accept
only 1 parameter (cls) which I don't want for the normal
situation.

Thanks for your patience.
Paulo

Arnaud Delobelle

On Mar 17, 9:31 pm, Paulo da Silva <psdasil...@esotericaX.ptXwrote:

Sorry to put here too many questions about __init__ __new__
stuff but I always found a new problem when using them.
I have searched for simple __new__ docs on how to do the
basic things but find none.

After trying the solutions people gently posted here
(thanks) I run into new trouble when I go with further
development.

Here is the new situation.

As suggested in a previous post, I used __new__ to
subclass date class but now cPickle/pickle loads
does not work.

from datetime import date
import cPickle,string

class MyDate(date):
def __new__(cls,year,month=None,day=None):
if type(year) is str:
year,month,day=map(int,string.split(year,'-'))
if year<100:
year+=2000
return date.__new__(cls,year,month,day)

class C1(object):
def __init__(self):
self.x=MyDate("2007-3-15")

def f(self):
print self.x

c1=C1()

d=cPickle.dumps(c1)
c2=cPickle.loads(d)
c2.f()

I haven't tried your code but I think that you may need to define a
__reduce__ method in your MyDate class in order to give a clue to the
python as to how to pickle its instances. For more details see:

http://docs.python.org/lib/node321.html

Something like:

class MyDate(date):
...
def __reduce__(self):
return type(self), (self.year, self.month, self.day)

might solve your problem.

HTH

--
Arnaud

Paul McGuire

On Mar 17, 4:31 pm, Paulo da Silva <psdasil...@esotericaX.ptXwrote:

Sorry to put here too many questions about __init__ __new__
stuff but I always found a new problem when using them.
I have searched for simple __new__ docs on how to do the
basic things but find none.

After trying the solutions people gently posted here
(thanks) I run into new trouble when I go with further
development.

Here is the new situation.

As suggested in a previous post, I used __new__ to
subclass date class but now cPickle/pickle loads
does not work.

from datetime import date
import cPickle,string

class MyDate(date):
def __new__(cls,year,month=None,day=None):
if type(year) is str:
year,month,day=map(int,string.split(year,'-'))
if year<100:
year+=2000
return date.__new__(cls,year,month,day)

class C1(object):
def __init__(self):
self.x=MyDate("2007-3-15")

def f(self):
print self.x

c1=C1()

d=cPickle.dumps(c1)
c2=cPickle.loads(d)
c2.f()

1. year is passed to __new__ as a string but with invalid
contents!

2. If I had a __new__ in class C1, cPickle.loads invokes it
with only one parameter. This forces to allow __new__ accept
only 1 parameter (cls) which I don't want for the normal
situation.

Thanks for your patience.
Paulo

Any special reason you have to use __new__ for this factory method?
This version works, without the problems with __new__:

def getDate(*args):
if isinstance(args[0],basestring):
year,month,day = map(int,string.split(args[0],'-'))
else:
year,month,day = args
if year < 100:
year += 2000
return date(year,month,day)

class C1(object):
def __init__(self):
#self.x=MyDate("2007-3-15")
self.x = getDate("2007-3-15")

def f(self):
print self.x
-- Paul

Paulo da Silva

Arnaud Delobelle escreveu:

On Mar 17, 9:31 pm, Paulo da Silva <psdasil...@esotericaX.ptXwrote:

....

>I used __new__ to
subclass date class but now cPickle/pickle loads
does not work.

from datetime import date
import cPickle,string

class MyDate(date):
def __new__(cls,year,month=None,day=None):
if type(year) is str:
year,month,day=map(int,string.split(year,'-'))
if year<100:
year+=2000
return date.__new__(cls,year,month,day)

class C1(object):
def __init__(self):
self.x=MyDate("2007-3-15")

def f(self):
print self.x

c1=C1()

d=cPickle.dumps(c1)
c2=cPickle.loads(d)
c2.f()

I haven't tried your code but I think that you may need to define a
__reduce__ method in your MyDate class in order to give a clue to the
python as to how to pickle its instances. For more details see:

http://docs.python.org/lib/node321.html

Something like:

class MyDate(date):
...
def __reduce__(self):
return type(self), (self.year, self.month, self.day)

might solve your problem.

HTH

--
Arnaud

Thanks. This works exactly the way you wrote.
Yet I am misunderstanding something. Can't pickle "see" that being
MyDate derived from date it also has to look at variables from date?
When do I need to do this? I am using pickle with a lot more complex
classes without this problem.

Thank you
Paulo

Arnaud Delobelle

On Mar 17, 11:48 pm, Paulo da Silva <psdasil...@esotericaX.ptXwrote:

Arnaud Delobelle escreveu:On Mar 17, 9:31 pm, Paulo da Silva <psdasil...@esotericaX.ptXwrote:

[snip]

Thanks. This works exactly the way you wrote.
Yet I am misunderstanding something. Can't pickle "see" that being
MyDate derived from date it also has to look at variables from date?
When do I need to do this? I am using pickle with a lot more complex
classes without this problem.

Without the MyDate.__reduce__ method, Python uses the
datetime.date.__reduce__ method to pickle your MyDate instances, then
it uses MyDate.__new__ to unpickle them. But your MyDate.__new__
method does not understand the format of a pickled date. You could
also change the MyDate.__new__ method so that it does understand it,
but it wouldn't be that easy as you want it to accept a string, and
this is the format that dates are pickled in.

--
Arnaud

Paulo da Silva

Arnaud Delobelle escreveu:

On Mar 17, 11:48 pm, Paulo da Silva <psdasil...@esotericaX.ptXwrote:
>Arnaud Delobelle escreveu:On Mar 17, 9:31 pm, Paulo da Silva <psdasil...@esotericaX.ptXwrote:

[snip]

>Thanks. This works exactly the way you wrote.
Yet I am misunderstanding something. Can't pickle "see" that being
MyDate derived from date it also has to look at variables from date?
When do I need to do this? I am using pickle with a lot more complex
classes without this problem.

Without the MyDate.__reduce__ method, Python uses the
datetime.date.__reduce__ method to pickle your MyDate instances, then
it uses MyDate.__new__ to unpickle them. But your MyDate.__new__
method does not understand the format of a pickled date. You could
also change the MyDate.__new__ method so that it does understand it,
but it wouldn't be that easy as you want it to accept a string, and
this is the format that dates are pickled in.

--
Arnaud

I see now. I need to read a little further about this stuff as soon as
I get some time to do it.

Thanks Arnaud.

Bruno Desthuilliers

Paulo da Silva a écrit :
(snip)

Not an answer to your question, just a couple advices:

from datetime import date
import cPickle,string

The string module is mostly deprecated. You should use str type methods
whenever possible (cf below)

class MyDate(date):
def __new__(cls,year,month=None,day=None):
if type(year) is str:

And what if it's a unicode string ?
The correct idiom here is:
if isinstance(year, basestring):

year,month,day=map(int,string.split(year,'-'))

year, month, day = map(int, year.split('-'))

if year < 100:
year += 2000
return date.__new__(cls,year,month,day)

(snip)

Paulo da Silva

Bruno Desthuilliers escreveu:

Paulo da Silva a écrit :

....

>class MyDate(date):
def __new__(cls,year,month=None,day=None):
if type(year) is str:

And what if it's a unicode string ?
The correct idiom here is:
if isinstance(year, basestring):

Thanks.
If I do type(year) I get either int or str (may be unicode for unicode
strings) but never anything like basestring. As a relatively inexperient
in python, how could I know that a 'string' is an instance of
basestring? x=u"xxxxxx"; help(x) says this is unicode based on
basestring but help does not "work" for x="xxxxxxxx".
May be the python tutorial should be upgraded to include these new
concepts. Also covering the basics of __init__/__new__ (why have both?)
would be nice.

Paulo

greg

Paulo da Silva wrote:

As a relatively inexperient
in python, how could I know that a 'string' is an instance of
basestring?

isinstance(x, basestring)

This works because basestring is defined as the
tuple (str, unicode) and isinstance accepts a
tuple of types as well as just a single type.

--
Greg

Steve Holden

greg wrote:

Paulo da Silva wrote:
>As a relatively inexperient
in python, how could I know that a 'string' is an instance of
basestring?

isinstance(x, basestring)

This works because basestring is defined as the
tuple (str, unicode) and isinstance accepts a
tuple of types as well as just a single type.

The idea is right, but the detail is completely wrong.

basestring is a *type*.

>>basestring

<type 'basestring'>

It's the base class of which both str and unicode are subclasses.

regards
Steve
--
Steve Holden +44 150 684 7255 +1 800 494 3119
Holden Web LLC/Ltd http://www.holdenweb.com
Skype: holdenweb http://del.icio.us/steve.holden
Recent Ramblings http://holdenweb.blogspot.com

Alex Martelli

Steve Holden <st***@holdenweb.comwrote:

greg wrote:
Paulo da Silva wrote:
As a relatively inexperient
in python, how could I know that a 'string' is an instance of
basestring?
isinstance(x, basestring)

This works because basestring is defined as the
tuple (str, unicode) and isinstance accepts a
tuple of types as well as just a single type.
The idea is right, but the detail is completely wrong.

basestring is a *type*.

>>basestring

<type 'basestring'>

It's the base class of which both str and unicode are subclasses.

I believe it used to be a tuple back in Python 2.2 (sorry, don't have a
Python 2.2 installation to check this right now).
Alex

greg

Steve Holden wrote:

>>basestring

<type 'basestring'>

You're right, I'm not sure what made me think it
was a tuple. Maybe because people used to write
isinstance(x, (str, unicode)) before basestring
existed.

--
Greg

Aahz

In article <1h***************************@mac.com>,
Alex Martelli <al***@mac.comwrote:

>Steve Holden <st***@holdenweb.comwrote:
>>
basestring is a *type*.

> >>basestring
<type 'basestring'>

It's the base class of which both str and unicode are subclasses.

I believe it used to be a tuple back in Python 2.2 (sorry, don't have a
Python 2.2 installation to check this right now).

Python 2.2.2 (#1, Feb 24 2003, 19:13:11)
[GCC 3.2.2 20030222 (Red Hat Linux 3.2.2-4)] on linux2
Type "help", "copyright", "credits" or "license" for more information.

>>basestring

Traceback (most recent call last):
File "<stdin>", line 1, in ?
NameError: name 'basestring' is not defined
--
Aahz (aa**@pythoncraft.com) <* http://www.pythoncraft.com/

"Typing is cheap. Thinking is expensive." --Roy Smith

Gabriel Genellina

En Tue, 20 Mar 2007 10:16:30 -0300, Aahz <aa**@pythoncraft.comescribió:

In article <1h***************************@mac.com>,
Alex Martelli <al***@mac.comwrote:
>Steve Holden <st***@holdenweb.comwrote:
>>>
basestring is a *type*.

I believe it used to be a tuple back in Python 2.2 (sorry, don't have a
Python 2.2 installation to check this right now).

Python 2.2.2 (#1, Feb 24 2003, 19:13:11)
[GCC 3.2.2 20030222 (Red Hat Linux 3.2.2-4)] on linux2
Type "help", "copyright", "credits" or "license" for more information.

>>>basestring

Traceback (most recent call last):
File "<stdin>", line 1, in ?
NameError: name 'basestring' is not defined

On 2.2 it appeared types.StringTypes == (str,unicode)

--
Gabriel Genellina

Alex Martelli

Aahz <aa**@pythoncraft.comwrote:

In article <1h***************************@mac.com>,
Alex Martelli <al***@mac.comwrote:
Steve Holden <st***@holdenweb.comwrote:
>
basestring is a *type*.

>>basestring
<type 'basestring'>

It's the base class of which both str and unicode are subclasses.
I believe it used to be a tuple back in Python 2.2 (sorry, don't have a
Python 2.2 installation to check this right now).

Python 2.2.2 (#1, Feb 24 2003, 19:13:11)
[GCC 3.2.2 20030222 (Red Hat Linux 3.2.2-4)] on linux2
Type "help", "copyright", "credits" or "license" for more information.

>basestring

Traceback (most recent call last):
File "<stdin>", line 1, in ?
NameError: name 'basestring' is not defined

Thanks for double checking on my vague and apparently incorrect
historical memory! Obviously I must have been thinking of some _trick_
whereby one bound basestring to the pair to make isinstance on
basestring work in 2.2 much as it did later, rather than an intrinsic
2.2 feature.
Alex

Bruno Desthuilliers

Paulo da Silva a écrit :

Bruno Desthuilliers escreveu:
>Paulo da Silva a écrit :
...

>>class MyDate(date):
def __new__(cls,year,month=None,day=None):
if type(year) is str:
And what if it's a unicode string ?
The correct idiom here is:
if isinstance(year, basestring):

Thanks.
If I do type(year) I get either int or str (may be unicode for unicode
strings) but never anything like basestring.

It's an abstract class.

As a relatively inexperient
in python, how could I know that a 'string' is an instance of
basestring?

By reading this newsgroup ?-)

x=u"xxxxxx"; help(x) says this is unicode based on
basestring but help does not "work" for x="xxxxxxxx".

help(str)

May be the python tutorial should be upgraded to include these new
concepts. Also covering the basics of __init__/__new__ (why have both?)

It's a long story. But it's a GoodThing(tm) IMVHO.

Duncan Booth

Paulo da Silva <ps********@esotericaX.ptXwrote:

As a relatively inexperient
in python, how could I know that a 'string' is an instance of
basestring?

I think a good tip for anyone learning Python (or upgrading from an earlier
version) is to do:

dir(__builtins__)

at the interactive prompt and read through the list it produces working out
what each builtin does. Some of them you can ignore(1), some you can file
away as interesting to know they exist but you don't need to know the
details today(2), and others are indispensible(3).

Do that and you'll find basestring (category 2 until you needed to ask your
question) between apply (category 1) and bool (category 3).

Gabriel Genellina

En Wed, 21 Mar 2007 07:51:32 -0300, Bruno Desthuilliers
<br********************@wtf.websiteburo.oops.comes cribió:

Paulo da Silva a écrit :
>As a relatively inexperient
in python, how could I know that a 'string' is an instance of
basestring?

By reading this newsgroup ?-)

Or asking Python itself:

>>type("any string").mro()

[<type 'str'>, <type 'basestring'>, <type 'object'>]

--
Gabriel Genellina

Paulo da Silva

Gabriel Genellina escreveu:

En Wed, 21 Mar 2007 07:51:32 -0300, Bruno Desthuilliers
<br********************@wtf.websiteburo.oops.comes cribió:

>Paulo da Silva a écrit :
>>As a relatively inexperient
in python, how could I know that a 'string' is an instance of
basestring?

By reading this newsgroup ?-)

Or asking Python itself:

>>>type("any string").mro()

[<type 'str'>, <type 'basestring'>, <type 'object'>]

--Gabriel Genellina

Good! ;-)
Paulo

Still the new hell ...

This discussion thread is closed

Similar topics

Still the __new__ hell ...

This discussion thread is closed

Similar topics

Still the new hell ...