Hacking with __new__

[Sandra-24]

Ok here's the problem, I'm modifying a 3rd party library (boto) to
have more specific exceptions. I want to change S3ResponseError into
about 30 more specific errors. Preferably I want to do this by
changing as little code as possible. I also want the new exceptions to
be a subclass of the old S3ResponseError so as to not break old code
that catches it. If I meet these two requirements I expect my
modification to make it into boto and then I won't have to worry about
maintaining a seperate version.

So thinking myself clever with python I thought I could change
S3ResponseError to have a __new__ method which returns one of the 30
new exceptions. That way none of the raise S3ResponseError code needs
changing. No problem. The trouble comes with those exceptions being
subclasses of S3ResponseError, because __new__ is called again and
goofs everything up.

I think there may be a way to solve this but after playing around in
the shell for a while, I give up. I'm less concerned with the original
problem than I am curious about the technical challenge. Can anyone
tell me if it's possible to do meet both of my requirements?

Thanks,
-Sandra

Here's my shell code if you want to play with it too (Bar is
S3ResponseError, Zoo is a more specific error, Foo is just the base
class of Bar.)

>>class Foo(object):

.... def __new__(cls, *args):
.... print 'Foo.__new__', len(args)
.... return super(Foo, cls).__new__(cls, *args)
....
.... def __init__(self, a, b, c):
.... print 'Foo.__init__', 3
.... self.a = a
.... self.b = b
.... self.c = c
....

>>class Bar(Foo):

.... def __new__(cls, a, b, c, *args):
.... print 'Bar.__new__', len(args)
.... if args:
.... return super(Bar, cls).__new__(cls, a, b, c, *args)
....
.... return Zoo(a, b, c, 7)
....

>>class Zoo(Bar):

.... def __init__(self, a, b, c, d):
.... print 'Zoo.__init__', 4
.... Foo.__init__(self, a, b, c)
.... self.d = d
....

>>Bar(1,2,3)

Bar.__new__ 0
Bar.__new__ 1
Foo.__new__ 4
Zoo.__init__ 4
Foo.__init__ 3
Traceback (most recent call last):
File "<input>", line 1, in <module>
TypeError: __init__() takes exactly 5 arguments (4 given)

[Duncan Booth]

Sandra-24 <sa***********@yahoo.comwrote:

So thinking myself clever with python I thought I could change
S3ResponseError to have a __new__ method which returns one of the 30
new exceptions. That way none of the raise S3ResponseError code needs
changing. No problem. The trouble comes with those exceptions being
subclasses of S3ResponseError, because __new__ is called again and
goofs everything up.

I think what you want for Bar is something more along the lines:

class Foo(object):
def __new__(cls, *args):
print 'Foo.__new__', len(args)
return super(Foo, cls).__new__(cls, *args)

def __init__(self, a, b, c):
print 'Foo.__init__', 3
self.a = a
self.b = b
self.c = c

class Bar(Foo):
def __new__(cls, a, b, c, *args):
print 'Bar.__new__', len(args)
target = cls
if not args:
cls = Zoo
obj = super(Bar, cls).__new__(cls, a, b, c, *args)
if args:
return obj

obj.__init__(a, b, c, 7)

class Zoo(Bar):
def __init__(self, a, b, c, d):
print 'Zoo.__init__', 4
Foo.__init__(self, a, b, c)
self.d = d

Bar(1,2,3)

Output from this is:

Bar.__new__ 0
Foo.__new__ 3
Zoo.__init__ 4
Foo.__init__ 3

Bar can instantiate a subclass, but to do that it wants to call the
constructor __new__ only for the base classes, and because it has
changed the type of the object being constructed it has to call __init__
explicitly.

[Bruno Desthuilliers]

Duncan Booth a écrit :
(snip)

I think what you want for Bar is something more along the lines:

(snip)

class Bar(Foo):
def __new__(cls, a, b, c, *args):
print 'Bar.__new__', len(args)
target = cls

You don't use 'target' anywhere...

if not args:
cls = Zoo
obj = super(Bar, cls).__new__(cls, a, b, c, *args)
if args:
return obj

obj.__init__(a, b, c, 7)

IIRC, __new__ is supposed to return the newly created object - which you
are not doing here.

class Bar(Foo):
def __new__(cls, a, b, c, *args):
print 'Bar.__new__', len(args)
if not args:
cls = Zoo
obj = super(Bar, cls).__new__(cls, a, b, c, *args)
if not args:
obj.__init__(a, b, c, 7)
return obj

[Sandra-24]

On Jul 24, 5:20 am, Bruno Desthuilliers <bruno.
[e-mail]:

IIRC, __new__ is supposed to return the newly created object - which you
are not doing here.

class Bar(Foo):
def __new__(cls, a, b, c, *args):
print 'Bar.__new__', len(args)
if not args:
cls = Zoo
obj = super(Bar, cls).__new__(cls, a, b, c, *args)
if not args:
obj.__init__(a, b, c, 7)
return obj

Thanks guys, but you are right Bruno, you have to return the newly
created object or you get:

>>b = Bar(1,2,3)

Bar.__new__ 0
Foo.__new__ 3
Zoo.__init__ 4
Foo.__init__ 3

>>b is None

True

However, if you return the object you get:

>>b = Bar(1, 2, 3)

Bar.__new__ 0
Foo.__new__ 3
Zoo.__init__ 4
Foo.__init__ 3
Traceback (most recent call last):
File "<input>", line 1, in <module>
TypeError: __init__() takes exactly 5 arguments (4 given)

Which is the same blasted error, because it seems to want to call init
on the returned object and it's calling it with 4 args :( Is there any
way around that?

Thanks,
-Sandra

[Duncan Booth]

Bruno Desthuilliers <br********************@wtf.websiteburo.oops.com >
wrote:

Duncan Booth a écrit :
(snip)

>I think what you want for Bar is something more along the lines:

(snip)

>class Bar(Foo):
def __new__(cls, a, b, c, *args):
print 'Bar.__new__', len(args)
target = cls

You don't use 'target' anywhere...

> if not args:
cls = Zoo
obj = super(Bar, cls).__new__(cls, a, b, c, *args)
if args:
return obj

obj.__init__(a, b, c, 7)

IIRC, __new__ is supposed to return the newly created object - which you
are not doing here.

Yup. I committed that cardinal sin of checking the code and then tidying it
up by hand but not checking the tidied version. :(

[Chris Mellon]

On 7/24/07, Sandra-24 <sa***********@yahoo.comwrote:

On Jul 24, 5:20 am, Bruno Desthuilliers <bruno.
[e-mail]:

IIRC, __new__ is supposed to return the newly created object - which you
are not doing here.

class Bar(Foo):
def __new__(cls, a, b, c, *args):
print 'Bar.__new__', len(args)
if not args:
cls = Zoo
obj = super(Bar, cls).__new__(cls, a, b, c, *args)
if not args:
obj.__init__(a, b, c, 7)
return obj

Thanks guys, but you are right Bruno, you have to return the newly
created object or you get:

>b = Bar(1,2,3)

Bar.__new__ 0
Foo.__new__ 3
Zoo.__init__ 4
Foo.__init__ 3

>b is None

True

However, if you return the object you get:

>b = Bar(1, 2, 3)

Bar.__new__ 0
Foo.__new__ 3
Zoo.__init__ 4
Foo.__init__ 3
Traceback (most recent call last):
File "<input>", line 1, in <module>
TypeError: __init__() takes exactly 5 arguments (4 given)

Which is the same blasted error, because it seems to want to call init
on the returned object and it's calling it with 4 args :( Is there any
way around that?

__init__ is going to be called with the arguments passed to __new__
and there doesn't seem to be any way to affect that. Your only option
to make it not happen is for Zoo not to be a subclass of Bar.

Possible workarounds:
Each subclass of Bar needs a corresponding __new__

Each subclass of Bar needs a cooperating __init__ that detects (via
optional arguments) if it's being called by Bar.__new__ or directly
and noops in the second case.