cooperation of buitlin methods usingtsuper

[Matthias Oberlaender]

I would like to adopt the cooperation paradigm in conjunction with builtin
methods and operators, such as len, iter, +, * etc.

But the direct approach does not work with the current implementation of
super. For example, 'len(super(Y, y)' will always result in 'len() of unsized
object'.

As far as I understand, this is because builtins don't use a dynamic lookup
chain, but go directly to the slots for the builtins. However, super returns
an instance of class 'super'. Since all super objects share this class, its
slots will not be filled as one might hope.

My workaround is this: I create a subclass of 'super' on the fly each time I
call 'mysuper'. Look at the definition below. It seems to work. But maybe I
have overlooked something. Is my understanding correct? Is 'mysuper' a good
solution? Possible improvements? (e.g. caching of subclasses)

Thanks for comments!
import new

class X(object):
def __len__(self): return 2222

class Y(X):
def __len__(self): return 1111

def mysuper(cls, inst):
return new.classobj('mysuper', (super,) + cls.__bases__, {})(cls, inst)

y = Y()
try:
print len(super(Y, y))
except Exception, msg:
print msg

try:
print len(mysuper(Y, y))
except Exception, msg:
print msg

Output:

len() of unsized object
2222

--
____ __ _/_/ .
( / / ( / / / /

================================================== ===================
Matthias Oberlaender, DaimlerChrysler AG, Research Center Ulm
RIC/AP (Machine Perception)
Wilhelm-Runge-Str. 11, P.O. Box 2360, 89013 Ulm, Germany
Phone: [phone] Fax: [phone]
Email: ma******************@VOID.daimlerchrysler.com
================================================== ===================

[Michele Simionato]

Matthias Oberlaender <ma******************@REMOVE.daimlerchrysler.com > wrote in message news:<bd**********@news.sns-felb.debis.de>...> I would like to adopt the cooperation paradigm in conjunction with builtin

methods and operators, such as len, iter, +, * etc.
Fine.
But the direct approach does not work with the current implementation of
super. For example, 'len(super(Y, y)' will always result in 'len() of
unsized object'.
Of course, it must give an error! I think you do not understand how
super works. But don't worry, that's quite common ;)
As far as I understand, this is because builtins don't use a dynamic lookup
chain, but go directly to the slots for the builtins. However, super returns
an instance of class 'super'. Since all super objects share this class, its
slots will not be filled as one might hope. My workaround is this: I create a subclass of 'super' on the fly each time I
call 'mysuper'. Look at the definition below. It seems to work. But maybe
I have overlooked something. Is my understanding correct? Is 'mysuper' a
good solution? Possible improvements? (e.g. caching of subclasses) Thanks for comments!
import new class X(object):
def __len__(self): return 2222 class Y(X):
def __len__(self): return 1111 def mysuper(cls, inst):
return new.classobj('mysuper', (super,) + cls.__bases__, {})(cls, inst) y = Y()
try:
print len(super(Y, y))
except Exception, msg:
print msg try:
print len(mysuper(Y, y))
except Exception, msg:
print msg Output: len() of unsized object
2222

I think you should re-read the documentation and
google on the newsgroup for 'super'. The use case
for super is in multiple inheritance, as in this example:

class B(object):
def __len__(self):
print 'called B.__len__'
return 1111

class C(B):
def __len__(self):
print 'called C.__len__'
return super(C,self).__len__()

class D(B):
def __len__(self):
print 'called D.__len__'
return super(D,self).__len__()

class E(C,D):
pass

print len(E())

The output of this is

called C.__len__
called D.__len__
called B.__len__
1111

Do you see why? Feel free to ask if not. I do not understand what behavior
you expect from 'super'. Can you give more details on your specific use case?

P.S. BTW, in Python 2.2+ you can replace ``new.classobj(name,bases,dic)``
with the built-in ``type(name,bases,dic)``.

--
____ __ _/_/ .
( / / ( / / / /
Michele

[Michele Simionato]

Matthias Oberlaender <ma******************@REMOVE.daimlerchrysler.com > wrote in message news:<bd**********@news.sns-felb.debis.de>...> I would like to adopt the cooperation paradigm in conjunction with builtin

methods and operators, such as len, iter, +, * etc.
Fine.
But the direct approach does not work with the current implementation of
super. For example, 'len(super(Y, y)' will always result in 'len() of
unsized object'.
Of course, it must give an error! I think you do not understand how
super works. But don't worry, that's quite common ;)
As far as I understand, this is because builtins don't use a dynamic lookup
chain, but go directly to the slots for the builtins. However, super returns
an instance of class 'super'. Since all super objects share this class, its
slots will not be filled as one might hope. My workaround is this: I create a subclass of 'super' on the fly each time I
call 'mysuper'. Look at the definition below. It seems to work. But maybe
I have overlooked something. Is my understanding correct? Is 'mysuper' a
good solution? Possible improvements? (e.g. caching of subclasses) Thanks for comments!
import new class X(object):
def __len__(self): return 2222 class Y(X):
def __len__(self): return 1111 def mysuper(cls, inst):
return new.classobj('mysuper', (super,) + cls.__bases__, {})(cls, inst) y = Y()
try:
print len(super(Y, y))
except Exception, msg:
print msg try:
print len(mysuper(Y, y))
except Exception, msg:
print msg Output: len() of unsized object
2222

I think you should re-read the documentation and
google on the newsgroup for 'super'. The use case
for super is in multiple inheritance, as in this example:

class B(object):
def __len__(self):
print 'called B.__len__'
return 1111

class C(B):
def __len__(self):
print 'called C.__len__'
return super(C,self).__len__()

class D(B):
def __len__(self):
print 'called D.__len__'
return super(D,self).__len__()

class E(C,D):
pass

print len(E())

The output of this is

called C.__len__
called D.__len__
called B.__len__
1111

Do you see why? Feel free to ask if not. I do not understand what behavior
you expect from 'super'. Can you give more details on your specific use case?

P.S. BTW, in Python 2.2+ you can replace ``new.classobj(name,bases,dic)``
with the built-in ``type(name,bases,dic)``.

--
____ __ _/_/ .
( / / ( / / / /
Michele

[Bengt Richter]

On 3 Jul 2003 07:52:44 -0700, mi**@pitt.edu (Michele Simionato) wrote:

In <22**************************@posting.google.com > Michele Simionato wrote:

> Matthias Oberlaender <ma******************@REMOVE.daimlerchrysler.com >

wrote in message news:<bd**********@news.sns-felb.debis.de>...> I would like
to adopt the cooperation paradigm in conjunction with builtin

> > methods and operators, such as len, iter, +, * etc.
>
> Fine.
>
> > But the direct approach does not work with the current implementation of
> > super. For example, 'len(super(Y, y)' will always result in 'len() of
> > unsized object'.
>
> Of course, it must give an error! I think you do not understand how
> super works. But don't worry, that's quite common ;)

Oh, wait a minute. I think this "of course" is a bit presumptuous.

Uhm... I do realize now that what I wrote sounds quite presumptuous
indeed.
It was not my intention. The "of course" refers to the current
implementation
of ``super`` which does not do what you ask for. To me this was well
known
because of recent threads on the subject by Bjorn Pettersen:

http://groups.google.com/groups?hl=e...ogle.com#link1

http://groups.google.com/groups?hl=e....lang.python.*

You see that for sure you are not the only one who is confused about
``super`` and there are dark corners about it. I myself do not know
nothing about its
implementation.

>
> I think you should re-read the documentation and
> google on the newsgroup for 'super'. The use case
> for super is in multiple inheritance, as in this example:
>
> class B(object):
> def __len__(self):
> print 'called B.__len__' return 1111
>
> class C(B):
> def __len__(self):
> print 'called C.__len__'
> return super(C,self).__len__()

This is exactly the style of syntax I want to avoid! I'd rather write more
concisely 'len(super(C,self))'.

I see now what's your point, which is the same of Pettersen: why

class B(object): def __len__(self): return 1111 class C(B): pass... c=C()
super(C,c).__len__()1111

works, whereas
len(super(C,c))Traceback (most recent call last):
File "<pyshell#7>", line 1, in ?
len(super(C,c))
TypeError: len() of unsized object

does not work? As you say, the reason is that

at least in Python 2.2.1, its is
a matter of fact that builtins/special method names are treated differently
from ordinary class methods in conjunction with super objects. Do you agree?
My implementation of super seems to fix this gap. Perhaps there are some
other people who would appreciate that too. Or are there good
logical/conceptual reasons againts it? This is what I would like to know.

BTW, the same is true for Python2.3b2. I always use the longest form
of ``super``,so this problems does not bother me, nevertheless I
understand
your point.

I think you should submit the issue to python-dev; maybe there are
technical reasons such that it is necessary to treat special methods
differently and this cannot be avoided. In such a case the
documentation should report that
only the long form ``super(C,c).__len__()`` is correct and that users
should
not use ``len(super(C,c))`` (idem for other special methods).

I would also submit a bug report on sourceforge, since at the best
this is
a documentation bug. It does not seem to have been reported before and
has
already bitten at least two persons on c.l.py., therefore it should be
made
known to the developers

ISTM (and I don't know super much) that in the above, if you factor out
x = super(C,c)
and then compare
len(x)
vs
x.__len__()

then the results above suggest that implementation is not
getattr(x,'__len__')()
but effectively more like
getattr(x.__class__,'__len__')(x)

Note:

class B(object): ... def __len__(self): return 1111
... class C(B): pass ... c=C()
super(C,c).__len__() 1111 len(super(C,c)) Traceback (most recent call last):
File "<stdin>", line 1, in ?
TypeError: len() of unsized object

Ok, now let's look at c vs x:
getattr(c,'__len__')() 1111 getattr(c.__class__,'__len__')(c) 1111

vs
getattr(x,'__len__')() 1111 getattr(x.__class__,'__len__')(x)

Traceback (most recent call last):
File "<stdin>", line 1, in ?
AttributeError: type object 'super' has no attribute '__len__'

I guess generally continuing a failed method lookup in x.__class__.__dict__ or equivalent
and trying to find it as an instance attribute might make this super problem
work, but weren't instance methods intentionally bypassed for the new style?

That would mean solving the problem specifically in super somehow, but how? I guess by
fiddling with lookup of methods of/via super instances? I guess it could be done in C.
I'll have to look in typeobject.c more sometime. It looks already tricky ;-)

Regards,
Bengt Richter

[Michele Simionato]

bo**@oz.net (Bengt Richter) wrote in message news:<be**********@216.39.172.122>...

ISTM (and I don't know super much) that in the above, if you factor out
x = super(C,c)
and then compare
len(x)
vs
x.__len__()

then the results above suggest that implementation is not
getattr(x,'__len__')()
but effectively more like
getattr(x.__class__,'__len__')(x)

Note:

>>> class B(object): ... def __len__(self): return 1111
... >>> class C(B): pass ... >>> c=C()
>>> super(C,c).__len__() 1111 >>> len(super(C,c)) Traceback (most recent call last):
File "<stdin>", line 1, in ?
TypeError: len() of unsized object

Ok, now let's look at c vs x:
>>> getattr(c,'__len__')() 1111 >>> getattr(c.__class__,'__len__')(c) 1111

vs
>>> getattr(x,'__len__')() 1111 >>> getattr(x.__class__,'__len__')(x) Traceback (most recent call last):
File "<stdin>", line 1, in ?
AttributeError: type object 'super' has no attribute '__len__'

I guess generally continuing a failed method lookup in x.__class__.__dict__ or equivalent
and trying to find it as an instance attribute might make this super problem
work, but weren't instance methods intentionally bypassed for the new style?

That would mean solving the problem specifically in super somehow, but how? I guess by
fiddling with lookup of methods of/via super instances? I guess it could be done in C.
I'll have to look in typeobject.c more sometime. It looks already tricky ;-)

Regards,
Bengt Richter

To add trickyness to trickyness, I show you a couple of examples where
the same problems appear. These examples (or examples similar to them)
where pointed out to me by Bjorn Pettersen. The real problem seems to be
in the lookup rule for len(x) which is not always equivalent to
x.__len__() when tricks with __getattr__ are performed:

First example: defining __len__ on the object does not work

class E(object):
def __getattr__(self,name):
if name=='__len__': return lambda:0

e=E()
e.__len__() 0 len(e) Traceback ...

Second example: defining __len__ on the class does not work:

class M(type):
def __getattr__(self,name):
if name=='__len__': return lambda self:0

class F: __metaclass__=M
f=F()
F.__len__(f) # okay 0 len(f) # TypeError: len() of unsized object

f.__len__() # AttributeError: 'F' object has no attribute '__len__'

As you see, the problem is that len(x) is not calling x.__len__(),
nor x.__class__.__len__(x); I really would like to know how ``len``
(or ``str``, ``repr``, etc.) work: I think this is the core of
the problem, whereas ``super`` is probably a red herring.

Michele