Conditionally implementing __iter__ in new style classes

[Thomas Heller]

I'm trying to implement __iter__ on an abstract base class while I don't
know whether subclasses support that or not.
Hope that makes sense, if not, this code should be clearer:

class Base:
def __getattr__(self, name):
if name == "__iter__" and hasattr(self, "Iterator"):
return self.Iterator
raise AttributeError, name

class Concrete(Base):
def Iterator(self):
yield 1
yield 2
yield 3

The idea is that if a subclass of Base defines an 'Iterator' method,
instances are iterable. They are not iterable otherwise.

The above gives the expected behaviour: iter(Base()) raises a
"TypeError: iteration over non-sequence", and iter(Concrete()) returns a
generator.

If, however, I make Base a newstyle class, this will not work any
longer. __getattr__ is never called for "__iter__" (neither is
__getattribute__, btw). Probably this has to do with data descriptors
and non-data descriptors, but I'm too tired at the moment to think
further about this.

Is there any way I could make the above code work with new style
classes?

Thanks,

Thomas

[Thomas Heller]

Thomas Heller <th*****@python.net> writes:

I'm trying to implement __iter__ on an abstract base class while I don't
know whether subclasses support that or not.
Hope that makes sense, if not, this code should be clearer:

class Base:
def __getattr__(self, name):
if name == "__iter__" and hasattr(self, "Iterator"):
return self.Iterator
raise AttributeError, name

class Concrete(Base):
def Iterator(self):
yield 1
yield 2
yield 3

The idea is that if a subclass of Base defines an 'Iterator' method,
instances are iterable. They are not iterable otherwise.

The above gives the expected behaviour: iter(Base()) raises a
"TypeError: iteration over non-sequence", and iter(Concrete()) returns a
generator.

If, however, I make Base a newstyle class, this will not work any
longer. __getattr__ is never called for "__iter__" (neither is
__getattribute__, btw). Probably this has to do with data descriptors
and non-data descriptors, but I'm too tired at the moment to think
further about this.

Is there any way I could make the above code work with new style
classes?

I forgot to mention this: The Base class also implements a __getitem__
method which should be used for iteration if the .Iterator method in the
subclass is not available. So it seems impossible to raise an exception
in the __iter__ method if .Iterator is not found - __iter__ MUST return
an iterator if present.

Thomas

[harold fellermann]

> I'm trying to implement __iter__ on an abstract base class while I

don't
know whether subclasses support that or not.
Hope that makes sense, if not, this code should be clearer:

class Base:
def __getattr__(self, name):
if name == "__iter__" and hasattr(self, "Iterator"):
return self.Iterator
raise AttributeError, name

class Concrete(Base):
def Iterator(self):
yield 1
yield 2
yield 3

I don't know how to achieve it, but why don't you simply use

class Base:
pass

class Concrete(Base):
def __iter__(self) :
yield 1
yield 2
yield 3
What is the advantage to have a baseclass that essentially does
some method renaming __iter__ ==> Iterator?

- harold -

--
Always remember that you are unique;
just like everyone else.
--

[infidel]

I'm not sure I understand why you would want to. Just don't define
__iter__ on your newstyle class and you'll get the expected behavior.

[infidel]

Why not define an Iterator method in your Base class that does the
iteration using __getitem__, and any subclass that wants to do
something else just defines its own Iterator method? For that matter,
you could just use the __iter__ methods of Base and Concrete instead of
a separate method.

[infidel]

Something like this:

class Base(object): .... def __getitem__(self, key):
.... return key
.... def __iter__(self):
.... yield self[1]
.... yield self['foo']
.... yield self[3.0]
.... class ConcreteIterable(Base): .... def __iter__(self):
.... yield True
.... yield 'Blue'
.... yield 'Foo'
.... class ConcreteNotIterable(Base): .... pass
.... [x for x in Base()] [1, 'foo', 3.0] [x for x in ConcreteIterable()] [True, 'Blue', 'Foo'] [x for x in ConcreteNotIterable()] [1, 'foo', 3.0]

[Leif K-Brooks]

Thomas Heller wrote:

I forgot to mention this: The Base class also implements a __getitem__
method which should be used for iteration if the .Iterator method in the
subclass is not available. So it seems impossible to raise an exception
in the __iter__ method if .Iterator is not found - __iter__ MUST return
an iterator if present.

def Iterator(self):
for index in xrange(len(self)):
yield self[index]

def __iter__(self):
return self.Iterator()

....and then override Iterator in subclasses. But this raises the
question of why you need to use a specially-named method instead of
having subclasses override the __iter__.

[Peter Otten]

Thomas Heller wrote:

I'm trying to implement __iter__ on an abstract base class while I don't
know whether subclasses support that or not.
Hope that makes sense, if not, this code should be clearer:

class Base:
def __getattr__(self, name):
if name == "__iter__" and hasattr(self, "Iterator"):
return self.Iterator
raise AttributeError, name Is there any way I could make the above code work with new style
classes?

Obligatory metaclass approach:

class Base:
class __metaclass__(type):
def __new__(mcl, name, bases, classdict):
try:
classdict["__iter__"] = classdict["Iterator"]
except KeyError:
pass
return type.__new__(mcl, name, bases, classdict)

class Alpha(Base):
def Iterator(self): yield 42

class Beta(Base):
def __getitem__(self, index):
return [1, 2, 3, "ganz viele"][index]
for item in Alpha(): print item
for item in Beta(): print item,
print

Peter

[Bengt Richter]

On Wed, 06 Jul 2005 17:57:42 +0200, Thomas Heller <th*****@python.net> wrote:

I'm trying to implement __iter__ on an abstract base class while I don't
know whether subclasses support that or not.
Hope that makes sense, if not, this code should be clearer:

class Base:
def __getattr__(self, name):
if name == "__iter__" and hasattr(self, "Iterator"):
return self.Iterator
raise AttributeError, name

class Concrete(Base):
def Iterator(self):
yield 1
yield 2
yield 3

The idea is that if a subclass of Base defines an 'Iterator' method,
instances are iterable. They are not iterable otherwise.

The above gives the expected behaviour: iter(Base()) raises a
"TypeError: iteration over non-sequence", and iter(Concrete()) returns a
generator.

If, however, I make Base a newstyle class, this will not work any
longer. __getattr__ is never called for "__iter__" (neither is
__getattribute__, btw). Probably this has to do with data descriptors
and non-data descriptors, but I'm too tired at the moment to think
further about this.

Is there any way I could make the above code work with new style
classes?

Will a property or custom descriptor do what you want? E.g.

class Base(object): ... def __getIter(self):
... if hasattr(self, "Iterator"):
... return self.Iterator
... raise AttributeError, name
... __iter__ = property(__getIter)
... class Concrete(Base): ... def Iterator(self):
... yield 1
... yield 2
... yield 3
... iter(Base()) Traceback (most recent call last):
File "<stdin>", line 1, in ?
TypeError: iteration over non-sequence iter(Concrete()) <generator object at 0x02EF152C> list(iter(Concrete()))

[1, 2, 3]

Regards,
Bengt Richter

[Thomas Heller]

bo**@oz.net (Bengt Richter) writes:

On Wed, 06 Jul 2005 17:57:42 +0200, Thomas Heller <th*****@python.net> wrote:

I'm trying to implement __iter__ on an abstract base class while I don't
know whether subclasses support that or not.

Will a property or custom descriptor do what you want? E.g.
>>> class Base(object): ... def __getIter(self):
... if hasattr(self, "Iterator"):
... return self.Iterator
... raise AttributeError, name
... __iter__ = property(__getIter)
... >>> class Concrete(Base): ... def Iterator(self):
... yield 1
... yield 2
... yield 3
... >>> iter(Base()) Traceback (most recent call last):
File "<stdin>", line 1, in ?
TypeError: iteration over non-sequence >>> iter(Concrete()) <generator object at 0x02EF152C> >>> list(iter(Concrete()))

[1, 2, 3]

Yep, that's exactly what I need - thanks.

Thomas

[Bengt Richter]

On Thu, 07 Jul 2005 09:51:42 +0200, Thomas Heller <th*****@python.net> wrote:

bo**@oz.net (Bengt Richter) writes:

On Wed, 06 Jul 2005 17:57:42 +0200, Thomas Heller <th*****@python.net> wrote:

I'm trying to implement __iter__ on an abstract base class while I don't
know whether subclasses support that or not.

Will a property or custom descriptor do what you want? E.g.

>>> class Base(object): ... def __getIter(self):
... if hasattr(self, "Iterator"):
... return self.Iterator
... raise AttributeError, name
... __iter__ = property(__getIter)

[...]
Yep, that's exactly what I need - thanks.

BTW, I forgot to mention that you could use property as a decorator
in the above single-argument case:

class Base(object): ... @property
... def __iter__(self):
... if hasattr(self, "Iterator"):
... return self.Iterator
... raise AttributeError, name
... class Concrete(Base): ... def Iterator(self):
... yield 1
... yield 2
... yield 3
... iter(Base()) Traceback (most recent call last):
File "<stdin>", line 1, in ?
TypeError: iteration over non-sequence iter(Concrete()) <generator object at 0x02EF152C> list(iter(Concrete())) [1, 2, 3]

Hope there isn't a gotcha for your use case in the way an instance attribute
of the same name is allowed. A custom descriptor could eliminate that.
inst = Concrete()
list(iter(inst)) [1, 2, 3] inst.__init__ = 'abc'
list(iter(inst)) [1, 2, 3] inst.__init__

'abc'

Regards,
Bengt Richter

[Dieter Maurer]

Thomas Heller <th*****@python.net> writes on Wed, 06 Jul 2005 18:07:10 +0200:

Thomas Heller <th*****@python.net> writes:
...

class Base:
def __getattr__(self, name):
if name == "__iter__" and hasattr(self, "Iterator"):
return self.Iterator
raise AttributeError, name

class Concrete(Base):
def Iterator(self):
yield 1

...

If, however, I make Base a newstyle class, this will not work any
longer. __getattr__ is never called for "__iter__" (neither is
__getattribute__, btw). Probably this has to do with data descriptors
and non-data descriptors, but I'm too tired at the moment to think
further about this.

Is there any way I could make the above code work with new style
classes?

I forgot to mention this: The Base class also implements a __getitem__
method which should be used for iteration if the .Iterator method in the
subclass is not available. So it seems impossible to raise an exception
in the __iter__ method if .Iterator is not found - __iter__ MUST return
an iterator if present.

Then, it should return an interator (a new object) that uses
the "__getitem__" method to iterate.
Dieter

[Thomas Heller]

bo**@oz.net (Bengt Richter) writes:

On Thu, 07 Jul 2005 09:51:42 +0200, Thomas Heller <th*****@python.net> wrote:

bo**@oz.net (Bengt Richter) writes:

On Wed, 06 Jul 2005 17:57:42 +0200, Thomas Heller <th*****@python.net> wrote:

I'm trying to implement __iter__ on an abstract base class while I don't
know whether subclasses support that or not.

Will a property or custom descriptor do what you want? E.g.

>>> class Base(object):
... def __getIter(self):
... if hasattr(self, "Iterator"):
... return self.Iterator
... raise AttributeError, name
... __iter__ = property(__getIter) [...]

Yep, that's exactly what I need - thanks.

BTW, I forgot to mention that you could use property as a decorator
in the above single-argument case:
>>> class Base(object): ... @property
... def __iter__(self):
... if hasattr(self, "Iterator"):
... return self.Iterator
... raise AttributeError, name

Of course. I didn't spot this, but I cannot use this anyway for 2.3
compatibility.
... >>> class Concrete(Base): ... def Iterator(self):
... yield 1
... yield 2
... yield 3
... >>> iter(Base()) Traceback (most recent call last):
File "<stdin>", line 1, in ?
TypeError: iteration over non-sequence >>> iter(Concrete()) <generator object at 0x02EF152C> >>> list(iter(Concrete())) [1, 2, 3]

Hope there isn't a gotcha for your use case in the way an instance attribute
of the same name is allowed. A custom descriptor could eliminate that.
>>> inst = Concrete()
>>> list(iter(inst)) [1, 2, 3] >>> inst.__init__ = 'abc'
>>> list(iter(inst)) [1, 2, 3] >>> inst.__init__

'abc'

I don't understand what you mean here. A __iter__ instance attribute?

Thomas

[Bengt Richter]

On Thu, 07 Jul 2005 22:04:31 +0200, Thomas Heller <th*****@python.net> wrote:

bo**@oz.net (Bengt Richter) writes:

On Thu, 07 Jul 2005 09:51:42 +0200, Thomas Heller <th*****@python.net> wrote:

bo**@oz.net (Bengt Richter) writes:

On Wed, 06 Jul 2005 17:57:42 +0200, Thomas Heller <th*****@python.net> wrote:

>I'm trying to implement __iter__ on an abstract base class while I don't
>know whether subclasses support that or not.

Will a property or custom descriptor do what you want? E.g.

>>> class Base(object):
... def __getIter(self):
... if hasattr(self, "Iterator"):
... return self.Iterator
... raise AttributeError, name
... __iter__ = property(__getIter)

[...]

Yep, that's exactly what I need - thanks.

BTW, I forgot to mention that you could use property as a decorator
in the above single-argument case:

>>> class Base(object):

... @property
... def __iter__(self):
... if hasattr(self, "Iterator"):
... return self.Iterator
... raise AttributeError, name

Of course. I didn't spot this, but I cannot use this anyway for 2.3
compatibility.

...

>>> class Concrete(Base):

... def Iterator(self):
... yield 1
... yield 2
... yield 3
...

>>> iter(Base())

Traceback (most recent call last):
File "<stdin>", line 1, in ?
TypeError: iteration over non-sequence

>>> iter(Concrete())

<generator object at 0x02EF152C>

>>> list(iter(Concrete()))

[1, 2, 3]

Hope there isn't a gotcha for your use case in the way an instance attribute
of the same name is allowed. A custom descriptor could eliminate that.

>>> inst = Concrete()
>>> list(iter(inst))

[1, 2, 3]

>>> inst.__init__ = 'abc'
>>> list(iter(inst))

[1, 2, 3]

>>> inst.__init__

'abc'

I don't understand what you mean here. A __iter__ instance attribute?

Yes, but it seems very unlikely to cause a problem, especially since iter(inst)
bypasses it, as you probably would want. In other words, never mind ;-)

Regards,
Bengt Richter