Bug in New Style Classes

[David MacQuigg]

I have what looks like a bug trying to generate new style classes with
a factory function.

class Animal(object): pass
class Mammal(Animal): pass

def newAnimal(bases =(Animal,), dict={}):
class C(object): pass
C.__bases__ = bases
dict['_count'] = 0
C.__dict__ = dict
return C

Canine = newAnimal((Mamm al,))
TypeError: __bases__ assignment: 'Mammal' deallocator differs from
'object'

If I remove the 'object' from the class C(object) statement, then I
get a different, equally puzzling error message:

TypeError: __bases__ items must be classes

The function works only if I remove 'object' from all base classes.

-- Dave

*************** *************** *************** *************** * *
* David MacQuigg, PhD * email: dmq at gain.com * *
* IC Design Engineer * phone: USA 520-721-4583 * * *
* Analog Design Methodologies * * *
* * 9320 East Mikelyn Lane * * *
* VRS Consulting, P.C. * Tucson, Arizona 85710 *
*************** *************** *************** *************** * *

[Michele Simionato]

Michael Hudson <mw*@python.net > wrote in message news:<m3******* *****@pc150.mat hs.bris.ac.uk>. ..

Michele Simionato wrote:

Uhm, I can see that in certain cases it may work, both what if a
class has "object" as base and we change it to "type" ? What if we
change the base class from new style to old style? What if we change
one of the base classes with another one which is instance of a
different metaclass? How do you solve the conflicts? What about
ExtensionClasse s and replacing normal bases with C-coded ones?

You get an exception. Obviously. Did you actually read what I said?

Actually that paragraph was not aimed to you. It was just to
clarify to the OP why there could be issues when changing __bases__.
What could be addressed to you is the question "why make your life
so difficult and force yourself to think to all possible things that
can go wrong when you could just make __bases__ read-only and be done
with it?" ;)

I see a recipe for disasters here, so that's why I thought the
developer removed the option of changing the bases. That would be
understable. What I do not understand is why I cannot change the
__name__ of a function, what could possible go wrong with that??

Beats me too! Maybe it's because you haven't written a patch yet :-)

Cheers,
mwh

Touche' ;)

[Alexander Schmolck]

Michael Hudson <mw*@python.net > writes:

I see a recipe for disasters here, so that's why I thought the
developer removed the option of changing the bases. That would be
understable. What I do not understand is why I cannot change the
__name__ of a function, what could possible go wrong with that??

Beats me too! Maybe it's because you haven't written a patch yet :-)

I seem to recall that it was possible but Guido removed it at some point
because he felt it was not needed (I've occasionally found this restriction to
be a minor annoyance).

'as

[David MacQuigg]

On Fri, 18 Jun 2004 10:18:06 GMT, Michael Hudson <mw*@python.net >
wrote:

David MacQuigg <dm*@gain.com > writes:

On Thu, 17 Jun 2004 11:05:58 GMT, Michael Hudson <mw*@python.net >
wrote:

>mi************ ***@poste.it (Michele Simionato) writes:
>
>> David MacQuigg <dm*@gain.com > wrote in message news:<rm******* *************** **********@4ax. com>...
>> > I have what looks like a bug trying to generate new style classes with
>> > a factory function.
>> >
>> > class Animal(object): pass
>> > class Mammal(Animal): pass
>> >
>> > def newAnimal(bases =(Animal,), dict={}):
>> > class C(object): pass
>> > C.__bases__ = bases
>> > dict['_count'] = 0
>> > C.__dict__ = dict
>> > return C
>> >
>> > Canine = newAnimal((Mamm al,))
>> > TypeError: __bases__ assignment: 'Mammal' deallocator differs from
>> > 'object'
>> >
>> > If I remove the 'object' from the class C(object) statement, then I
>> > get a different, equally puzzling error message:
>> >
>> > TypeError: __bases__ items must be classes
>> >
>> > The function works only if I remove 'object' from all base classes.
>> >
>> > -- Dave
>>
>> This is not a bug. The developers removed the possibility to change
>> the bases of a new-style class.
>
>Bad news for you: I put it back in for 2.3.
>
>If you read the error message, you'll notice that it's phrased to
>suggest that assignment to __bases__ is *sometimes* possible :-)
>
>David's assignment probably should work -- there's a bug on sf about
>this -- but there are definitely situations where assignment to bases
>*shouldn't* be allowed -- e.g. when the so-called 'solid base' changes
>-- but noone's put in the thinking time to make this precise in code.
>Being over-restrictive seems the better course.
This may be just a documentation problem then. The error message is
definitely misleading.

Well, possibly. It's generally assumed that you know what you're
doing if you want to muck with things on this level.

I have no desire to muck with things. The main challenge I face in
teaching Python to non-CIS engineering students, is separating the
useful, simple stuff from the unecessary complexities. Originally, I
had just a simple hierarchy of Animal classes, with each __init__
calling its parent, and a variable _count in each class to keep track
of the total number of instances. This was criticised on another
thread for being not "robust" and a terrible example to show students.

The alternatives offered included factory functions, metaclasses, weak
references, and other complexities, all of which I felt would distract
from the basic presentation, but did have some value in solving
particular problems. So I included them in the Advanced Topics
section. The example above is attempt to generate a class that
automatically includes some attributes like _count, and thus make the
creation of new animal classes more "robust".

The "mucking" is a result of trying to solve a simple problem using
what we already know (__bases__, __dict__, etc.), and avoiding the
introduction of metaclasses. The resulting error messages are
confusing, and in many cases incorrect.

class Animal(object): pass
class C: pass
C.__bases__ = (Animal,) TypeError: __bases__ items must be classes class C(object): pass
C.__bases__ = (Animal,) TypeError: __bases__ assignment: 'Animal' deallocator differs from
'object' class ClassicAnimal: pass
C.__bases__ = (ClassicAnimal, ) TypeError: a new-style class can't have only classic bases C.__bases__ (<type 'object'>,) C.__bases__ = (object, Animal) TypeError: Cannot create a consistent method resolution
order (MRO) for bases object, Animal C.__bases__ = (Animal,object) TypeError: __bases__ assignment: 'Animal' deallocator differs from
'object' C.__bases__ = Animal TypeError: can only assign tuple to C.__bases__, not type
Clarifying patches welcome :-)

I can't even begin to figure out the complexities of what leads to
these strange messagea, but I can provide some test cases like the
above. My general recommedation would be, if you are unsure that the
error message will be understood, provide a more general message and a
unique error number. Something like:

TypeError: Illegal __bases__ assignment: Err#32175

Then the user can either try something different, or search the docs
and comp.lang.pytho n for something very specific. He won't waste time
when the message says "items must be classes" and they already are
classes.
>However, newAnimal could be written like this:
>
>def newAnimal(bases =(Animal,), ns=None):
> if ns is None:
> ns = {}
> ns['_count'] = 0
> return type('C', bases, ns)
>
>which
>
>a) doesn't use the name of a builtin as a variable
>b) doesn't suffer the 'mutable default arguments' problem
>c) is rather less insane
>d) actually works :-) (probably, haven't tested it)

It works great. The only thing I would change is the return line,
making that

globals()[name] = type(name, bases, nsdict)

Ugh!

Is there a better way to avoid spelling out the classname twice. I
worry about subtle errors that could result from simple typos like

BoaConstrictor = newAnimal('BoaC onstritcor', bases, nsdict)

so we don't have to type the name twice when creating a new class.
I've also added an __init__ function. Using the factory is now very
easy:

>>> newAnimal('Dog' ,(Mammal,))
>>> dog1 = Dog()

Hello from __init__ in Dog

>>> Dog._count

1

The main limitation I see in using a factory function like this,
instead of a metaclass,

^^^^^^^^^^^^^^^ ^^^^^^^
What a what? I _really_ don't think you mean metaclass here.

Yes, I do mean metaclasses, and this is one of the few applications I
have found that make sense for non-experts. I've been back-and-forth
a couple of times on the question whether to include metaclasses in my
OOP chapter. Most experts say no. Alex Martelli seems to think this
is an over-reaction. His example in Nutshell is the best I've seen,
however, I just modified it to move his metaclass methods, __init__
and __repr__ to the parent class, MetaBunch, and it runs just fine
using normal inheritance rather than metaclassing. It also seems to
be more efficient in memory usage, although I can't verify that. I'm
just assuming that you get one copy of a method when you use
inheritance, and multiple copies when you put the method in a
metaclass.

is that I can't customize the new animal as easily, because I don't
have an indented block like in a class definition. I've got to call
the newAnimal function, then add a bunch of attributes one at a
time, with fully-qualified names.

Dog.temperature = 102
Dog.pulse = 82
Dog.respiration = 36

Well, there are ways around that, too, eg:

def newAnimal(bases =(Animal,), **kw):
kw['_count'] = 0
return type('C', bases, kw)

Dog = newAnimal('Dog' , (Mammal,), temperature=102 , respiration=36)

If I'm adding methods, it gets even messier, because I've got to
define functions at the module level, then assign them to attributes
of Dog, then maybe delete all the excess names from the module
namespace.

Well, not necessarily. Use the third parameter to the call to type().

I don't see how this avoids the need to define a temporary function at
the module level:

def bmr(t,p,r):
... some complex function of temperature, pulse, and respiration

BoaConstrictor =
newAnimal('BoaC onstrictor',(Re ptile,),tempera ture=102, pulse=82,
repsiration=36, bmr=bmr)
del bmr

Compare this mess to:

class BoaConstrictor( Reptile):
temperature = 102
pulse = 82
respiration = 36
def bmr(t,p,r):
...

Reptile inherits from Animal, which has a metaclass providing all the
common class variables, like _count. All the data and methods unique
to BoaConstrictor are in a normal class definition, not cluttering the
module namespace.

I have one last question. In reviewing all the ways to solve the
problem of creating specialized classes, I see there is a function
new.classobj(na me, bases, dict) which appears to do the same thing as
type(name, bases, dict).

new.classobj () is a holdover from the days of old-style classes,
though I see it creates new-style classes if passed new-style bases...

What is the purpose of classobj()? The name is a little more
self-explanatory than 'type', but using it requires a module import.

Historical cruft, more-or-less.

Then we need a note saying this in section 3.27 of the Library
Reference. Otherwise new code will continue to use these functions
instead of type().

-- Dave