Order in metaclass

[Nicolas Fleury]

In the following example:

class MyMetaclass(type): pass
class MyBaseType(object): __metaclass__ = MyMetaclass
class MyType(MyBaseType):
x = 4
y = 5
z = 6

Is there any way to modify MyMetaclass to keep the order of x,y,z somewhere?

Thx and regards,
Nicolas

[Jeremy Bowers]

On Tue, 12 Oct 2004 15:37:22 -0400, Nicolas Fleury wrote:

In the following example:

class MyMetaclass(type): pass
class MyBaseType(object): __metaclass__ = MyMetaclass
class MyType(MyBaseType):
x = 4
y = 5
z = 6

Is there any way to modify MyMetaclass to keep the order of x,y,z somewhere?

Thx and regards,
Nicolas

What order?

Yeah, they are listed in the source in a particular order, but there is
absolutely no difference between the six permutations of that order, in
theory or in fact, since the vars end up in a dict.

If you want a concept of "order", you're going to have do define one
yourself. What do you want to do, exactly?

I'm going to guess you're trying to get the vars out in order upon an
iteration, which may not be right but extends to other cases as well
fairly cleanly.

The easy solution is a list:

class MyClass(object):
classData = [4, 5, 6]
offsets = {'x': 0, 'y': 1, 'z':2}
def getVarMaker(var):
def getVar(self):
return self.classData[self.offsets[var]]
return getVar
def setVarMaker(var):
def setVar(self, value):
self.classData[self.offsets[var]] = value
return setVar
x = property(getVarMaker('x'), setVarMaker('x'))
y = property(getVarMaker('y'), setVarMaker('y'))
z = property(getVarMaker('z'), setVarMaker('z'))
def __iter__(self):
for value in self.classData:
yield value

A metaclass could take classData and offsets and make the access vars for
you.
--------
def getVarMaker(var):
def getVar(self):
return self.classData[self.offsets[var]]
return getVar

def setVarMaker(var):
def setVar(self, value):
self.classData[self.offsets[var]] = value
return setVar

class OrderedClassAttributes(type):
def __init__(cls, name, bases, dict):
super(OrderedClassAttributes, cls).__init__(name, bases, dict)
for name in cls.offsets:
setattr(cls, name, property(getVarMaker(name),
setVarMaker(name)))

class MyClass(object):
classData = [4, 5, 6]
offsets = {'x': 0, 'y': 1, 'z':2}
__metaclass__ = OrderedClassAttributes
def __iter__(self):
for value in self.classData:
yield value

---------
(Assuming the above is copy/pasted into Python:)

m = MyClass()
m.x 4 m.y 5 m.z 6 m.y = 33
m.y 33 list(m)

[4, 33, 6]
---------

Lightly tested but working as I designed.

Is that what you wanted? If not, feel free to ask more. There are
certainly other options but without more data, it is hard to know what you
need.

This is a pretty good metaclass use, I think.

[Michele Simionato]

Nicolas Fleury <ni******@yahoo.com_remove_the_> wrote in message news:<Rn********************@news20.bellglobal.com >...

In the following example:

class MyMetaclass(type): pass
class MyBaseType(object): __metaclass__ = MyMetaclass
class MyType(MyBaseType):
x = 4
y = 5
z = 6

Is there any way to modify MyMetaclass to keep the order of x,y,z somewhere?

Thx and regards,
Nicolas

Short answer: No.

x, y, z are in the class dictionary. dictionaries have no order.

You could use something like

class MyType(MyBaseType):
__mylist__ = [("x", 4), ("y", 5), ("z",6)]

and use the metaclass to set the class attributes x,y and z for you.
Michele Simionato

[Peter Otten]

Nicolas Fleury wrote:

In the following example:

class MyMetaclass(type): pass
class MyBaseType(object): __metaclass__ = MyMetaclass
class MyType(MyBaseType):
x = 4
y = 5
z = 6

Is there any way to modify MyMetaclass to keep the order of x,y,z
somewhere?

If you want to record the order of these definitions, you need to pass a
custom dictionary that keeps track of assignments in the class generation
code (basically a normal python function comprising the class suite).
Unfortunately that dictionary - which you see later as the classdict
parameter of the metaclass __new__() method - is always a dict created in C,
as was recently discussed on c.l.py (sorry, but all keywords I remember are
'metaclass' and 'martelli' - not very selective :-). Below is my (clumsy)
attempt for a workaround:

import itertools

class OrderedValue(object):
newIndex = itertools.count(1).next
def __init__(self, value):
self.value = value
self.index = self.newIndex()

class Meta(type):
def __new__(mcl, name, bases, classdict):
assert "ordered_names" not in classdict

values = []
for (n, v) in classdict.iteritems():
try:
v, i = v.value, v.index
except AttributeError:
pass
else:
values.append((i, n, v))
values.sort()

ordered_names = []
for (i, n, v) in values:
ordered_names.append(n)
classdict[n] = v
classdict["ordered_names"] = ordered_names

return type.__new__(mcl, name, bases, classdict)
class Base:
__metaclass__ = Meta

class Demo(Base):
alpha = 0
beta = OrderedValue(1)
gamma = OrderedValue(17)
delta = OrderedValue(3)

print Demo.ordered_names
print Demo.alpha, Demo.beta
Peter

[Bengt Richter]

On Wed, 13 Oct 2004 09:04:03 +0200, Peter Otten <__*******@web.de> wrote:

Nicolas Fleury wrote:

In the following example:

class MyMetaclass(type): pass
class MyBaseType(object): __metaclass__ = MyMetaclass
class MyType(MyBaseType):
x = 4
y = 5
z = 6

Is there any way to modify MyMetaclass to keep the order of x,y,z
somewhere?

If you want to record the order of these definitions, you need to pass a
custom dictionary that keeps track of assignments in the class generation
code (basically a normal python function comprising the class suite).
Unfortunately that dictionary - which you see later as the classdict
parameter of the metaclass __new__() method - is always a dict created in C,
as was recently discussed on c.l.py (sorry, but all keywords I remember are
'metaclass' and 'martelli' - not very selective :-). Below is my (clumsy)
attempt for a workaround:

import itertools

class OrderedValue(object):
newIndex = itertools.count(1).next
def __init__(self, value):
self.value = value
self.index = self.newIndex()

class Meta(type):
def __new__(mcl, name, bases, classdict):
assert "ordered_names" not in classdict

values = []
for (n, v) in classdict.iteritems():
try:
v, i = v.value, v.index
except AttributeError:
pass
else:
values.append((i, n, v))
values.sort()

ordered_names = []
for (i, n, v) in values:
ordered_names.append(n)
classdict[n] = v
classdict["ordered_names"] = ordered_names

return type.__new__(mcl, name, bases, classdict)
class Base:
__metaclass__ = Meta

class Demo(Base):
alpha = 0
beta = OrderedValue(1)
gamma = OrderedValue(17)
delta = OrderedValue(3)

print Demo.ordered_names
print Demo.alpha, Demo.beta

Or, an ugly hack that might work for a while, depending on how
co_names is really generated. It seems in order of occurrence
(including right hand sides of assignment, but still top down) ...

import sys
def getnames(): return sys._getframe(1).f_code.co_names ... def MC(cname, cbases, cdict): ... names = cdict.get('ordic',[])
... names = [name for name in names if name in cdict]
... cdict['ordic'] = dict([(name,i) for i,name in enumerate(names)])
... return type(cname, cbases, cdict)
... class C(object): ... __metaclass__ = MC # really a function shortcut
... x = 123
... y = sys
... z = 0
... ordic = getnames()
... C.ordic {'ordic': 5, '__module__': 0, '__metaclass__': 1, 'y': 3, 'x': 2, 'z': 4}
class D(object): ... __metaclass__ = MC # really a function shortcut
... x = 123
... def m(self): pass
... def sm(): print getnames()
... sm = staticmethod(sm)
... ordic = getnames()
... D.ordic {'ordic': 5, '__module__': 0, '__metaclass__': 1, 'm': 3, 'sm': 4, 'x': 2} D.sm <function sm at 0x00906030> D.sm()

('getnames',)

Regards,
Bengt Richter

[Michael Hudson]

bo**@oz.net (Bengt Richter) writes:

Nicolas Fleury wrote:

In the following example:

class MyMetaclass(type): pass
class MyBaseType(object): __metaclass__ = MyMetaclass
class MyType(MyBaseType):
x = 4
y = 5
z = 6

Is there any way to modify MyMetaclass to keep the order of x,y,z
somewhere?

[snip]
Or, an ugly hack that might work for a while, depending on how
co_names is really generated. It seems in order of occurrence
(including right hand sides of assignment, but still top down) ...

I don't know whether to laugh or cry! Nice one.

Cheers,
mwh

--
<lament> Slashdot karma, unfortunately, is not real karma, because
it doesn't involve the death of the people who have it
-- from Twisted.Quotes

[Peter Otten]

Bengt Richter wrote:

Or, an ugly hack that might work for a while, depending on how
co_names is really generated. It seems in order of occurrence
(including right hand sides of assignment, but still top down) ...

Â*importÂ*sys
Â*defÂ*getnames():Â*returnÂ*sys._getframe(1).f_ code.co_names ...Â*defÂ*MC(cname,Â*cbases,Â*cdict): ...Â*Â*Â*Â*Â*namesÂ*=Â*cdict.get('ordic',[])
...Â*Â*Â*Â*Â*namesÂ*=Â*[nameÂ*forÂ*nameÂ*inÂ*namesÂ*ifÂ*nameÂ*inÂ*cdict]
...Â*Â*Â*Â*Â*cdict['ordic']Â*=Â*dict([(name,i)Â*forÂ*i,nameÂ*inÂ*enumerate(names)])
...Â*Â*Â*Â*Â*returnÂ*type(cname,Â*cbases,Â*cdict)
...Â*classÂ*C(object): ...Â*Â*Â*Â*Â*__metaclass__Â*=Â*MCÂ*#Â*reallyÂ*aÂ*f unctionÂ*shortcut
...Â*Â*Â*Â*Â*xÂ*=Â*123
...Â*Â*Â*Â*Â*yÂ*=Â*sys
...Â*Â*Â*Â*Â*zÂ*=Â*0
...Â*Â*Â*Â*Â*ordicÂ*=Â*getnames()
...Â*C.ordic

{'ordic':Â*5,Â*'__module__':Â*0,Â*'__metaclass__': Â*1,Â*'y':Â*3,Â*'x':Â*2,Â*'z':Â*4}

A metaclass /function/ and sys._getframe() exercised on a class definition
- I think you have raised the bar for what qualifies as a hack :-)

Peter

[Carlos Ribeiro]

On Tue, 12 Oct 2004 15:37:22 -0400, Nicolas Fleury
<ni******@yahoo.com_remove_the_> wrote:

In the following example:

class MyMetaclass(type): pass
class MyBaseType(object): __metaclass__ = MyMetaclass
class MyType(MyBaseType):
x = 4
y = 5
z = 6

Is there any way to modify MyMetaclass to keep the order of x,y,z somewhere?

Well, some people have already pointed to you about a recent
discussion on this topic. I was the one who started it all :-) What I
have learned is:

-- Due to the way the class statement is handled, it's not possible
(without really black magic) to guarantee ordering in a transparent
way. If you need to put attributes in order, you have either to
provide a list with all attributes in order, or use derive your
attributes from a base class that provide some ordering mechanism of
its own.

-- You have to include an extra member into your class to record the
order. I have chosen to include a list, named "_fields", with all
attribute _names_. Storing the attribute name is guaranteed way to
keep the ordering even when a descendant override an attribute, or
when the user modify its value.

I have written a small package called GenericTemplates, that works all
the magic needed. It's the basis for a much more ambitious package for
'declarative-style' data structures using class statements. It handles
nested classes also, and keeps the ordering of all attributes that
inherit from a AbstractAttribute class (it includes nested classes
that inherit from GenericTemplate, and also OrderedAttributes and
TypedAttributes). I have a page for it in portuguese only at:

http://www.pythonbrasil.com.br/moin....tesGen%E9ricos

The code is as follows -- still with debug code included, and lots of comments:

"""
metatemplate.py

Template class that can be used to write complex data structures using
nested classes. Template classes can store other template classes (nested)
or user-defined attributes (typed or untyped). The original definition
order information is preserved, allowing for true templating use for
applications such as html templates, data entry forms, and configuration
files.

(c) 2004 Carlos Ribeiro
ca********@gmail.com
http:///pythonnotes.blogspot.com

"""

import sys
from inspect import isclass, isdatadescriptor
from types import StringType, IntType, FloatType, ListType
import itertools

#----------------------------------------------------------------------
# Debug constants. I don't intend to remove them, even from production
# code, but I intend to use the logging module to print the messages

debug_generic_attribute = 0
debug_typed_attribute = 0
debug_auto_instantiation = 0

#----------------------------------------------------------------------
# AbstractAttribute is the ancestor of all classes that can be used
# in the metacontainer framework.

class AbstractAttribute(object):
pass

#----------------------------------------------------------------------
# GenericAttribute is the ancestor of all simple elements that are
# used as attributes of user defined Container subclasses
#
# GenericAttributes are simpler than full containers. They're both
# derived from the same AbstractAttribute class, but GenericAttributes
# have only a single value associated with them.
#
# When referred from a instance, the __get__ method returns the value
# associated with the attribute. If called from the class, the __get__
# method returns the property itself.

class GenericAttribute(AbstractAttribute):
""" Generic attributes for generic containers """
def __init__(self, default = None):
self._seqno = next_attribute_id()
self.value = default
def __repr__(self):
return "<Attr '%s'>" % (self.__class__.__name__)
def __get__(self, instance, owner):
if debug_generic_attribute:
print "GET self:[%s], instance:[%s], owner:[%s]" % \
(self, instance, owner)
if instance:
attrdict = instance.__dict__.setdefault('__attr__', {})
return attrdict.get(self.name, self.value)
else:
return owner
def __set__(self, instance, value):
if debug_generic_attribute:
print "SET self:[%s], instance:[%s], value:[%s]" % \
(self, instance, value)
attrdict = instance.__dict__.setdefault('__attr__', {})
attrdict[self.name] = value

class TypedAttribute(GenericAttribute):
""" Typed attributes for generic containers """
def __init__(self, default = None, mytype = None):
self._seqno = next_attribute_id()
self.value = default
if mytype:
if isclass(mytype):
self.mytype = mytype
else:
raise TypeError("Argument <mytype> expects None "
"or a valid type/class")
else:
self.mytype = type(default)
def __repr__(self):
return "<TypedAttr '%s':%s>" % \
(self.__class__.__name__, self.mytype.__name__)
def __get__(self, instance, owner):
if debug_typed_attribute:
print "GET self:[%s], instance:[%s], owner:[%s]" % \
(self, instance, owner)
if instance:
attrdict = instance.__dict__.setdefault('__attr__', {})
return attrdict.get(self.name, self.value)
else:
return self.value
def __set__(self, instance, value):
if debug_typed_attribute:
print "SET self:[%s], instance:[%s], value:[%s]" % \
(self, instance, value)
if not isinstance(value, self.mytype):
# if it's a string, tries to convert to the correct
# target type (this is needed because most things read
# from files will be strings anyway)
if isinstance(value, StringType):
value = self.mytype(value)
else:
raise TypeError, "Expected %s attribute" % \
self.mytype.__name__
attrdict = instance.__dict__.setdefault('__attr__', {})
attrdict[self.name] = value

#----------------------------------------------------------------------
# auxiliary functions

next_attribute_id = itertools.count().next

def getfields(dct):
"""
takes a dictionary of class attributes and returns a decorated list
containing all valid field instances and their relative position.

"""
for fname, fobj in dct.items():
if isinstance(fobj,GenericAttribute):
yield (fobj._seqno, (fname, fobj))
elif isclass(fobj) and issubclass(fobj,AbstractAttribute):
yield (fobj._seqno, (fname, fobj))
elif (fname[0] != '_'):
# conventional attributes from basic types are just stored
# as GenericAttributes, and put at the end of the list,
# in alphabetical order
if (isinstance(fobj,StringType) or
isinstance(fobj,IntType) or
isinstance(fobj,FloatType) or
isinstance(fobj,ListType)):
yield (sys.maxint, (fname, GenericAttribute(fobj)))
else:
yield (0, (fname, fobj))
else:
yield (0, (fname, fobj))

def makefieldsdict(dct, bases):
# build the field list and sort it
fields = list(getfields(dct))
fields.sort()
# undecorate the list and build a dict that will be returned later
sorted_field_list = [field[1] for field in fields]
field_dict = dict(sorted_field_list)
# finds all attributes and nested classes that are containers
attribute_list = [field for field in sorted_field_list
if (isinstance(field[1],AbstractAttribute) or
(isclass(field[1]) and
issubclass(field[1],AbstractAttribute)
))]
# check baseclasses for attributes inherited but not overriden
# !!WARNING: this code does not checks correctly for multiple
# base classes if there are name clashes between overriden
# members. This is not recommended anyway.
inherited = []
for baseclass in bases:
base_field_list = getattr(baseclass, '_fields', None)
# looks for a valid _fields attribute in an ancestor
if isinstance(base_field_list, ListType):
fnames = [f[0] for f in attribute_list]
for fname, fobj in base_field_list:
# checks for overriden attributes
if (fname in fnames):
# overriden - inherited list contains the new value
newobj = field_dict[fname]
inherited.append((fname, newobj))
# remove attribute and quick check field names list
attribute_list.remove((fname, field_dict[fname]))
fnames.remove(fname)
else:
# copy the original entry into the inherited list
inherited.append((fname, fobj))
field_dict['_fields'] = inherited + attribute_list
return field_dict

#----------------------------------------------------------------------
# MetaTemplate metaclass
#
# Most of the hard work is done outside the class by the auxiliary
# functions makefieldsdict() and getfields()

class MetaTemplate(type):
def __new__(cls, name, bases, dct):
# creates the class using only the processed field list
newdct = makefieldsdict(dct, bases)
newclass = type.__new__(cls, name, bases, newdct)
newclass._seqno = next_attribute_id()
newclass.name = name
return newclass

#----------------------------------------------------------------------
# GenericTemplate superclass

class GenericTemplate(AbstractAttribute):
__metaclass__ = MetaTemplate

def __init__(self):
""" instantiates all nested classes upon creation """

# builds a copy of the field list. this is needed to allow
# customizations of the instance not to be reflected in the
# original class field list.
self._fields = list(self.__class__._fields)

# auto instantiates nested classes and attributes
if debug_auto_instantiation:
print "AutoInstantiation <%s>: fieldlist = %s" % \
(self.name, self._fields)
for fname, fobj in self._fields:
if isclass(fobj) and issubclass(fobj,Container):
# found a nested class
if debug_auto_instantiation:
print "AutoInstantiation <%s>: field[%s] is a "
"Container Subclass" % (self.name, fname)
fobj = fobj()
setattr(self, fname, fobj)
elif isinstance(fobj, AbstractAttribute):
# found an attribute instance
if debug_auto_instantiation:
print "AutoInstantiation <%s>: field[%s] is an "
"Attribute Instance" % (self.name, fname)
# removed: parent links are still being thought out,
# and I'm not even sure if they're a good idea
# setattr(fobj, 'parent', self)
setattr(fobj, 'name', fname)
else:
if debug_auto_instantiation:
print "AutoInstantiation <%s>: field[%s] is "
"unknown" % (self.name, fname)

def iterfields(self):
for fname, fobj in self._fields:
yield getattr(self, fname)

def __repr__(self):
return "<%s '%s'>" % (self.__class__.__name__, self.name,)

--
Carlos Ribeiro
Consultoria em Projetos
blog: http://rascunhosrotos.blogspot.com
blog: http://pythonnotes.blogspot.com
mail: ca********@gmail.com
mail: ca********@yahoo.com

[Carlos Ribeiro]

On Wed, 13 Oct 2004 13:50:00 +0200, Peter Otten <__*******@web.de> wrote:

Bengt Richter wrote:

Or, an ugly hack that might work for a while, depending on how
co_names is really generated. It seems in order of occurrence
(including right hand sides of assignment, but still top down) ...

>> import sys
>> def getnames(): return sys._getframe(1).f_code.co_names

...

>> def MC(cname, cbases, cdict):

... names = cdict.get('ordic',[])
... names = [name for name in names if name in cdict]
... cdict['ordic'] = dict([(name,i) for i,name in enumerate(names)])
... return type(cname, cbases, cdict)
...

>> class C(object):

... __metaclass__ = MC # really a function shortcut
... x = 123
... y = sys
... z = 0
... ordic = getnames()
...

>> C.ordic

{'ordic': 5, '__module__': 0, '__metaclass__': 1, 'y': 3, 'x': 2, 'z': 4}

A metaclass /function/ and sys._getframe() exercised on a class definition
- I think you have raised the bar for what qualifies as a hack :-)

Hey, I came first with that a couple of weeks ago :-) but in truth, it
was Alex Martelli that pointed to me that a metaclass function would
work... but not only is it not recommended, it's also said to make
Guido shudder ;-) Seriously, although it works, it's not recommended
practice. Metaclasses are supposed to be classes, not functions.

As for the getframe, I have played with it a little bit also. But in
the end I have chosen to use a simple counter, using
itertools.count(). More pythonic IMHO. And no need for clever hacks,
when all that is needed is to order the elements in the order they are
executed (which count() can guarantee). There are two situations where
the simpler counter works, but the getframe hack doesn't:

-- if I have a loop inside my class that is used to declare bunch of
attributes, all of them will have the same line number... but a
different sequence number if the simpler method is chosen.

-- if a function is called that returns a bunch of attributes (not
common practice, but still possible). All attributes are at the same
line in this case. Example:

class Foo:
a,b,c = myfunc(...)

Of course, we are now getting into corner cases that show how much are
we pushing class statements in Python. The _sane_ way to make it all
work would be to have a hook to provide a user-defined dict to the
class locals() dir; aternatively, the natice dict() could provide a
ordered interface (but then it wouldn't be a simple hash mapping, a
more complex structure such as a tree would be needed). Both are far
from happening in Python 2.x, IMHO... and I really doubt if dicts will
ever be changed to accomodate ordering, even in Python 3.0. Too much
hassle for too little gain. A hook function seems to be more sensible.

--
Carlos Ribeiro
Consultoria em Projetos
blog: http://rascunhosrotos.blogspot.com
blog: http://pythonnotes.blogspot.com
mail: ca********@gmail.com
mail: ca********@yahoo.com

[Nicolas Fleury]

Peter Otten wrote:

If you want to record the order of these definitions, you need to pass a
custom dictionary that keeps track of assignments in the class generation
code (basically a normal python function comprising the class suite).
Unfortunately that dictionary - which you see later as the classdict
parameter of the metaclass __new__() method - is always a dict created in C,
as was recently discussed on c.l.py (sorry, but all keywords I remember are
'metaclass' and 'martelli' - not very selective :-). Below is my (clumsy)
attempt for a workaround:

import itertools

class OrderedValue(object):
newIndex = itertools.count(1).next
def __init__(self, value):
self.value = value
self.index = self.newIndex()

That's the solution I finally did after my post (I think I found the
thread you were referring to). In my case, having to create instances
has not been so bad, as I can use them to put other information:

class SomeClass(BinType):
x = Member(Int32)
y = Member(Int16)

class SomeOtherClass(BinType):
a = ArrayMember(Int16, size=256)
b = Member(SomeClass)
c = Member(Int32)

This way, I can generate abstract documentation of my binary formats
without instantiating my types and can instantiate them to dump binary data.

Thx and Regards,
Nicolas

[Nicolas Fleury]

Carlos Ribeiro wrote:

Of course, we are now getting into corner cases that show how much are
we pushing class statements in Python. The _sane_ way to make it all
work would be to have a hook to provide a user-defined dict to the
class locals() dir; aternatively, the natice dict() could provide a
ordered interface (but then it wouldn't be a simple hash mapping, a
more complex structure such as a tree would be needed). Both are far
from happening in Python 2.x, IMHO... and I really doubt if dicts will
ever be changed to accomodate ordering, even in Python 3.0. Too much
hassle for too little gain. A hook function seems to be more sensible.

I don't know. I was thinking that maybe an ordereddict class would be
useful, something like:

myDict = {'a': 4, 'b': 5 } # normal dict
myDict = o{'a': 4, 'b': 5 } # ordered dict, keeping order ['a', 'b']

The dictionary could only be use in specific cases, like configuration
stuff or temporarily when instantiating a metaclass... I don't know,
maybe I'm just crazy...

Regards,
Nicolas

[Carlos Ribeiro]

On Wed, 13 Oct 2004 10:42:28 -0400, Nicolas Fleury
<ni******@yahoo.com_remove_the_> wrote:

Carlos Ribeiro wrote:

Of course, we are now getting into corner cases that show how much are
we pushing class statements in Python. The _sane_ way to make it all
work would be to have a hook to provide a user-defined dict to the
class locals() dir; aternatively, the natice dict() could provide a
ordered interface (but then it wouldn't be a simple hash mapping, a
more complex structure such as a tree would be needed). Both are far
from happening in Python 2.x, IMHO... and I really doubt if dicts will
ever be changed to accomodate ordering, even in Python 3.0. Too much
hassle for too little gain. A hook function seems to be more sensible.

I don't know. I was thinking that maybe an ordereddict class would be
useful, something like:

myDict = {'a': 4, 'b': 5 } # normal dict
myDict = o{'a': 4, 'b': 5 } # ordered dict, keeping order ['a', 'b']

The dictionary could only be use in specific cases, like configuration
stuff or temporarily when instantiating a metaclass... I don't know,
maybe I'm just crazy...

I don't think you're crazy. OTOH, defining 'ordering' is difficult. Is
it the alphabethical ordering, or is it the ordering of insertion?
Either way is arbitrary and it not going to satisfy everyone.

I am beginning to get a glimpse of the theorethical implications of
Python's metaclass design and implementation. It's an amazingly clever
solution (the __metaclass__ magic name, I mean); but I suspect that it
could be more powerful if there were a way to detect the metaclass
*before* the class statement is first processed; in this way, the
metaclass could provide a custom locals() dict, or implement other
types of hook for the class being created. But this is really hard
stuff, with implications beyond my current understanding, and I don't
feel prepared to argue at python-dev about it, really.
--
Carlos Ribeiro
Consultoria em Projetos
blog: http://rascunhosrotos.blogspot.com
blog: http://pythonnotes.blogspot.com
mail: ca********@gmail.com
mail: ca********@yahoo.com

[Peter Otten]

Carlos Ribeiro wrote:

executed (which count() can guarantee). There are two situations where
the simpler counter works, but the getframe hack doesn't:

-- if I have a loop inside my class that is used to declare bunch of
attributes, all of them will have the same line number... but a
different sequence number if the simpler method is chosen.

-- if a function is called that returns a bunch of attributes (not
common practice, but still possible). All attributes are at the same
line in this case. Example:

class Foo:
a,b,cÂ*=Â*myfunc(...)

Do you have an example handy? I had so far no success breaking Bengt's code.

Peter

[Nicolas Fleury]

Carlos Ribeiro wrote:

I don't think you're crazy. OTOH, defining 'ordering' is difficult. Is
it the alphabethical ordering, or is it the ordering of insertion?
Insertion. The point is that *there is* an order in the code.
Either way is arbitrary and it not going to satisfy everyone.
The ordereddict object could provide a method to change the order of
keys. Right now, in configuration dictionaries I must do (simplified):

SomeConfig(['y', 'x'], {'y': 1, 'x': 2})
or
SomeConfig(MyOrderedDict([('y', 1), ('x', 2)]))

what I would like is to do:
SomeConfig(o{'y': 1, 'x': 2})
I am beginning to get a glimpse of the theorethical implications of
Python's metaclass design and implementation. It's an amazingly clever
solution (the __metaclass__ magic name, I mean); but I suspect that it
could be more powerful if there were a way to detect the metaclass
*before* the class statement is first processed; in this way, the
metaclass could provide a custom locals() dict, or implement other
types of hook for the class being created. But this is really hard
stuff, with implications beyond my current understanding, and I don't
feel prepared to argue at python-dev about it, really.

I feel the same way. I was thinking about writing a PEP for an
ordereddict, suggesting in it that one of the applications could be the
metaclasses. An ordereddict could also contain a dict, and could be
used only temporarily only during metaclass creation and after just keep
the normal dictionary in it.

Regards,
Nicolas

[Carlos Ribeiro]

On Wed, 13 Oct 2004 12:41:03 -0400, Nicolas Fleury
<ni******@yahoo.com_remove_the_> wrote:

Carlos Ribeiro wrote:

I don't think you're crazy. OTOH, defining 'ordering' is difficult. Is
it the alphabethical ordering, or is it the ordering of insertion?

Insertion. The point is that *there is* an order in the code.

In this case, yes. But I meant to say that a ordereddict would need to
make a choice here. Either way, our particular need doesn't seem to be
common enough to warrant such a patch.

BTW, I'm curious about why are doing it yourself. In my particular
case, I'm interested in declarative idioms for templating code: data
entry forms, configuration files, and similar stuff. I also think
that, whatever you're looking after, we could share some code and
experiences, and perhaps bundle it all into a single library. What do
you think of it?

--
Carlos Ribeiro
Consultoria em Projetos
blog: http://rascunhosrotos.blogspot.com
blog: http://pythonnotes.blogspot.com
mail: ca********@gmail.com
mail: ca********@yahoo.com

[Carlos Ribeiro]

On Wed, 13 Oct 2004 18:18:02 +0200, Peter Otten <__*******@web.de> wrote:

Carlos Ribeiro wrote:

executed (which count() can guarantee). There are two situations where
the simpler counter works, but the getframe hack doesn't:

-- if I have a loop inside my class that is used to declare bunch of
attributes, all of them will have the same line number... but a
different sequence number if the simpler method is chosen.

-- if a function is called that returns a bunch of attributes (not
common practice, but still possible). All attributes are at the same
line in this case. Example:

class Foo:
a,b,c = myfunc(...)

Do you have an example handy? I had so far no success breaking Bengt's code.

Ok. Just as an exercise -- at this point we're far from safe Python
land anyway, and it's not recommended to push it beyond this limit...
;-) Pseudo code only:

class Foo:
# loop that creates attributes out of a list
for name, val in list:
locals()[name] = val

or (actual code tested):

class Bar: .... a,b,c,d = (1,2,3,4)
.... vars(Bar)

{'a': 1, '__module__': '__main__', 'b': 2, 'd': 4, 'c': 3, '__doc__': None}

The ordering became (a,b,d,c); and the code would have no way to tell
that 'c' was supposed to be classified before 'd', because the
getframe trick would return the same line.

--
Carlos Ribeiro
Consultoria em Projetos
blog: http://rascunhosrotos.blogspot.com
blog: http://pythonnotes.blogspot.com
mail: ca********@gmail.com
mail: ca********@yahoo.com

[Nicolas Fleury]

Carlos Ribeiro wrote:

In this case, yes. But I meant to say that a ordereddict would need to
make a choice here. Either way, our particular need doesn't seem to be
common enough to warrant such a patch.
I don't know. Store the order only adds a list that would not affect
lookup performance in the dictionary. Right now, the order is simply
lost; that's not nice. Alphabetical order can be get anytime, so I
really see that as different, since there are a lot of solutions for
these needs, while there's only hacks for our need.
BTW, I'm curious about why are doing it yourself. In my particular
case, I'm interested in declarative idioms for templating code: data
entry forms, configuration files, and similar stuff. I also think
that, whatever you're looking after, we could share some code and
experiences, and perhaps bundle it all into a single library. What do
you think of it?

My solution for now is almost exactly what Peter Otten posted,
consisting only of a metaclass and a base class for attributes. I do
some specific stuff, like instanciate some type stored in the attributes
in each instance. I could inherit from your classes instead, but I
don't have a lot to share for now. But yes, I would like very much to
share what I have once it is completed. What I'm doing looks a lot like
a metastruct module (see example in previous post).

Regards,
Nicolas

[Thomas Heller]

Nicolas Fleury <ni******@yahoo.com_remove_the_> writes:

like very much to share what I have once it is completed. What I'm
doing looks a lot like a metastruct module (see example in previous
post).

In this case, you could also look at the ctypes module, it implements C
compatible structures and unions.

And for your original question: why do you insist on your new syntax,
why don't you simply (as ctypes also does it) define the attributes in a
list, when they have to have a certain order?

Thomas

[Peter Otten]

Carlos Ribeiro wrote:

Ok. Just as an exercise -- at this point we're far from safe Python
land anyway, and it's not recommended to push it beyond this limit...
;-) Pseudo code only:

class Foo:
#Â*loopÂ*thatÂ*createsÂ*attributesÂ*outÂ*ofÂ*aÂ*li st
forÂ*name,Â*valÂ*inÂ*list:
locals()[name]Â*=Â*val
Agreed.
or (actual code tested):

class Bar: ... Â*Â*Â*Â*Â*Â*Â*Â*a,b,c,d = (1,2,3,4)
... vars(Bar) {'a': 1, '__module__': '__main__', 'b': 2, 'd': 4, 'c': 3, '__doc__':
{None}
You cannot rely on a vars() because it produces a dict - and dictionaries
don't preserve the insertion order by design.
The ordering became (a,b,d,c); and the code would have no way to tell
that 'c' was supposed to be classified before 'd', because the
getframe trick would return the same line.

No, it doesn't:

class Bar: .... a, b, c, d = "xyzt"
.... print sys._getframe().f_code.co_names
....
('__name__', '__module__', 'a', 'b', 'c', 'd', 'sys', '_getframe', 'f_code',
'co_names')

whereas
vars(Bar).keys() ['a', '__module__', 'b', 'd', 'c', '__doc__']

Of course there are other ways to garble the co_names order:
class Any: .... def __getattr__(self, name): return name
.... any = Any()
class Bar:

.... x = any.d
.... a, b, c, d = "xyzt"
.... print sys._getframe().f_code.co_names
....
('__name__', '__module__', 'any', 'd', 'x', 'a', 'b', 'c', 'sys',
'_getframe', 'f_code', 'co_names')

All very useless/interesting stuff :-)

Peter

[Nicolas Fleury]

Thomas Heller wrote:

In this case, you could also look at the ctypes module, it implements C
compatible structures and unions.

And for your original question: why do you insist on your new syntax,
why don't you simply (as ctypes also does it) define the attributes in a
list, when they have to have a certain order?

I just looked at ctypes, and yes, that's exactly what I'm doing. I want
the structures definitions to be modified by C++ programmers and I find
the metaclass mechanism much more readable and easy to modify (and it's
not a lot of code to make everything work). I agree everything can be
done using normal Python code, but my goal here is really to make a
pseudo-language to define our formats, and the pseudo-language is just a
simpler solution than XML in my specific case. Probably all cases of
such using of metaclasses are like that...

Regards,
Nicolas

[Thomas Heller]

Nicolas Fleury <ni******@yahoo.com_remove_the_> writes:

Thomas Heller wrote:

In this case, you could also look at the ctypes module, it implements C
compatible structures and unions.
And for your original question: why do you insist on your new syntax,
why don't you simply (as ctypes also does it) define the attributes in a
list, when they have to have a certain order?

I just looked at ctypes, and yes, that's exactly what I'm doing. I
want the structures definitions to be modified by C++ programmers and
I find the metaclass mechanism much more readable and easy to modify
(and it's not a lot of code to make everything work). I agree
everything can be done using normal Python code, but my goal here is
really to make a pseudo-language to define our formats, and the
pseudo-language is just a simpler solution than XML in my specific
case. Probably all cases of such using of metaclasses are like
that...

FWIW, I have a parser(code generator (using gccxml as frontend). This
toolchain creates ctypes' compatible structure definitions from C header
files.

Thomas

[Carlos Ribeiro]

On Wed, 13 Oct 2004 21:32:20 +0200, Thomas Heller <th*****@python.net> wrote:

And for your original question: why do you insist on your new syntax,
why don't you simply (as ctypes also does it) define the attributes in a
list, when they have to have a certain order?

I can't speak for Nicolas, but I can for myself. There are three
reasons, in my particular case:

-- The order is already there, explicit in the way the code is
written. I see no reason to be forced to state it _again_, specially
considering the fact that this is prone to errors; for example,
missing elements, duplicated elements, or out-of-order elements (in
ordered differently in the code and in the list).

-- I make extensive use of nested structs. In this case it's _much_
easier to make mistakes as the ones mentioned above.

-- Using classes and inheritance, it's easy to add new members or
redefine existing ones. But I would need to re-state the full list for
any descendant. To make matters worse, in this case, whenever I make a
change to the base class, I would have to change _all_ descendant
classes. Not exactly good OO design. Of course, clever hacks could
possibly be used, but then, it wouldn't look any better than Nicolas
(or my own) ordered-attributes hack.
It's specially important to point out that I consider the source code
ordering to be _explicit_, and not implicit, as some people may
possibly say. In fact, is there anything more explicit than that? Only
people used to Python dicts would say otherwise.

--
Carlos Ribeiro
Consultoria em Projetos
blog: http://rascunhosrotos.blogspot.com
blog: http://pythonnotes.blogspot.com
mail: ca********@gmail.com
mail: ca********@yahoo.com

[Carlos Ribeiro]

On Wed, 13 Oct 2004 22:03:47 +0200, Peter Otten <__*******@web.de> wrote:

The ordering became (a,b,d,c); and the code would have no way to tell
that 'c' was supposed to be classified before 'd', because the
getframe trick would return the same line.

No, it doesn't:

:-P Oh well, I just missed that Nicolas was using co_names, instead
of line numbering magic as I did in my first tests. :-) Seems he was a
little bit more clever than I was :-)

class Bar: ... a, b, c, d = "xyzt"
... print sys._getframe().f_code.co_names
...
('__name__', '__module__', 'a', 'b', 'c', 'd', 'sys', '_getframe', 'f_code',
'co_names')

whereas
vars(Bar).keys() ['a', '__module__', 'b', 'd', 'c', '__doc__']

Of course there are other ways to garble the co_names order:
class Any: ... def __getattr__(self, name): return name
... any = Any()
class Bar:

... x = any.d
... a, b, c, d = "xyzt"
... print sys._getframe().f_code.co_names
...
('__name__', '__module__', 'any', 'd', 'x', 'a', 'b', 'c', 'sys',
'_getframe', 'f_code', 'co_names')

All very useless/interesting stuff :-)

Useless, in a sense, yes. But we learned a lot (or at least, I did).
So it was not totally useless...

--
Carlos Ribeiro
Consultoria em Projetos
blog: http://rascunhosrotos.blogspot.com
blog: http://pythonnotes.blogspot.com
mail: ca********@gmail.com
mail: ca********@yahoo.com

[Bengt Richter]

On Wed, 13 Oct 2004 09:28:29 -0300, Carlos Ribeiro <ca********@gmail.com> wrote:

On Wed, 13 Oct 2004 13:50:00 +0200, Peter Otten <__*******@web.de> wrote:

Bengt Richter wrote:

> Or, an ugly hack that might work for a while, depending on how
> co_names is really generated. It seems in order of occurrence
> (including right hand sides of assignment, but still top down) ...
>
> >>> import sys
> >>> def getnames(): return sys._getframe(1).f_code.co_names
> ...
> >>> def MC(cname, cbases, cdict):
> ... names = cdict.get('ordic',[])
> ... names = [name for name in names if name in cdict]
> ... cdict['ordic'] = dict([(name,i) for i,name in enumerate(names)])
> ... return type(cname, cbases, cdict)
> ...
> >>> class C(object):
> ... __metaclass__ = MC # really a function shortcut
> ... x = 123
> ... y = sys
> ... z = 0
> ... ordic = getnames()
> ...
> >>> C.ordic
> {'ordic': 5, '__module__': 0, '__metaclass__': 1, 'y': 3, 'x': 2, 'z': 4}
A metaclass /function/ and sys._getframe() exercised on a class definition
- I think you have raised the bar for what qualifies as a hack :-)

Hey, I came first with that a couple of weeks ago :-) but in truth, it
was Alex Martelli that pointed to me that a metaclass function would
work... but not only is it not recommended, it's also said to make
Guido shudder ;-) Seriously, although it works, it's not recommended
practice. Metaclasses are supposed to be classes, not functions.

IIRC it was Guido himself who let the cat out of the bag about a function, in
some early metaclass notes. But maybe I don't RC ;-)
As for the getframe, I have played with it a little bit also. But in
the end I have chosen to use a simple counter, using
itertools.count(). More pythonic IMHO. And no need for clever hacks,
when all that is needed is to order the elements in the order they are
executed (which count() can guarantee). There are two situations where
the simpler counter works, but the getframe hack doesn't:

-- if I have a loop inside my class that is used to declare bunch of
attributes, all of them will have the same line number... but a
different sequence number if the simpler method is chosen. You mean like
for name in 'a b c'.split(): locals()[name] = ord(name[0])
? Yes, I think the names have to be visible to the compiler to show up in co_names.
-- if a function is called that returns a bunch of attributes (not
common practice, but still possible). All attributes are at the same
line in this case. Example:

class Foo:
a,b,c = myfunc(...)
That one _will_ show a,b,c in order. They're all visible to the compiler as names.
Of course, we are now getting into corner cases that show how much are
we pushing class statements in Python. The _sane_ way to make it all
work would be to have a hook to provide a user-defined dict to the
class locals() dir; aternatively, the natice dict() could provide a interestingly, you can pass a dict subtype instance as the last arg to type, but
it seems just to grab the instance's base dict. I.e., you apparently can't
get the subtype instance back as .__dict__. Oh well ;-)
ordered interface (but then it wouldn't be a simple hash mapping, a
more complex structure such as a tree would be needed). Both are far
from happening in Python 2.x, IMHO... and I really doubt if dicts will
ever be changed to accomodate ordering, even in Python 3.0. Too much
hassle for too little gain. A hook function seems to be more sensible.

class C(object, cdict=myDict): pass ??

Regards,
Bengt Richter

[Carlos Ribeiro]

On Wed, 13 Oct 2004 21:32:20 +0200, Thomas Heller <th*****@python.net> wrote:

And for your original question: why do you insist on your new syntax,
why don't you simply (as ctypes also does it) define the attributes in a
list, when they have to have a certain order?

It's funny, but I was working with a good case for native (as in
source-code-defined) ordering right now and had missed it. I'm
learning to write test cases using unittest.py, and it just stroke me
that it's better to have tests to run in the order of complexity, in a
predictable way; it makes reading the list and analyzing the results
easier. My tests are beginning to get complex, and I just thought that
this would be a good use of our techniques (specially because defs
already have ordering information stored as function code attributes).

(in fact, I just checked unittest.py; it seems to be relatively easy
to change, because it has a hook for a cmp function that can use this
extra information)

--
Carlos Ribeiro
Consultoria em Projetos
blog: http://rascunhosrotos.blogspot.com
blog: http://pythonnotes.blogspot.com
mail: ca********@gmail.com
mail: ca********@yahoo.com

[Bengt Richter]

On Wed, 13 Oct 2004 15:07:32 -0300, Carlos Ribeiro <ca********@gmail.com> wrote:

On Wed, 13 Oct 2004 18:18:02 +0200, Peter Otten <__*******@web.de> wrote:

Carlos Ribeiro wrote:

> executed (which count() can guarantee). There are two situations where
> the simpler counter works, but the getframe hack doesn't:
>
> -- if I have a loop inside my class that is used to declare bunch of
> attributes, all of them will have the same line number... but a
> different sequence number if the simpler method is chosen.
>
> -- if a function is called that returns a bunch of attributes (not
> common practice, but still possible). All attributes are at the same
> line in this case. Example:
>
> class Foo:
> a,b,c = myfunc(...)
Do you have an example handy? I had so far no success breaking Bengt's code.

Ok. Just as an exercise -- at this point we're far from safe Python
land anyway, and it's not recommended to push it beyond this limit...
;-) Pseudo code only:

class Foo:
# loop that creates attributes out of a list
for name, val in list:
locals()[name] = val

Sure, the compiler has to see the names in order for them to get into co_names in the frame.
or (actual code tested):

class Bar:... a,b,c,d = (1,2,3,4)
... vars(Bar){'a': 1, '__module__': '__main__', 'b': 2, 'd': 4, 'c': 3, '__doc__': None}

The ordering became (a,b,d,c); and the code would have no way to tell
that 'c' was supposed to be classified before 'd', because the
getframe trick would return the same line.

The "getframe trick" (if you mean my hack ;-) is not using line numbers,
so I don't know what you mean:

class C(object): ... __metaclass__ = MC # really a function shortcut
... a,b,c,d = (1,2,3,4)
... ordic = getnames()
... C.ordic

{'a': 2, 'ordic': 6, '__module__': 0, 'b': 3, '__metaclass__': 1, 'd': 5, 'c': 4}

Regards,
Bengt Richter

[Carlos Ribeiro]

On Wed, 13 Oct 2004 21:35:47 GMT, Bengt Richter <bo**@oz.net> wrote:

Hey, I came first with that a couple of weeks ago :-) but in truth, it
was Alex Martelli that pointed to me that a metaclass function would
work... but not only is it not recommended, it's also said to make
Guido shudder ;-) Seriously, although it works, it's not recommended
practice. Metaclasses are supposed to be classes, not functions.

IIRC it was Guido himself who let the cat out of the bag about a function, in
some early metaclass notes. But maybe I don't RC ;-)

Alex said that he was the one that shown it to Guido, but even he
couldn't recollect it very well. But that's not the point anyway.

class Foo:
a,b,c = myfunc(...)

That one _will_ show a,b,c in order. They're all visible to the compiler as names.

Sure. I misunderstand your trick, and assumed you did similar to what
I did. Your trick is _much_ better than my original one. But still, it
can be fooled, as Peter has shown in another message; but OTOH, it
doesn't _require_ all names to be subclasses of some OrderedAttribute
class.

I'm now thinking about how to use your trick, maybe combined with
mine, into the same framework. I'll let you know if I managed to make
it.

Of course, we are now getting into corner cases that show how much are
we pushing class statements in Python. The _sane_ way to make it all
work would be to have a hook to provide a user-defined dict to the
class locals() dir; aternatively, the natice dict() could provide a

Interestingly, you can pass a dict subtype instance as the last arg to type, but
it seems just to grab the instance's base dict. I.e., you apparently can't
get the subtype instance back as .__dict__. Oh well ;-)

I don't get it; would you mind to explain it better? Did you test it?

ordered interface (but then it wouldn't be a simple hash mapping, a
more complex structure such as a tree would be needed). Both are far
from happening in Python 2.x, IMHO... and I really doubt if dicts will
ever be changed to accomodate ordering, even in Python 3.0. Too much
hassle for too little gain. A hook function seems to be more sensible.

class C(object, cdict=myDict): pass ??

My best bet is something like metaclasses implemented as class
decorators -- something that is read by the compiler _before_ the
class statement is executed. Being a class (not a function), it could
provide methods to customize some aspects of the class statement
execution; for example, a __getlocals__() method, or something along
these lines. and, of course, it would be able to put the __metaclass__
name right into the locals() dict, which would make it backward
compatible. The problem is, function decorators are only executed
_after_ the function def is executed; and having class decorators work
differently is a big no-no.

--
Carlos Ribeiro
Consultoria em Projetos
blog: http://rascunhosrotos.blogspot.com
blog: http://pythonnotes.blogspot.com
mail: ca********@gmail.com
mail: ca********@yahoo.com

[Bengt Richter]

On Wed, 13 Oct 2004 18:56:32 -0300, Carlos Ribeiro <ca********@gmail.com> wrote:
[...]

I'm now thinking about how to use your trick, maybe combined with
mine, into the same framework. I'll let you know if I managed to make
it. I doubt if it's a good idea to be dependent on such implementation details,
except to make throwaway prototypes of functionality with what's available.

>Of course, we are now getting into corner cases that show how much are
>we pushing class statements in Python. The _sane_ way to make it all
>work would be to have a hook to provide a user-defined dict to the
>class locals() dir; aternatively, the natice dict() could provide a Interestingly, you can pass a dict subtype instance as the last arg to type, but
it seems just to grab the instance's base dict. I.e., you apparently can't
get the subtype instance back as .__dict__. Oh well ;-)

I don't get it; would you mind to explain it better? Did you test it?

I just did an experiment:

class CD(dict): ... def __repr__(self): return '<CD %s>'%dict.__repr__(self)
... cd = CD(some='content')
cd <CD {'some': 'content'}> T = type('T',(),cd)
T.some 'content' t=T()
t.some 'content' T.__dict__ <dictproxy object at 0x00901090>

.... not <CD {'some': 'content', ...}> in any form, AFAICS. I.e., it acts like
I just did T = type('T',(), {'some':'content')}

IOW, I think type must just extract the contents from the supplied dict or dict subtype.
I wonder what method one should override to customize the content extraction, or if that's possible.

So T.__dict__ seems to be a new container with its own dict methods... T.__dict__.items() [('__dict__', <attribute '__dict__' of 'T' objects>), ('__weakref__', <attribute '__weakref__' o
f 'T' objects>), ('__module__', '__main__'), ('some', 'content'), ('__doc__', None)] T.__dict__.__class__

<type 'dictproxy'>

>ordered interface (but then it wouldn't be a simple hash mapping, a
>more complex structure such as a tree would be needed). Both are far
>from happening in Python 2.x, IMHO... and I really doubt if dicts will
>ever be changed to accomodate ordering, even in Python 3.0. Too much
>hassle for too little gain. A hook function seems to be more sensible.

class C(object, cdict=myDict): pass ??

My best bet is something like metaclasses implemented as class
decorators -- something that is read by the compiler _before_ the
class statement is executed. Being a class (not a function), it could
provide methods to customize some aspects of the class statement
execution; for example, a __getlocals__() method, or something along
these lines. and, of course, it would be able to put the __metaclass__
name right into the locals() dict, which would make it backward
compatible. The problem is, function decorators are only executed
_after_ the function def is executed; and having class decorators work
differently is a big no-no.

ISTM a metaclass is a lot like a decorator for classes. I.e., it is called
just after the class definition executes and before the name is bound.

They are both somewhat like atexit hooked functions. Except that the process
is not an OS process, but just a little segment of the current thread having to
do with building a class. If you looked at it in an OO way, you might have
an object that you instantiate and set up before kicking off its run method.
Then you could have properties that acccumulated chains of decorator functions
to do between the default construct and bind operations, but you could also
have other property/hooks to add e.g., pre-processing of the arg list being
passed to the constructor, and properties to hold alternate components such
as dicts etc. IOW, a big factory object with lots of knobs that ultimately
produces some custom thing. There ought to be some unified concept behind
this process IMO, whether the thing produced is a function or class or type.
They are all factories that compile and pull together pieces and glue or bolt
them together. The tough part is to make it fast with pre-glued assemblies
and yet allow custom bolting together of custom components.
Well, metaphors can only be ridden so far ;-)

Regards,
Bengt Richter

[Alex Martelli]

Nicolas Fleury <ni******@yahoo.com_remove_the_> wrote:
...

from happening in Python 2.x, IMHO... and I really doubt if dicts will
ever be changed to accomodate ordering, even in Python 3.0. Too much
hassle for too little gain. A hook function seems to be more sensible.
I don't know. I was thinking that maybe an ordereddict class would be
useful, something like:

Yes, a mapping that keeps order can be useful (in either meaning of the
term: remembering order of item insertion [or last modification], or,
intrinsically ordering keys [directly or by value]). There are a bit
too many variants to warrant making them built-ins, but the collections
module, in Python 2.5, could certainly accomodate several.

myDict = {'a': 4, 'b': 5 } # normal dict
myDict = o{'a': 4, 'b': 5 } # ordered dict, keeping order ['a', 'b']

The dictionary could only be use in specific cases, like configuration
stuff or temporarily when instantiating a metaclass... I don't know,
maybe I'm just crazy...

Given the rarity of the need, I very much doubt it makes sense to come
up with special syntax, or implicitly use some choice out of these
special mappings (no doubt slower than regular dicts) in some cases. In
other words, I agree with Carlos that some way to tell Python what class
to use (in lieu of default dict) for class statements -- presumably a
sys.setsomething call -- is preferable; I do agree with you that users
shouldn't need to come up with their own "ordered mappings", the
standard library should provide them.
Alex

[Alex Martelli]

Carlos Ribeiro <ca********@gmail.com> wrote:
...

The code is as follows -- still with debug code included, and lots of comments:

Nice! I must be missing something (what's the role of the '__attr__'
key you're setdefaulting into instance dicts and never using
otherwise?), and there may be enhanceable spots such as:

if (isinstance(fobj,StringType) or
isinstance(fobj,IntType) or
isinstance(fobj,FloatType) or
isinstance(fobj,ListType)):

isinstance accepts a tuple as its 2nd arg exactly to avoid this need;
set once, somewhere,

elementaryTypes = StringType, IntType, FloatType, ListType

and then you can typecheck here with just

if isinstance(fobj, elementaryTypes):

....but if this, or the recommendation to follow PEP 8 (space after
comma, etc) is the most important suggestion I have to offer about your
code, I guess that's a good sign!-)
Alex

[Peter Otten]

Carlos Ribeiro wrote:

ifÂ*notÂ*isinstance(value,Â*self.mytype):
#Â*ifÂ*it'sÂ*aÂ*string,Â*triesÂ*toÂ*convertÂ*toÂ*t heÂ*correct
#Â*targetÂ*typeÂ*(thisÂ*isÂ*neededÂ*becauseÂ*most *thingsÂ*read
#Â*fromÂ*filesÂ*willÂ*beÂ*stringsÂ*anyway)
ifÂ*isinstance(value,Â*StringType):
valueÂ*=Â*self.mytype(value)

I haven't checked if this is relevant in the context, but implicit
conversion from string is usually a bad thing. E. g.:

bool("False"), bool("0")

(True, True)

Peter

[Thomas Heller]

Carlos Ribeiro <ca********@gmail.com> writes:

On Wed, 13 Oct 2004 21:32:20 +0200, Thomas Heller <th*****@python.net> wrote:

And for your original question: why do you insist on your new syntax,
why don't you simply (as ctypes also does it) define the attributes in a
list, when they have to have a certain order?
I can't speak for Nicolas, but I can for myself. There are three
reasons, in my particular case:

-- The order is already there, explicit in the way the code is
written. I see no reason to be forced to state it _again_, specially
considering the fact that this is prone to errors; for example,
missing elements, duplicated elements, or out-of-order elements (in
ordered differently in the code and in the list).

Yes, but a Python pogrammer wouldn't expect these two samples to give
different results:

class A(...):
x = 1
y = 2

class A(...):
y = 2
x = 1
-- I make extensive use of nested structs. In this case it's _much_
easier to make mistakes as the ones mentioned above.

-- Using classes and inheritance, it's easy to add new members or
redefine existing ones. But I would need to re-state the full list for
any descendant. To make matters worse, in this case, whenever I make a
change to the base class, I would have to change _all_ descendant
classes. Not exactly good OO design. Of course, clever hacks could
possibly be used, but then, it wouldn't look any better than Nicolas
(or my own) ordered-attributes hack.
In ctyoes you write:

class POINT(Structure):
_fields_ = [("x", c_int),
("y", c_int)]

which makes it explicit that _fields_ is a list, which *has* a certain
order. And for inheritance, it's easy and explicit as well:

class COLORED_POINT(POINT):
_fields_ = POINT._fields_ + [("color", RGB)]
It's specially important to point out that I consider the source code
ordering to be _explicit_, and not implicit, as some people may
possibly say. In fact, is there anything more explicit than that? Only
people used to Python dicts would say otherwise.

The point I'm trying to make is that too much magic most of the time
catches you later again. But that's only my own experience.

Thomas

[Carlos Ribeiro]

On Thu, 14 Oct 2004 09:42:24 +0200, Alex Martelli <al*****@yahoo.com> wrote:

Carlos Ribeiro <ca********@gmail.com> wrote:
...

The code is as follows -- still with debug code included, and lots of comments:

Nice! I must be missing something (what's the role of the '__attr__'
key you're setdefaulting into instance dicts and never using
otherwise?), and there may be enhanceable spots such as:

The relevant part of the code is as follows (debug code removed):

def __get__(self, instance, owner):
if instance:
attrdict = instance.__dict__.setdefault('__attr__', {})
return attrdict.get(self.name, self.value)
else:
return self
def __set__(self, instance, value):
attrdict = instance.__dict__.setdefault('__attr__', {})
attrdict[self.name] = value

The __attr__ is used as part of my descriptor implementation, and it's
an attribute of the instance. Its a dictionary whose keys are the
names of the GenericAttributes (or TypedAttributes) stored in the
instance.

The __get__() code checks if its being called from a class; in this
case, it returns the attribute itself (so I can check its name, and
other properties of the attribute, when working with class
references). But if the attribute is being called from an instance,
then it retrieves the current value of the attribute, as stored in the
instance; if the attribute wasn't initialized, it retrieves the
default value (that is an attribute of the descriptor).

An example may help to clarify things a little bit:

class Person(GenericTemplate)
name = TypedAttribute('unknown')

john = Person()
john.name = "John Doe"
peter = Person()

In this case, john.__attr__['name'] stores the current value of the
name attribute at the instance. john.name will retrieve this value.
peter.__attr__ wasn't still initialized; if you try to retrieve the
name, it will automatically initialize the __attr__ dictionary, and
try to retrieve the value of the 'name' key; as it doesnt exist, it
retrieves the default value.

The class code is also useful. If you refer to Person.name, it will
return the attribute itself, which allows one to retrieve its own
name:

obj = Person.name
print obj.name

if (isinstance(fobj,StringType) or
isinstance(fobj,IntType) or
isinstance(fobj,FloatType) or
isinstance(fobj,ListType)):
isinstance accepts a tuple as its 2nd arg exactly to avoid this need;
set once, somewhere,

elementaryTypes = StringType, IntType, FloatType, ListType

This is one of the old artifacts. I had four different tests, and for
each one of them I had a special subclass: ListAttribute,
StrAttribute, and so on. It went this way because at first I only
wrote the StrAttribute, then added another one... And I'm doing it
XP-style, "the simplest code that works". At some point I finally
managed to 'get' descriptors, and implemented the TypedAttribute
class. I simply collapsed the tests into a single one (I didn't even
retype them! just deleted the intermediate lines and "and'ed" them
together; now talk about "the simplest code that works" :-)).

In fact, there are several parts of the code that are artifacts of
some experiments that I did. I was trying to figure out how to
properly use metaclasses, descriptors, properties, a lot of stuff.
Some of the old code is still there. I also kept debug code, and I
intend to... this is something that still bothers me and is a good
topic for another thread... Debug code clutters the main code, but I
don't like to have to add it back if I ever need it. (what about
logging support at the language level, pretty much as a assert, that
can be removed from optimized code?)

I have finished right now a unittest for the metatemplate module,
including inheritance testing. Now I can start to remove the old cruft
step by step without fear that everything will break apart :-)
and then you can typecheck here with just

if isinstance(fobj, elementaryTypes):

...but if this, or the recommendation to follow PEP 8 (space after
comma, etc) is the most important suggestion I have to offer about your
code, I guess that's a good sign!-)

I re-read PEP8 a few days ago. Some things are a matter of personal
style, and a little harder to change; for instance, I prefer to align
sequences of assignments vertically, something that Guido reportedly
loathes. But I'm already adhering to some recommendations, such as
right-margin and docstrings format.

--
Carlos Ribeiro
Consultoria em Projetos
blog: http://rascunhosrotos.blogspot.com
blog: http://pythonnotes.blogspot.com
mail: ca********@gmail.com
mail: ca********@yahoo.com

[Carlos Ribeiro]

On Thu, 14 Oct 2004 10:11:37 +0200, Peter Otten <__*******@web.de> wrote:

Carlos Ribeiro wrote:

if not isinstance(value, self.mytype):
# if it's a string, tries to convert to the correct
# target type (this is needed because most things read
# from files will be strings anyway)
if isinstance(value, StringType):
value = self.mytype(value)

I haven't checked if this is relevant in the context, but implicit
conversion from string is usually a bad thing. E. g.:

bool("False"), bool("0")

(True, True)

I had to do it to be able to read values written in configuration
files. When I write the attributes to the text file, I lose the type
information, and all that I have when I re-read them is the string
representation. In this case, the best that I can do is to try to
force the conversion from the string to the correct type.

--
Carlos Ribeiro
Consultoria em Projetos
blog: http://rascunhosrotos.blogspot.com
blog: http://pythonnotes.blogspot.com
mail: ca********@gmail.com
mail: ca********@yahoo.com

[Carlos Ribeiro]

On Thu, 14 Oct 2004 11:55:54 +0200, Thomas Heller <th*****@python.net> wrote:

Yes, but a Python pogrammer wouldn't expect these two samples to give
different results:

class A(...):
x = 1
y = 2

class A(...):
y = 2
x = 1
I sincerely dont agree with this argument. A *experienced* Python
programmer wouldn't assume that these two samples would give different
results, because he *knows* that ordering doesn't matter. But many
programmers would be surprised if they tried to iterated over the
fields and had them return out of the original definition ordering.
It's not natural; it's something that you have to explain _why_.
which makes it explicit that _fields_ is a list, which *has* a certain
order. And for inheritance, it's easy and explicit as well:

class COLORED_POINT(POINT):
_fields_ = POINT._fields_ + [("color", RGB)]

I agree that it's clear, and this is not a strong argument that I
could use for my own case. My own approach has to handle similar
situations for nested classes, also.

It's specially important to point out that I consider the source code
ordering to be _explicit_, and not implicit, as some people may
possibly say. In fact, is there anything more explicit than that? Only
people used to Python dicts would say otherwise.

The point I'm trying to make is that too much magic most of the time
catches you later again. But that's only my own experience.

Agreed. But the lack of ordering (which is really a side effect of the
use of the dict mapping to store class attributes, and not a clear
design decision in itself) is also something that strikes newcomers as
a 'surprise factor'; in a sense, I think this makes the field even.

--
Carlos Ribeiro
Consultoria em Projetos
blog: http://rascunhosrotos.blogspot.com
blog: http://pythonnotes.blogspot.com
mail: ca********@gmail.com
mail: ca********@yahoo.com

[Michael Hudson]

bo**@oz.net (Bengt Richter) writes:

IIRC it was Guido himself who let the cat out of the bag about a
function, in some early metaclass notes. But maybe I don't RC ;-)

I don't get the fuss about a function vs a class for the value of
__metaclass__. It's a callable. It gets called. No mystery *here*
:)

Cheers,
mwh

--
... when all the programmes on all the channels actually were made
by actors with cleft pallettes speaking lines by dyslexic writers
filmed by blind cameramen instead of merely seeming like that, it
somehow made the whole thing more worthwhile. -- HHGTG, Episode 11

[Carlos Ribeiro]

On Thu, 14 Oct 2004 14:51:52 GMT, Michael Hudson <mw*@python.net> wrote:

bo**@oz.net (Bengt Richter) writes:

IIRC it was Guido himself who let the cat out of the bag about a
function, in some early metaclass notes. But maybe I don't RC ;-)

I don't get the fuss about a function vs a class for the value of
__metaclass__. It's a callable. It gets called. No mystery *here*
:)

I think it has something to do with the intended working of
metaclasses. After all, they're called meta_classes_, not
metafunctions :-) But on a more serious note: in other languages, such
as Smalltalk, metaclasses are a standard part of the language, and are
_classes_. There is a _metaclass hierarchy_, superimposed to the class
hierarchy. Metaclasses are supposed to be classes because they can be
inherited from, for example. And I believe that was only by accident,
and not by a explicit design choice, that Python's implementation
accepts any callable as a metaclass.

--
Carlos Ribeiro
Consultoria em Projetos
blog: http://rascunhosrotos.blogspot.com
blog: http://pythonnotes.blogspot.com
mail: ca********@gmail.com
mail: ca********@yahoo.com

[Michele Simionato]

Carlos Ribeiro <ca********@gmail.com> wrote in message news:<ma**************************************@pyt hon.org>...

On Thu, 14 Oct 2004 14:51:52 GMT, Michael Hudson <mw*@python.net> wrote:

bo**@oz.net (Bengt Richter) writes:

IIRC it was Guido himself who let the cat out of the bag about a
function, in some early metaclass notes. But maybe I don't RC ;-)

I don't get the fuss about a function vs a class for the value of
__metaclass__. It's a callable. It gets called. No mystery *here*
:)

I think it has something to do with the intended working of
metaclasses. After all, they're called meta_classes_, not
metafunctions :-) But on a more serious note: in other languages, such
as Smalltalk, metaclasses are a standard part of the language, and are
_classes_. There is a _metaclass hierarchy_, superimposed to the class
hierarchy. Metaclasses are supposed to be classes because they can be
inherited from, for example. And I believe that was only by accident,
and not by a explicit design choice, that Python's implementation
accepts any callable as a metaclass.

Using a "metafunction" is a kind of abuse, yes. Nevertheless, I used this trick
to resolve the metaclass conflict:

http://aspn.activestate.com/ASPN/Coo...Recipe/204197c

Later on, I also found a way to avoid the usage of the meta-function,
but it was too brain-exploder to publish ;-)

Michele Simionato

[Ville Vainio]

>>>>> "Michele" == Michele Simionato <mi***************@gmail.com> writes:

Michele> Using a "metafunction" is a kind of abuse,
Michele> yes. Nevertheless, I used this trick to resolve the
Michele> metaclass conflict:

Michele> http://aspn.activestate.com/ASPN/Coo...Recipe/204197c

Broken link, I guess you meant:

http://aspn.activestate.com/ASPN/Coo.../Recipe/204197

I don't think the term "metafunction" is applicable, because calling
it doesn't return a function, and everything that returns a callable
object is at some level a metafunction (at least in the sense
metaclass is a metaclass).

--
Ville Vainio http://tinyurl.com/2prnb

[Carlos Ribeiro]

On 15 Oct 2004 15:11:11 +0300, Ville Vainio <vi***@spammers.com> wrote:

>> "Michele" == Michele Simionato <mi***************@gmail.com> writes:

Michele> Using a "metafunction" is a kind of abuse,
Michele> yes. Nevertheless, I used this trick to resolve the
Michele> metaclass conflict:

Michele> http://aspn.activestate.com/ASPN/Coo...Recipe/204197c

Broken link, I guess you meant:

http://aspn.activestate.com/ASPN/Coo.../Recipe/204197

I don't think the term "metafunction" is applicable, because calling
it doesn't return a function, and everything that returns a callable
object is at some level a metafunction (at least in the sense
metaclass is a metaclass).

I used the term "metafunction" originally as a joke, followed by a
smile :-) For the lack of a better name, it was a reasonable play of
words at that particular time... but sure, a better name should exist.
As it is now, it's a degenerate metaclass of sorts -- pretty much like
a errant virotic RNA strand, lacking even the proteine capsule that
proper virus have... It's hard to classify.

--
Carlos Ribeiro
Consultoria em Projetos
blog: http://rascunhosrotos.blogspot.com
blog: http://pythonnotes.blogspot.com
mail: ca********@gmail.com
mail: ca********@yahoo.com