Proposal: reducing self.x=x; self.y=y; self.z=z boilerplate code

[Ralf W. Grosse-Kunstleve]

*************** *************** *************** *************** *************** ***
This posting is also available in HTML format:
http://cci.lbl.gov/~rwgk/python/adop...005_07_02.html
*************** *************** *************** *************** *************** ***

Hi fellow Python coders,

I often find myself writing::

class grouping:

def __init__(self, x, y, z):
self.x = x
self.y = y
self.z = z
# real code, finally

This becomes a serious nuisance in complex applications with long
argument lists, especially if long variable names are essential for
managing the complexity. Therefore I propose that Python includes
built-in support for reducing the ``self.x=x`` clutter. Below are
arguments for the following approach (*please* don't get too agitated
about the syntax right here, it really is a secondary consideration): :

class grouping:

def __init__(self, .x, .y, .z):
# real code right here

Emulation using existing syntax::

def __init__(self, x, y, z):
self.x = x
del x
self.y = y
del y
self.z = z
del z
Is it really that important?
----------------------------

For applications of non-trivial size, yes. Here is a real-world example
(one of many in that source tree):
http://cvs.sourceforge.net/viewcvs.p...py?view=markup

Fragment from this file::

class manager:

def __init__(self,
crystal_symmetr y=None,
model_indices=N one,
conformer_indic es=None,
site_symmetry_t able=None,
bond_params_tab le=None,
shell_sym_table s=None,
nonbonded_param s=None,
nonbonded_types =None,
nonbonded_funct ion=None,
nonbonded_dista nce_cutoff=None ,
nonbonded_buffe r=1,
angle_proxies=N one,
dihedral_proxie s=None,
chirality_proxi es=None,
planarity_proxi es=None,
plain_pairs_rad ius=None):
self.crystal_sy mmetry = crystal_symmetr y
self.model_indi ces = model_indices
self.conformer_ indices = conformer_indic es
self.site_symme try_table = site_symmetry_t able
self.bond_param s_table = bond_params_tab le
self.shell_sym_ tables = shell_sym_table s
self.nonbonded_ params = nonbonded_param s
self.nonbonded_ types = nonbonded_types
self.nonbonded_ function = nonbonded_funct ion
self.nonbonded_ distance_cutoff = nonbonded_dista nce_cutoff
self.nonbonded_ buffer = nonbonded_buffe r
self.angle_prox ies = angle_proxies
self.dihedral_p roxies = dihedral_proxie s
self.chirality_ proxies = chirality_proxi es
self.planarity_ proxies = planarity_proxi es
self.plain_pair s_radius = plain_pairs_rad ius
# real code, finally

Not exactly what you want to see in a high-level language.
Is there a way out with Python as-is?
-------------------------------------

Yes. If you take the time to look at the file in the CVS you'll find
that I was cheating a bit. To reduce the terrible clutter above, I am
actually using a simple trick::

adopt_init_args (self, locals())

For completeness, the implementation of ``adopt_init_ar gs()`` is here:
http://cvs.sourceforge.net/viewcvs.p...py?view=markup

While this obviously goes a long way, it has several disadvantages:

- The solution doesn't come with Python -> everybody has to reinvent.

- People are reluctant to use the trick since scripts become
dependent on a non-standard feature.

- It is difficult to remember which ``import`` to use for
``adopt_init_ar gs`` (since everybody has a local version/variety).

- The ``adopt_init_ar gs(self, locals())`` incantation is hard to
remember and difficult to explain to new-comers.

- Inside the ``__init__()`` method, the same object has two names,
e.g. ``x`` and ``self.x``. This lead to subtle bugs a few times
when I accidentally assigned to ``x`` instead of ``self.x`` or vice
versa in the wrong place (the bugs are typically introduced while
refactoring).

- In some cases the ``adopt_init_ar gs()`` overhead was found to
introduce a significant performance penalty (in particular the
enhanced version discussed below).

- Remember where Python comes from: it goes back to a teaching
language, enabling mere mortals to embrace programming.
``adopt_init_ar gs(self, locals())`` definitely doesn't live up
to this heritage.
Minimal proposal
----------------

My minimal proposal is to add an enhanced version of ``adopt_init_ar gs()``
as a standard Python built-in function (actual name secondary!)::

class grouping:

def __init__(self, x, y, z):
adopt_init_args ()
# real code

Here is a reference implementation:
http://cvs.sourceforge.net/viewcvs.p....2&view=markup

Implementation of this proposal would remove all the disadvantages
listed above. However, there is another problem not mentioned before:
It is cumbersome to disable adoption of selected variables. E.g.::

class grouping:

def __init__(self, keep_this, and_this, but_not_this, but_this_again) :
self.keep_this = keep_this
self.and_this = and_this
self.but_this_a gain = but_this_again
# real code, finally

would translate into::

class grouping:

def __init__(self, keep_this, and_this, but_not_this, but_this_again) :
adopt_init_args (exclusions=["but_not_th is"])
# real code
Enhanced syntax proposal
------------------------

The exclusion problem suggests these alternatives::

class grouping:

def __init__(self, self.keep_this, self.and_this, but_not_this,
self.but_this_a gain):
# real code right here

This is conceptually similar to the existing automatic unpacking of tuples.

A shorter alternative (my personal favorite since minimally redundant)::

class grouping:

def __init__(self, .keep_this, .and_this, but_not_this, .but_this_again ):
# real code right here

I guess both versions could be implemented such that users don't incur
a performance penalty compared to the ``self.x=x`` alternative. At the
danger of being overly optimistic: I can imagine that my favorite
alternative will actually be faster (and the fastest).
Enhanced __slot__ semantics proposal
------------------------------------

When ``__slots__`` are used (cool feature!) the boilerplate problem
becomes even worse::

class grouping:

__slots__ = ["keep_this" , "and_this", "but_this_again "]

def __init__(self, keep_this, and_this, but_not_this, but_this_again) :
self.keep_this = keep_this
self.and_this = and_this
self.but_this_a gain = but_this_again
# real code, finally

Each variable name appears four times! Imagine yourself having to
do this exercise given the real-world example above. Ouch.

Based on the "Enhanced syntax proposal" above I see this potential
improvement::

class grouping:

__slots__ = True

def __init__(self, .keep_this, .and_this, but_not_this, .but_this_again ):
# real code right here

Each variable name appears only once. Phew!

Author: rw**@yahoo.com, July 02, 2005

P.S.: If you reply to this message, please clearly separate
naming/syntax issues from the core issue of providing built-in support
designed to reduce clutter.


_______________ _______________ _______________ _______
Yahoo! Sports
Rekindle the Rivalries. Sign up for Fantasy Football
http://football.fantasysports.yahoo.com

[Roy Smith]

"Ralf W. Grosse-Kunstleve" <rw**@yahoo.com > wrote:

class grouping:

def __init__(self, .keep_this, .and_this, but_not_this, .but_this_again ):
# real code right here

I'm really torn about this. On the one hand, my first thought was "you
shouldn't be writing constructors with arguments lists so long that this is
a problem", but you showed a reasonable counter-example to that argument.
I'm a big fan of DRY (Don't Repeat Yourself), sometimes expressed as "Once
And Only Once", and your proposal lets you do that.

It also has the nice feature that it doesn't break any existing code; the
suggested syntax is not currently legal Python.

What happens if I do:

def __init__ (self, .x, .y, .z):
x = 0

what does the assignment x do? Does it automatically get promoted to an
assignment to self.x? Does it generate an error?

The big question in my mind is not "Is this useful" (since it clearly is),
but "Does the utility justify the cost?". In other words, will it be used
frequently enough to compensate for the added complexity to the language?
I'm not convinced of that.

There have been some proposals floating around to implement a "with"
keyword which would create implicit namespaces. That's sort of what you're
proposing here. I'm not convinced either is a good idea, but if they were
to be adopted, I'd certainly want to see the them done in a uniform,
logically consistent way.

[Andrew Koenig]

"Ralf W. Grosse-Kunstleve" <rw**@yahoo.com > wrote in message
news:ma******** *************** *************** *@python.org...

class grouping:

def __init__(self, .x, .y, .z):
# real code right here Emulation using existing syntax:: def __init__(self, x, y, z):
self.x = x
del x
self.y = y
del y
self.z = z
del z

I think this is a bad idea, for a subtle reason.

In Python, unlike many other languages, the names of formal parameters are
part of a function's interface. For example:

def f(x, y):
return x-y

Now f(3, 4) is -1 and f(y=3,x=4) is 1.

The names of instance variables are generally not part of a class'
interface--they are part of its implementation.

This proposed feature, whenever used, would tie a class' implementation to
the interface of every method that uses the feature. As far as I can see,
it is impossible to use the feature without constraining the implementation
in this way.

For this reason, I would much rather have the mapping between parameter
names and instance variables be explicit.

[Roy Smith]

"Andrew Koenig" <ar*@acm.org> wrote:

In Python, unlike many other languages, the names of formal parameters are
part of a function's interface. For example:

def f(x, y):
return x-y

Now f(3, 4) is -1 and f(y=3,x=4) is 1.

The names of instance variables are generally not part of a class'
interface--they are part of its implementation.

This proposed feature, whenever used, would tie a class' implementation to
the interface of every method that uses the feature. As far as I can see,
it is impossible to use the feature without constraining the implementation
in this way.

While I suppose that's true from a theoretical point of view, as a
practical matter, I don't see it being much of a big deal. I don't think
I've ever written an __init__ method which saved its parameters and used
different names for the parameter and the corresponding instance variable.
Doing so would just be confusing (at least for the kind of code I write).

Also, it doesn't really tie it in any hard and fast way. Right now, I
would write:

def __init__ (self, x, y, z):
self.x = x
self.y = y
self.z = z
blah

under the new proposal, I would write:

def __init__ (self, .x, .y, .z):
blah

If at some time in the future, if I decided I need to change the name of
the instance variable without changing the exposed interface, it would be
easy enough to do:

def __init__ (self, .x, .y, z):
self.zeta = z
blah

I'm still not convinced we need this, but the exposed interface issue
doesn't worry me much.

[Walter Brunswick]

Why not just update the local dictionary?

class Grouping:
def __init__(self,x ,y,z):
self.__dict__.u pdate(locals())

[Ron Adam]

Ralf W. Grosse-Kunstleve wrote:

class grouping:

def __init__(self, .x, .y, .z):
# real code right here
The way this would work seems a bit inconsistent to me. Args normally
create local variables that receive references to the objects passed to
them.

In this case, a function/method is *creating* non local names in a scope
outside it's own name space to receive it's arguments. I don't think
that's a good idea.

A better approach is to have a copy_var_to(des t, *var_list) function
that can do it. You should be able to copy only selected arguments, and
not all of them.

copy_var_to(sel f,x,z)

Not exactly straight forward to do as it runs into the getting an
objects name problem.

Emulation using existing syntax::

def __init__(self, x, y, z):
self.x = x
del x
self.y = y
del y
self.z = z
del z
The 'del's aren't needed as the references will be unbound as soon as
__init__ is finished. That's one of the reasons you need to do self.x=x
, the other is to share the objects with other methods.
Is there a way out with Python as-is?
-------------------------------------

With argument lists that long it might be better to use a single
dictionary to keep them in.

class manager:
def __init__(self, **args):
defaults = {
'crystal_symmet ry':None,
'model_indices' :None,
'conformer_indi ces':None,
'site_symmetry_ table':None,
'bond_params_ta ble':None,
'shell_sym_tabl es':None,
'nonbonded_para ms':None,
'nonbonded_type s':None,
'nonbonded_func tion':None,
'nonbonded_dist ance_cutoff':No ne,
'nonbonded_buff er':1,
'angle_proxies' :None,
'dihedral_proxi es':None,
'chirality_prox ies':None,
'planarity_prox ies':None,
'plain_pairs_ra dius':None }
defaults.update (args)
self.data = defaults

# real code

Regards,
Ron

[Roy Smith]

In article <Tm************ *********@news2 0.bellglobal.co m>,
"Walter Brunswick" <wa************ *@sympatico.ca> wrote:

Why not just update the local dictionary?

class Grouping:
def __init__(self,x ,y,z):
self.__dict__.u pdate(locals())

That's pretty clever. The only minor annoyance is that it creates a
self.self. If that bothers you, you can fix it with:

def __init__ (self, x, y, z):
vars = locals()
del vars["self"]
self.__dict__.u pdate(vars)

or, perhaps:

def __init__ (self, x, y, z):
self.__dict__.u pdate(locals())
del self.self

It doesn't give you all the flexibility of the original proposal (i.e.
name-by-name selectivity of what gets imported into self), but it does
solve the OP's OP (Original Poster's Original Problem).

[jcarlson]

Ralf W. Grosse-Kunstleve wrote:

A shorter alternative (my personal favorite since minimally redundant)::

class grouping:

def __init__(self, .keep_this, .and_this, but_not_this, .but_this_again ):
# real code right here
There is also the variant which I proposed on python-dev:

class grouping:
def __init__(self, _keep_this, _and_this, but_not_this, _but this
again):
InitAttrs(self, locals())
#real code goes here

Essentially you replace the '.' with '_', which doesn't require a
syntax change. Unfortunately, both are almost invisible. It does
offer you what you want right now (without that whole waiting a year+
for Python 2.5, PEP process, etc.).
Enhanced __slot__ semantics proposal
------------------------------------

When ``__slots__`` are used (cool feature!) the boilerplate problem
becomes even worse::

class grouping:

__slots__ = ["keep_this" , "and_this", "but_this_again "]

def __init__(self, keep_this, and_this, but_not_this, but_this_again) :
self.keep_this = keep_this
self.and_this = and_this
self.but_this_a gain = but_this_again
# real code, finally
There is also the AutoSlots metaclass (which I have fixed) that does
this as well.

class grouping(object ):
__metaclass__ = AutoSlots
def __init__(self, _keep_this, _and_this, but_not_this,
_but_this_again ):
InitAttrs(self, locals())
#real code goes here

Both AutoSlots and InitAttrs use leading underscores on the __init__
method to discover which attributes are to be __slots__, and which
should be automatically assigned.
P.S.: If you reply to this message, please clearly separate
naming/syntax issues from the core issue of providing built-in support
designed to reduce clutter.

Because you don't seem to have listened in python-dev, I'll say it
here. Not everything that reduces clutter should be syntax, and not
every function, class, and module which reduces programming time should
be builtin. Why?
1. Expanding Python syntax bloats the language. It increases what you
need to teach to new Python users. In my opinion, syntax additions
should really only be considered when significant gains in readability
and writability for many users are realized, that a lack of syntax
cannot offer.

2. Expanding the Python standard library bloats the distribution.
Right now, Python is a relatively light download. But if one were to
include the top packages in everything, the distribution would quickly
bloat to 40+ megs. This is not an option. Generally, the requirements
of getting code into the standard library is either a demonstrated need
for the addition, or a known best-of-breed implementation for a
commonly used module, or both.

I believe your syntax change is a non-starter. Why? Because I've
offered code that does essentially everything you want, without a
syntax change. If someone happens to include AutoSlots and InitAttrs
with their code, module, what have you, and it manages to make its way
into standard Python, so be it (I place the code into the public
domain).

The code for the InitAttrs and AutoSlots mechanism are available here:
http://aspn.activestate.com/ASPN/Coo.../Recipe/435880

- Josiah

[Bengt Richter]

On Sat, 02 Jul 2005 13:50:25 GMT, "Andrew Koenig" <ar*@acm.org> wrote:

"Ralf W. Grosse-Kunstleve" <rw**@yahoo.com > wrote in message
news:ma******* *************** *************** **@python.org.. .

class grouping:

def __init__(self, .x, .y, .z):
# real code right here

Emulation using existing syntax::

def __init__(self, x, y, z):
self.x = x
del x
self.y = y
del y
self.z = z
del z

I think this is a bad idea, for a subtle reason.

In Python, unlike many other languages, the names of formal parameters are
part of a function's interface. For example:

def f(x, y):
return x-y

Now f(3, 4) is -1 and f(y=3,x=4) is 1.

The names of instance variables are generally not part of a class'
interface--they are part of its implementation.

This proposed feature, whenever used, would tie a class' implementation to
the interface of every method that uses the feature. As far as I can see,
it is impossible to use the feature without constraining the implementation
in this way.

For this reason, I would much rather have the mapping between parameter
names and instance variables be explicit.

What if parameter name syntax were expanded to allow dotted names as binding
targets in the local scope for the argument or default values? E.g.,

def foometh(self, self.x=0, self.y=0): pass

would have the same effect as

def foometh(self, self.y=0, self.x=0): pass

and there would be a persistent effect in the attributes of self
(whatever object that was), even with a body of pass.

I'm not sure about foo(self, **{'self.x':0, 'self.y':0}), but if
you didn't capture the dict with a **kw formal parameter, IWT you'd
have to be consistent and effect the attribute bindings implied.

(Just a non-thought-out bf here, not too serious ;-)

Regards,
Bengt Richter

[Ron Adam]

Bengt Richter wrote:

What if parameter name syntax were expanded to allow dotted names as binding
targets in the local scope for the argument or default values? E.g.,

def foometh(self, self.x=0, self.y=0): pass

would have the same effect as

def foometh(self, self.y=0, self.x=0): pass

and there would be a persistent effect in the attributes of self
(whatever object that was), even with a body of pass.

I'm not sure about foo(self, **{'self.x':0, 'self.y':0}), but if
you didn't capture the dict with a **kw formal parameter, IWT you'd
have to be consistent and effect the attribute bindings implied.

(Just a non-thought-out bf here, not too serious ;-)

Regards,
Bengt Richter

Well it works the other way around to some degree.

def foo(self, x=x, y=y):pass

x=x binds the class variables to the arguments without the self. if no
value is given.

Which is kind of strange, since x by it self gives an error if no value
is given. The strange part is x=x is not the same as just x. I
understand why, but it still looks odd.
Why isn't there a dict method to get a sub dict from a key list?
Fromkeys doesn't quite do it.

sub-dict = dict.subdict(ke y_list)

Or have dict.copy() take a key list. (?)

<Just a thought.>

The following works and doesn't seem too obscure, although the x=x,
etc.. could be annoying if they were a lot of long names.

Seems like mutable default arguments is what's needed to make it work,
not that it's needed IMO. But it's an interesting problem.
def subdict(dict, keys):
d = {}
for k in keys:
d[k] = dict[k]
return d

class foo(object):
x = 1
y = 2
z = 3
def __init__(self,x =x,y=y,z=z):
save_these = subdict(locals( ),['x','y'])
self.__dict__.u pdate(save_thes e)

# rest of code
print self.x, self.y, self.z

f = foo()
f = foo(5,6,7)