Why are functions atomic? (I.e. they are not copied.)
For example, I would like to make a copy of a function so I can change
the default values:
>>from copy import copy
f = lambda x: x
f.func_defaults = (1,)
g = copy(f)
g.func_defaults = (2,)
f(),g()
(2, 2)
I would like the following behaviour:
>>f(),g()
(1,2)
I know I could use a 'functor' defining __call__ and using member
variables, but this is more complicated and quite a bit slower. (I
also know that I can use new.function to create a new copy, but I
would like to know the rational behind the decision to make functions
atomic before I shoot myself in the foot;-)
Thanks,
Michael. 31  1750 
Michael wrote:
Why are functions atomic? (I.e. they are not copied.)
For example, I would like to make a copy of a function so I can change
the default values:
>>>from copy import copy
f = lambda x: x
f.func_defaults = (1,)
g = copy(f)
g.func_defaults = (2,)
f(),g()
(2, 2)
I would like the following behaviour:
>>>f(),g()
(1,2)
I know I could use a 'functor' defining __call__ and using member
variables, but this is more complicated and quite a bit slower. (I
also know that I can use new.function to create a new copy, but I
would like to know the rational behind the decision to make functions
atomic before I shoot myself in the foot;-)
Thanks,
Michael.
This dont make functions copiable but may resolve your default arguments
problem. Under Python 2.5, see functools.partial(). http://docs.python.org/lib/partial-objects.html
On May 1, 5:06 am, Michael <michael.for...@gmail.comwrote:
Why are functions atomic? (I.e. they are not copied.)
For example, I would like to make a copy of a function so I can change
the default values:
>from copy import copy
f = lambda x: x
f.func_defaults = (1,)
g = copy(f)
g.func_defaults = (2,)
f(),g()
(2, 2)
I would like the following behaviour:
>f(),g()
(1,2)
I know I could use a 'functor' defining __call__ and using member
variables, but this is more complicated and quite a bit slower. (I
also know that I can use new.function to create a new copy, but I
would like to know the rational behind the decision to make functions
atomic before I shoot myself in the foot;-)
Thanks,
Michael.
Does deepcopy work?
7stud <bb**********@yahoo.comwrote:
Does deepcopy work?
It doesn't copy a function.
The easiest way to make a modified copy of a function is to use the 'new'
module.
>>def f(x=2): print "x=", x
>>g = new.function(f.func_code, f.func_globals, 'g', (3,),
f.func_closure)
>>g()
x= 3
>>f()
x= 2
On 1 May 2007 15:17:48 GMT, Duncan Booth <du**********@invalid.invalidwrote:
7stud <bb**********@yahoo.comwrote:
Does deepcopy work?
It doesn't copy a function.
The easiest way to make a modified copy of a function is to use the 'new'
module.
>def f(x=2): print "x=", x
>g = new.function(f.func_code, f.func_globals, 'g', (3,),
f.func_closure)
>g()
x= 3
>f()
x= 2
--
http://mail.python.org/mailman/listinfo/python-list
The copy module considers functions to be immutable and just returns
the object. This seems pretty clearly wrong to me - functions are
clearly not immutable and it's easy to copy a function using new, as
shown above.
>From copy.py:
def _copy_immutable(x):
return x
for t in (type(None), int, long, float, bool, str, tuple,
frozenset, type, xrange, types.ClassType,
types.BuiltinFunctionType,
types.FunctionType):
d[t] = _copy_immutable
Michael wrote:
Why are functions atomic? (I.e. they are not copied.)
Because Python has objects for when you need to associate
state with a function.
John Nagle
I know I could use a 'functor' defining __call__ and using member
variables, but this is more complicated and quite a bit slower. (I
also know that I can use new.function to create a new copy, but I
would like to know the rational behind the decision to make functions
atomic before I shoot myself in the foot;-)
Function objects were not copyable because they were immutable. For
an immutable object, a copy cannot reasonably be distinguished from
the original object. See copy.py, around line 105.
Regards,
Martin
On May 1, 9:34 am, John Nagle <n...@animats.comwrote:
Michael wrote:
Why are functions atomic? (I.e. they are not copied.)
Because Python has objects for when you need to associate
state with a function.
John Nagle
Then why are functions mutable?
I can understand to some extent why functions are not picklable,
because the bytecode may not be the same across python implementations
(is that true?), but I do not understand why copying functions is a
problem. The patch that allows copy to pass-through functions just
emulates pickle, but I can find no discussion or justification for not
allowing functions to be copied: http://thread.gmane.org/gmane.comp.python.devel/76636
Michael.
>From TFM
"Function objects also support getting and setting arbitrary
attributes, which can be used, for example, to attach metadata to
functions. Regular attribute dot-notation is used to get and set such
attributes. Note that the current implementation only supports
function attributes on user-defined functions. Function attributes on
built-in functions may be supported in the future." http://docs.python.org/ref/types.html
Again, rather inconsitent with the copy sematics.
On May 1, 9:34 am, John Nagle <n...@animats.comwrote:
Because Python has objects for when you need to associate
state with a function.
John Nagle
A bit more info, but still no clear picture about why functions are
mutable but have immutable copy symantics. There are arguments why
functions should be immutable, but the decision was to make user-
defined functions mutable. My question is still: why the present
ummutable copy symantics? http://www.python.org/dev/peps/pep-0232/
On Tue, 2007-05-01 at 17:28 -0700, Michael wrote:
A bit more info, but still no clear picture about why functions are
mutable but have immutable copy symantics. There are arguments why
functions should be immutable, but the decision was to make user-
defined functions mutable. My question is still: why the present
ummutable copy symantics?
One could make a case that this is a bug, a leftover from when functions
were mostly immutable. However, one can also make a case that correcting
this bug is not worth the effort. Your use case appears to be that you
want to make multiple copies of the same function, and those copies
should be almost, but not quite, the same.
The Pythonic solution is to produce the copies by a factory function
along these lines:
>>def powerfactory(exponent):
.... def inner(x):
.... return x**exponent
.... return inner
....
>>square = powerfactory(2)
cube = powerfactory(3)
square(2)
4
>>square(3)
9
>>cube(2)
8
>>cube(3)
27
This approach makes copying functions unnecessary, and as you have
pointed out yourself, if you find yourself needing to make a copy of an
existing function you can work around the unexpected copy semantics with
new.function.
-Carsten
Michael schrieb:
A bit more info, but still no clear picture about why functions are
mutable but have immutable copy symantics. There are arguments why
functions should be immutable, but the decision was to make user-
defined functions mutable. My question is still: why the present
ummutable copy symantics?
The answer is really really simple. The implementation of copy predates
mutability. When the copy code was written, functions *were* immutable.
When functions became mutable, the copy code was not changed, and
nobody noticed or complained.
Regards,
Martin
Your use case appears to be that you
want to make multiple copies of the same function, and those copies
should be almost, but not quite, the same.
The Pythonic solution is to produce the copies by a factory function...
>def powerfactory(exponent):
... def inner(x):
... return x**exponent
... return inner
Is there a reason for using the closure here? Using function defaults
seems to give better performance:
>>def powerfactory(exponent):
.... def inner(x,exponent=exponent):
.... return x**exponent
.... return inner
This is definitely one viable solution and is essentially what I had
in mind, but I did not want to have to carry the generator arround
with me: Instead, I wanted to use it once as a decorator and then
carry only the function around.
>>@declare_options(first_option='opt1')
def f(x,opt1,opt2,opt3):
.... return x*(opt1+opt2*opt3)
>>f.set_options(opt1=1,opt2=2,opt3=3)
f(1)
7
>>from copy import copy
g = copy(f)
g.set_options(opt1=4,opt2=5,opt3=6)
f(1)
7
>>g(1)
34
The decorator declare_options behaves like the generator above, but
adds some methods (set_options) etc. to allow me to manipulate the
options without generating a new function each time.
I have functions with many options that may be called in the core of
loops, and found that the most efficient solution was to provide all
of the options through func_defaults.
>>def f(x,opt1,opt2,opt3):
.... return x*(opt1 + opt2*opt3)
The cleanest (and fastest) solution I found was to set the options in
the defaults:
>>f.func_defaults = (1,2,3)
Then f can be passed to the inner loops and f(x) is very quick.
Other options include using lists and dict's:
>>opt = (1,2,3)
f(1,*opt)
7
but then I have to pass f and opt around. This also appears to be
somewhat slower than the defaults method. Dictionaries have the
advantage of associating the names with the values
>>opt = {'opt1':1, 'opt2':2, 'opt3':3}
f(1,**opt)
7
but this is much slower. Wrapping the function with a generator as
you suggest also works and packages everything together, but again
suffers in performance. It also complicates my code.
The result of my declare_options decorator is that the result is a
regular function, complete with docstring etc. but with added
annotations that allow the options to be set. In addition, the
performance optimal. I though this was a very clean solution until I
realized that I could not make copies of the functions to allow for
different option values with the usual python copy symantics (for
example, a __copy__ method is ignored). I can easily get around this
by adding a custom copy() method, but wondered if there was anything
inherently dangerous with this approach that would justify the added
complications of more complicated wrappings and the performance hit.
Pickling is an obvious issue, but it seems like there is nothing wrong
with the copy semantics and that the limitations are artificial and
out of place. (It is also easily fixed: if the object has a __copy__
method, use it. Truely immutable objects will never have one. There
may be subtle issues here, but I don't know what they are.)
Thanks for all of the suggestions,
Michael.
Martin v. Löwis wrote:
Michael schrieb:
>>A bit more info, but still no clear picture about why functions are
mutable but have immutable copy symantics. There are arguments why
functions should be immutable, but the decision was to make user-
defined functions mutable. My question is still: why the present
ummutable copy symantics?
The answer is really really simple. The implementation of copy predates
mutability. When the copy code was written, functions *were* immutable.
When functions became mutable, the copy code was not changed, and
nobody noticed or complained.
That's probably an indication that mutable functions don't
get used all that much. Are there any instances of them in the
standard Python libraries?
John Nagle
En Wed, 02 May 2007 01:42:17 -0300, Martin v. Löwis <ma****@v.loewis.de>
escribió:
Michael schrieb:
>A bit more info, but still no clear picture about why functions are
mutable but have immutable copy symantics. There are arguments why
functions should be immutable, but the decision was to make user-
defined functions mutable. My question is still: why the present
ummutable copy symantics?
The answer is really really simple. The implementation of copy predates
mutability. When the copy code was written, functions *were* immutable.
When functions became mutable, the copy code was not changed, and
nobody noticed or complained.
Likely scenario, but not true. Support for copying user functions was
added on 2.5 (see http://svn.python.org/view/python/tr...38995&r2=42573)
and functions were mutable since a long time ago. On previous versions,
functions could be pickled but not copied.
The same thing happens for classes: they are mutable too, but copy
considers them immutable and returns the same object. This is clearly
stated on the documentation (but the module docstring is still outdated).
--
Gabriel Genellina
On Tue, 2007-05-01 at 22:21 -0700, Michael wrote:
Is there a reason for using the closure here? Using function defaults
seems to give better performance:[...]
It does? Not as far as I can measure it to any significant degree on my
computer.
This is definitely one viable solution and is essentially what I had
in mind, but I did not want to have to carry the generator arround
with me:
I don't know what you mean by "carry it around." Just put it in a module
and import it where you need it.
An overriding theme in this thread is that you are greatly concerned
with the speed of your solution rather than the structure and
readability of your code. How often is your function going to get called
and how much of a performance benefit are you expecting?
-Carsten
Michael <mi************@gmail.comwrote:
Is there a reason for using the closure here? Using function defaults
seems to give better performance:
What measurements show you that...?
brain:~ alex$ cat powi.py
def powerfactory1(exponent):
def inner(x):
return x**exponent
return inner
def powerfactory2(exponent):
def inner(x, exponent=exponent):
return x**exponent
return inner
brain:~ alex$ python -mtimeit -s'import powi; p=powi.powerfactory1(3)'
'p(27)'
1000000 loops, best of 3: 0.485 usec per loop
brain:~ alex$ python -mtimeit -s'import powi; p=powi.powerfactory2(3)'
'p(27)'
1000000 loops, best of 3: 0.482 usec per loop
Alex
>The answer is really really simple. The implementation of copy predates
>mutability. When the copy code was written, functions *were* immutable.
When functions became mutable, the copy code was not changed, and
nobody noticed or complained.
That's probably an indication that mutable functions don't
get used all that much. Are there any instances of them in the
standard Python libraries?
All user-defined functions are mutable. To me, this issue is rather that
copying of functions isn't used all that much.
In addition, it is certainly true that functions are rarely modified,
even though all of them are mutable. See this for an example
pydef foo():pass
....
pyfoo
<function foo at 0xb7db5f44>
pyfoo.__name__='bar'
pyfoo
<function bar at 0xb7db5f44>
pyfoo.value
Traceback (most recent call last):
File "<stdin>", line 1, in ?
AttributeError: 'function' object has no attribute 'value'
pyfoo.value=100
pyfoo.value
100
Regards,
Martin
>The answer is really really simple. The implementation of copy predates
>mutability. When the copy code was written, functions *were* immutable.
When functions became mutable, the copy code was not changed, and
nobody noticed or complained.
Likely scenario, but not true.
Interesting. I (clearly) missed that part of the defaultdict discussion
(that was a long thread).
Regards,
Martin al***@mac.com (Alex Martelli) wrote:
>Is there a reason for using the closure here? Using function
defaults seems to give better performance:
What measurements show you that...?
....
>
brain:~ alex$ python -mtimeit -s'import powi; p=powi.powerfactory1(3)'
'p(27)'
1000000 loops, best of 3: 0.485 usec per loop
brain:~ alex$ python -mtimeit -s'import powi; p=powi.powerfactory2(3)'
'p(27)'
1000000 loops, best of 3: 0.482 usec per loop
Your own benchmark seems to support Michael's assertion although the
difference in performance is so slight that it is unlikely ever to
outweigh the loss in readability.
Modifying powi.py to reduce the weight of the function call overhead and
the exponent operation indicates that using default arguments is faster,
but you have to push it to quite an extreme case before it becomes
significant:
def powerfactory1(exponent, plus):
def inner(x):
for i in range(1000):
res = x+exponent+plus
return res
return inner
def powerfactory2(exponent, plus):
def inner(x, exponent=exponent, plus=plus):
for i in range(1000):
res = x+exponent+plus
return res
return inner
C:\Temp>\python25\python -mtimeit -s "import powi; p=powi.powerfactory1
(3,999)" "p(27)"
10000 loops, best of 3: 159 usec per loop
C:\Temp>\python25\python -mtimeit -s "import powi; p=powi.powerfactory2
(3,999)" "p(27)"
10000 loops, best of 3: 129 usec per loop
On May 2, 6:08 am, Carsten Haese <cars...@uniqsys.comwrote:
On Tue, 2007-05-01 at 22:21 -0700, Michael wrote:
Is there a reason for using the closure here? Using function defaults
seems to give better performance:[...]
It does? Not as far as I can measure it to any significant degree on my
computer.
I agree the performance gains are minimal. Using function defaults
rather than closures, however, seemed much cleaner an more explicit to
me. For example, I have been bitten by the following before:
>>def f(x):
.... def g():
.... x = x + 1
.... return x
.... return g
>>g = f(3)
g()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<stdin>", line 3, in g
UnboundLocalError: local variable 'x' referenced before assignment
If you use default arguments, this works as expected:
>>def f(x):
.... def g(x=x):
.... x = x + 1
.... return x
.... return g
>>g = f(3)
g()
4
The fact that there also seems to be a performance gain (granted, it
is extremely slight here) led me to ask if there was any advantage to
using closures. It seems not.
An overriding theme in this thread is that you are greatly concerned
with the speed of your solution rather than the structure and
readability of your code.
Yes, it probably does seem that way, because I am burying this code
deeply and do not want to revisit it when profiling later, but my
overriding concern is reliability and ease of use. Using function
attributes seemed the best way to achieve both goals until I found out
that the pythonic way of copying functions failed. Here was how I
wanted my code to work:
@define_options(first_option='abs_tol')
def step(f,x,J,abs_tol=1e-12,rel_tol=1e-8,**kwargs):
"""Take a step to minimize f(x) using the jacobian J.
Return (new_x,converged) where converged is true if the tolerance
has been met.
"""
<compute dx and check convergence>
return (x + dx, converged)
@define_options(first_option='min_h')
def jacobian(f,x,min_h=1e-6,max_h=0.1):
"""Compute jacobian using a step min_h < h < max_h."""
<compute J>
return J
class Minimizer(object):
"""Object to minimize a function."""
def __init__(self,step,jacobian,**kwargs):
self.options = step.options + jacobian.options
self.step = step
self.jacobian = jacobian
def minimize(self,f,x0,**kwargs):
"""Minimize the function f(x) starting at x0."""
step = self.step
jacobian = self.jacobian
step.set_options(**kwargs)
jacobian.set_options(**kwargs)
converged = False
while not converged:
J = jacobian(f,x)
(x,converged) = step(f,x,J)
return x
@property
def options(self):
"""List of supported options."""
return self.options
The idea is that one can define different functions for computing the
jacobian, step etc. that take various parameters, and then make a
custom minimizer class that can provide the user with information
about the supported options etc.
The question is how to define the decorator define_options?
1) I thought the cleanest solution was to add a method f.set_options()
which would set f.func_defaults, and a list f.options for
documentation purposes. The docstring remains unmodified without any
special "wrapping", step and jacobian are still "functions" and
performance is optimal.
2) One could return an instance f of a class with f.__call__,
f.options and f.set_options defined. This would probably be the most
appropriate OO solution, but it makes the decorator much more messy,
or requires the user to define classes rather than simply define the
functions as above. In addition, this is at least a factor of 2.5
timese slower on my machine than option 1) because of the class
instance overhead. (This is my only real performance concern because
this is quite a large factor. Otherwise I would just use this
method.)
3) I could pass generators to Minimize and construct the functions
dynamically. This would have the same performance, but would require
the user to define generators, or require the decorator to return a
generator when the user appears to be defining a function. This just
seems much less elegant.
....
@define_options_generator(first_option='min_h')
def jacobian_gen(f,x,min_h=1e-6,max_h=0.1):
"""Compute jacobian using a step min_h < h < max_h."""
<compute J>
return J
class Minimizer(object):
"""Object to minimize a function."""
def __init__(self,step_gen,jacobian_gen,**kwargs):
self.options = step_gen.options + jacobian_gen.options
self.step_gen = step_gen
self.jacobian_gen = jacobian_gen
def minimize(self,f,x0,**kwargs):
"""Minimize the function f(x) starting at x0."""
step = self.step_gen(**kwargs)
jacobian = self.jacobian_gen(**kwargs)
converged = False
while not converged:
J = jacobian(f,x)
(x,converged) = step(f,x,J)
return x
...
4) Maybe there is a better, cleaner way to do this, but I thought that
my option 1) was the most clear, readable and fast. I would
appreciate any suggestions. The only problem is that it does use
mutable functions, and so the user might be tempted to try:
new_step = copy(step)
which would fail (because modifying new_step would also modify step).
I guess that this is a pretty big problem (I could provide a custom
copy function so that
new_step = step.copy()
would work) and I wondered if there was a better solution (or if maybe
copy.py should be fixed. Checking for a defined __copy__ method
*before* checking for pre-defined mutable types does not seem to break
anything.)
Thanks again everyone for your suggestions, it is really helping me
learn about python idioms.
Michael.
On May 4, 1:36 am, Michael <michael.for...@gmail.comwrote:
On May 2, 6:08 am, Carsten Haese <cars...@uniqsys.comwrote:
On Tue, 2007-05-01 at 22:21 -0700, Michael wrote:
Is there a reason for using the closure here? Using function defaults
seems to give better performance:[...]
It does? Not as far as I can measure it to any significant degree on my
computer.
I agree the performance gains are minimal. Using function defaults
rather than closures, however, seemed much cleaner an more explicit to
me. For example, I have been bitten by the following before:
>def f(x):
... def g():
... x = x + 1
... return x
... return g>>g = f(3)
>g()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<stdin>", line 3, in g
UnboundLocalError: local variable 'x' referenced before assignment
If you use default arguments, this works as expected:>>def f(x):
... def g(x=x):
... x = x + 1
... return x
... return g
>g = f(3)
g()
4
>>g()
4
>>g()
4
>>g() # what is going on here????
4
The fact that there also seems to be a performance gain (granted, it
is extremely slight here) led me to ask if there was any advantage to
using closures. It seems not.
An overriding theme in this thread is that you are greatly concerned
with the speed of your solution rather than the structure and
readability of your code.
Yes, it probably does seem that way, because I am burying this code
deeply and do not want to revisit it when profiling later, but my
overriding concern is reliability and ease of use. Using function
attributes seemed the best way to achieve both goals until I found out
that the pythonic way of copying functions failed. Here was how I
wanted my code to work:
@define_options(first_option='abs_tol')
def step(f,x,J,abs_tol=1e-12,rel_tol=1e-8,**kwargs):
"""Take a step to minimize f(x) using the jacobian J.
Return (new_x,converged) where converged is true if the tolerance
has been met.
"""
<compute dx and check convergence>
return (x + dx, converged)
@define_options(first_option='min_h')
def jacobian(f,x,min_h=1e-6,max_h=0.1):
"""Compute jacobian using a step min_h < h < max_h."""
<compute J>
return J
class Minimizer(object):
"""Object to minimize a function."""
def __init__(self,step,jacobian,**kwargs):
self.options = step.options + jacobian.options
self.step = step
self.jacobian = jacobian
def minimize(self,f,x0,**kwargs):
"""Minimize the function f(x) starting at x0."""
step = self.step
jacobian = self.jacobian
step.set_options(**kwargs)
jacobian.set_options(**kwargs)
converged = False
while not converged:
J = jacobian(f,x)
(x,converged) = step(f,x,J)
return x
@property
def options(self):
"""List of supported options."""
return self.options
The idea is that one can define different functions for computing the
jacobian, step etc. that take various parameters, and then make a
custom minimizer class that can provide the user with information
about the supported options etc.
The question is how to define the decorator define_options?
1) I thought the cleanest solution was to add a method f.set_options()
which would set f.func_defaults, and a list f.options for
documentation purposes. The docstring remains unmodified without any
special "wrapping", step and jacobian are still "functions" and
performance is optimal.
2) One could return an instance f of a class with f.__call__,
f.options and f.set_options defined. This would probably be the most
appropriate OO solution, but it makes the decorator much more messy,
or requires the user to define classes rather than simply define the
functions as above. In addition, this is at least a factor of 2.5
timese slower on my machine than option 1) because of the class
instance overhead. (This is my only real performance concern because
this is quite a large factor. Otherwise I would just use this
method.)
3) I could pass generators to Minimize and construct the functions
dynamically. This would have the same performance, but would require
the user to define generators, or require the decorator to return a
generator when the user appears to be defining a function. This just
seems much less elegant.
...
@define_options_generator(first_option='min_h')
def jacobian_gen(f,x,min_h=1e-6,max_h=0.1):
"""Compute jacobian using a step min_h < h < max_h."""
<compute J>
return J
class Minimizer(object):
"""Object to minimize a function."""
def __init__(self,step_gen,jacobian_gen,**kwargs):
self.options = step_gen.options + jacobian_gen.options
self.step_gen = step_gen
self.jacobian_gen = jacobian_gen
def minimize(self,f,x0,**kwargs):
"""Minimize the function f(x) starting at x0."""
step = self.step_gen(**kwargs)
jacobian = self.jacobian_gen(**kwargs)
converged = False
while not converged:
J = jacobian(f,x)
(x,converged) = step(f,x,J)
return x
...
4) Maybe there is a better, cleaner way to do this, but I thought that
my option 1) was the most clear, readable and fast. I would
appreciate any suggestions. The only problem is that it does use
mutable functions, and so the user might be tempted to try:
new_step = copy(step)
which would fail (because modifying new_step would also modify step).
I guess that this is a pretty big problem (I could provide a custom
copy function so that
new_step = step.copy()
would work) and I wondered if there was a better solution (or if maybe
copy.py should be fixed. Checking for a defined __copy__ method
*before* checking for pre-defined mutable types does not seem to break
anything.)
Thanks again everyone for your suggestions, it is really helping me
learn about python idioms.
Michael.
On 3 May 2007 23:36:11 -0700, Michael <mi************@gmail.comwrote:
On May 2, 6:08 am, Carsten Haese <cars...@uniqsys.comwrote:
On Tue, 2007-05-01 at 22:21 -0700, Michael wrote:
Is there a reason for using the closure here? Using function defaults
seems to give better performance:[...]
It does? Not as far as I can measure it to any significant degree on my
computer.
I agree the performance gains are minimal. Using function defaults
rather than closures, however, seemed much cleaner an more explicit to
me. For example, I have been bitten by the following before:
>def f(x):
... def g():
... x = x + 1
... return x
... return g
>g = f(3)
g()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<stdin>", line 3, in g
UnboundLocalError: local variable 'x' referenced before assignment
If you use default arguments, this works as expected:
>def f(x):
... def g(x=x):
... x = x + 1
... return x
... return g
>g = f(3)
g()
4
You aren't getting "bit" by any problem with closures - this is a
syntax problem.
Assigning to x within the scope of g() makes x a local variable. x =
x+1 doesn't work, then, because "x+1" can't be evaluated.
If you just used "return x+1" or if you named the local variable in g
something other than "x", the closure approach would also work as
expected.
Michael wrote:
On May 2, 6:08 am, Carsten Haese <cars...@uniqsys.comwrote:
>>On Tue, 2007-05-01 at 22:21 -0700, Michael wrote:
I agree the performance gains are minimal. Using function defaults
rather than closures, however, seemed much cleaner an more explicit to
me. For example, I have been bitten by the following before:
>>>>def f(x):
... def g():
... x = x + 1
Too cute. Don't nest functions in Python; the scoping model
isn't really designed for it.
>>An overriding theme in this thread is that you are greatly concerned
with the speed of your solution rather than the structure and
readability of your code.
....
>
@define_options(first_option='abs_tol')
def step(f,x,J,abs_tol=1e-12,rel_tol=1e-8,**kwargs):
"""Take a step to minimize f(x) using the jacobian J.
Return (new_x,converged) where converged is true if the tolerance
has been met.
"""
Python probably isn't the right language for N-dimensional optimization
if performance is a major concern. That's a very compute-intensive operation.
I've done it in C++, with heavy use of inlines, and had to work hard to
get the performance up. (I was one of the first to do physics engines for
games and animation, which is a rather compute-intensive problem.)
If you're doing number-crunching in Python, it's essential to use
NumPy or some other C library for matrix operations, or it's going to
take way too long.
John Nagle
On May 4, 5:49 am, "Chris Mellon" <arka...@gmail.comwrote:
You aren't getting "bit" by any problem with closures - this is a
syntax problem.
I understand that it is not closures that are specifically biting me.
However, I got bit, it was unplesant and I don't want to be bit
again;-)
Thus, whenever I need to pass information to a function, I use default
arguments now. Is there any reason not to do this other than the fact
that it is a bit more typing?
Michael
On May 4, 9:19 am, John Nagle <n...@animats.comwrote:
... def g():
... x = x + 1
Too cute. Don't nest functions in Python; the scoping model
isn't really designed for it.
How can you make generators then if you don't nest?
Python probably isn't the right language for N-dimensional optimization
if performance is a major concern. That's a very compute-intensive operation.
I've done it in C++, with heavy use of inlines, and had to work hard to
get the performance up. (I was one of the first to do physics engines for
games and animation, which is a rather compute-intensive problem.)
If you're doing number-crunching in Python, it's essential to use
NumPy or some other C library for matrix operations, or it's going to
take way too long.
I know. I am trying to flesh out a modular optimization proposal for
SciPy. Using C++ would defeat the purpose of making it easy to extend
the optimizers. I just want to make things as clean and efficient as
possible when I stumbled on this python copy problem.
Michael.
On 4 May 2007 12:55:03 -0700, Michael <mi************@gmail.comwrote:
On May 4, 5:49 am, "Chris Mellon" <arka...@gmail.comwrote:
You aren't getting "bit" by any problem with closures - this is a
syntax problem.
I understand that it is not closures that are specifically biting me.
However, I got bit, it was unplesant and I don't want to be bit
again;-)
Thus, whenever I need to pass information to a function, I use default
arguments now. Is there any reason not to do this other than the fact
that it is a bit more typing?
There are different semantics when the thing you're passing is
mutable. There's also different semantics when it's rebound within the
calling scope, but then the default argument technique is probably
what you want.
Michael
--
http://mail.python.org/mailman/listinfo/python-list
On 4 May 2007 12:59:39 -0700, Michael <mi************@gmail.comwrote:
On May 4, 9:19 am, John Nagle <n...@animats.comwrote:
... def g():
... x = x + 1
Too cute. Don't nest functions in Python; the scoping model
isn't really designed for it.
How can you make generators then if you don't nest?
There's all kinds of good reasons to nest functions, and the "scoping
model isn't really designed for it" somewhat overstates the case -
it's not relevant to many of the reasons you might nest functions, and
it's not (much) of a problem for the rest of them. What you can't do
is rebind values in the enclosing scope, unless the enclosing scope is
global. That's a real, but fairly minor, limitation and you'll be able
to explicitly address your enclosing scope in 3k (or perhaps sooner).
On May 4, 4:13 am, Dustan <DustanGro...@gmail.comwrote:
On May 4, 1:36 am, Michael <michael.for...@gmail.comwrote:
... def g(x=x):
... x = x + 1
... return x
... return g
>>g = f(3)
>>g()>
4
>g()
4
>g()
4
>g() # what is going on here????
4
Okay, so it is a bad design, but it illustrates the point. What is
happening is that in the body of the function f, a new function is
defined using the value of x passed as an argument to f. Thus, after
the call g = f(3), the body of f is equivalent to
def g(x=3):
x = x + 1
return x
This function is returned, so the call g() uses the default argument
x=3, then computes x = x+1 = 3+1 = 4 and returns 4. Every call is
equivalent to g() == g(3) = 4. Inside g, x is a local variable: it
does not maintain state between function calls. (You might think that
the first example would allow you to mutate the x in the closure, but
this is dangerous and exactly what python is trying to prevent by
making x a local variable when you make assignments in g. This is why
the interpreter complains.)
If you actually want to maintain state, you have to use a mutable
object like a list. The following would do what you seem to expect.
>>def f(x0):
.... def g(x=[x0]):
.... x[0] = x[0] + 1
.... return x[0]
.... return g
....
>>g = f(3)
g()
4
>>g()
5
>>h = f(0)
h()
1
>>h()
2
>>g()
6
Michael <mi************@gmail.comwrote:
Thus, whenever I need to pass information to a function, I use default
arguments now. Is there any reason not to do this other than the fact
that it is a bit more typing?
You're giving your functions a signature that's different from the one
you expect it to be called with, and so making it impossible for the
Python runtime to diagnose certain errors on the caller's part.
For example, consider:
def makecounter_good():
counts = {}
def count(item):
result = counts[item] = 1 + counts.get(item, 0)
return result
return count
c = makecounter_good()
for i in range(3): print c(23)
def makecounter_hmmm():
counts = {}
def count(item, counts=counts):
result = counts[item] = 1 + counts.get(item, 0)
return result
return count
cc = makecounter_hmmm()
for i in range(3): print cc(23)
print cc(23, {})
print c(23, {})
Counters made by makecounter_good take exactly one argument, and
properly raise exceptions if incorrectly called with two; counters made
by makecounter_hmmm take two arguments (of which one is optional), and
thus hide some runtime call errors.
From "import this":
"""
Errors should never pass silently.
Unless explicitly silenced.
"""
The miniscule "optimization" of giving a function an argument it's not
_meant_ to have somewhat breaks this part of the "Zen of Python", and
thus I consider it somewhat unclean.
Alex
On May 4, 7:54 pm, a...@mac.com (Alex Martelli) wrote:
Michael <michael.for...@gmail.comwrote:
Thus, whenever I need to pass information to a function, I use default
arguments now. Is there any reason not to do this other than the fact
that it is a bit more typing?
You're giving your functions a signature that's different from the one
you expect it to be called with, and so making it impossible for the
Python runtime to diagnose certain errors on the caller's part.
....
The miniscule "optimization" of giving a function an argument it's not
_meant_ to have somewhat breaks this part of the "Zen of Python", and
thus I consider it somewhat unclean.
That is a pretty good reason in some contexts. Usually, the arguments
I pass are values that the user might like to change, so the kwarg
method often serves an explicit purpose allowing parameters to be
modified, but I can easily imagine cases where the extra arguments
should really not be there. I still like explicitly stating the
dependencies of a function, but I suppose I could do that with
decorators.
Thanks,
Michael. This thread has been closed and replies have been disabled. Please start a new discussion. Similar topics | |
by: David MacQuigg |
last post by:
I'm not getting any feedback on the most important benefit in my
proposed "Ideas for Python 3" thread - the unification of methods and
functions. Perhaps it was buried among too many other less...
|
by: Glenn Kasten |
last post by:
I am wondering which operations in Python
are guaranteed to be atomic in the presence
of multi-threading. In particular, are assignment
and reading of a dictionary entry atomic?
For example,...
|
by: Shayan |
last post by:
Is there a boolean flag that can be set atomically without needing to
wrap it in a mutex? This flag will be checked constantly by multiple
threads so I don't really want to deal with the overhead...
|
by: Pegboy |
last post by:
What does it mean to make a function atomic?
Something I read on it wasn't very clear, but made me think that I needed to
disable interrupts for that function. Whether that's the case or not,...
|
by: robert |
last post by:
In very rare cases a program crashes (hard to reproduce) :
* several threads work on an object tree with dict's etc. in it. Items
are added, deleted, iteration over .keys() ... ). The threads are...
| | |
by: blackstreetcat |
last post by:
consider this code :
int i; //gobal var
Thread1:
i=some value;
Thread2:
if (i==2) dosomething();
else dosomethingelse();
|
by: japhy |
last post by:
Is there a way to read a line (a series of characters ending in a
newline) from a file (either by descriptor or stream) atomically,
without buffering additional contents of the file?
|
by: yogeeswar |
last post by:
Hi All
I wrote SP with number of delete commands in an atomic block.And there is a possibility of deleting records from parent table before child table so I wrote a handler to handle the...
|
by: Freedom fighter |
last post by:
Hello,
Is a singleton class the same as an atomic class? I know that a singleton class
can only be instantiated once, but does that concept apply to an atomic class?
Thank you.
|
by: marktang |
last post by:
ONU (Optical Network Unit) is one of the key components for providing high-speed Internet services. Its primary function is to act as an endpoint device located at the user's premises. However,...
|
by: Hystou |
last post by:
Most computers default to English, but sometimes we require a different language, especially when relocating. Forgot to request a specific language before your computer shipped? No problem! You can...
| | |
by: Oralloy |
last post by:
Hello folks,
I am unable to find appropriate documentation on the type promotion of bit-fields when using the generalised comparison operator "<=>".
The problem is that using the GNU compilers,...
|
by: jinu1996 |
last post by:
In today's digital age, having a compelling online presence is paramount for businesses aiming to thrive in a competitive landscape. At the heart of this digital strategy lies an intricately woven...
|
by: tracyyun |
last post by:
Dear forum friends,
With the development of smart home technology, a variety of wireless communication protocols have appeared on the market, such as Zigbee, Z-Wave, Wi-Fi, Bluetooth, etc. Each...
|
by: agi2029 |
last post by:
Let's talk about the concept of autonomous AI software engineers and no-code agents. These AIs are designed to manage the entire lifecycle of a software development project—planning, coding, testing,...
|
by: conductexam |
last post by:
I have .net C# application in which I am extracting data from word file and save it in database particularly. To store word all data as it is I am converting the whole word file firstly in HTML and...
|
by: TSSRALBI |
last post by:
Hello
I'm a network technician in training and I need your help.
I am currently learning how to create and manage the different types of VPNs and I have a question about LAN-to-LAN VPNs.
The...
| | |
by: bsmnconsultancy |
last post by:
In today's digital era, a well-designed website is crucial for businesses looking to succeed. Whether you're a small business owner or a large corporation in Toronto, having a strong online presence...
| |
