Weird lambda rebinding/reassignment without me doing it

[ssecorp]

I am never redefining the or reassigning the list when using validate
but since it spits the modified list back out that somehow means that
the modified list is part of the environment and not the old one.
i thought what happend inside a function stays inside a function
meaning what comes out is independent of what comes in.
Meaning if I want the list I send as a parameter to the function I
have to do x = func(x) and not just func(x) and x is magically what
comes out of func().
Doesnt Python have closure or that isnt what this is about?
def validate(placed):
student = round(random.random()*401)
if student in placed:
return validate(placed)
else:
placed.append(student)
return student, placed

def val(placed):
student = round(random.random()*401)
if student in placed:
return validate(placed)
else:
placed.append(student)
return student

>>g = lambda x:validate(x)
l=[]
for x in range(1,10):

g(l)
(141.0, [141.0])
(19.0, [141.0, 19.0])
(86.0, [141.0, 19.0, 86.0])
(120.0, [141.0, 19.0, 86.0, 120.0])
(76.0, [141.0, 19.0, 86.0, 120.0, 76.0])
(262.0, [141.0, 19.0, 86.0, 120.0, 76.0, 262.0])
(234.0, [141.0, 19.0, 86.0, 120.0, 76.0, 262.0, 234.0])
(74.0, [141.0, 19.0, 86.0, 120.0, 76.0, 262.0, 234.0, 74.0])
(325.0, [141.0, 19.0, 86.0, 120.0, 76.0, 262.0, 234.0, 74.0, 325.0])

>>g = lambda x:val(x)
l=[]
for x in range(1,10):

g(l)
183.0
33.0
315.0
244.0
308.0
168.0
146.0
378.0
297.0

>>>

[A.T.Hofkamp]

Python doesn't use value semantics for variables but reference semantics:

a = [1]
b = a

In many languages, you'd now have 2 lists. In Python you still have one list,
and both a and b refer to it.

Now if you modify the data (the list), both variables will change

a.append(2) # in-place modification of the list
print b # will print [1,2]
If you don't want this, you should make a copy somewhere

a = a + [2]

or

b = a[:]
It takes a bit of getting used to, but it is very handy in practice.
Read the docs about 'immutable' and 'mutable' types.
(I expected this to be a FAQ, but I couldn't quickly find a matching entry)
Albert

[David C. Ullrich]

In article
<02**********************************@z72g2000hsb. googlegroups.com>,
ssecorp <ci**********@gmail.comwrote:

I am never redefining the or reassigning the list when using validate
but since it spits the modified list back out that somehow means that
the modified list is part of the environment and not the old one.
i thought what happend inside a function stays inside a function
meaning what comes out is independent of what comes in.
Meaning if I want the list I send as a parameter to the function I
have to do x = func(x) and not just func(x) and x is magically what
comes out of func().

A function cannot modify the value of a global variable
(unless it specifies "global"). It doesn't reassign anything.
But in the functions below you're not reassigning a variable,
you're _modifiying_ an object. A function _can_ modify an
object you pass to it:

>>def DoesntReassign(x):

.... x = 0
....

>>def Modifies(x):

.... x.append(0)
....

>>x=42
DoesntReassign(x)
x

42

>>x=[]
Modifies(x)
x

[0]

Hmm, look at this:

>>x=[]
id(x)

404296

>>Modifies(x)
x

[0]

>>id(x)

404296

'x' refers to exactly the same object before and after
the call to Modifies. The function has _modified_ that
object, but it hasn't redefined or reassigned anything.

Doesnt Python have closure or that isnt what this is about?
def validate(placed):
student = round(random.random()*401)
if student in placed:
return validate(placed)
else:
placed.append(student)
return student, placed

def val(placed):
student = round(random.random()*401)
if student in placed:
return validate(placed)
else:
placed.append(student)
return student

>g = lambda x:validate(x)
l=[]
for x in range(1,10):

g(l)
(141.0, [141.0])
(19.0, [141.0, 19.0])
(86.0, [141.0, 19.0, 86.0])
(120.0, [141.0, 19.0, 86.0, 120.0])
(76.0, [141.0, 19.0, 86.0, 120.0, 76.0])
(262.0, [141.0, 19.0, 86.0, 120.0, 76.0, 262.0])
(234.0, [141.0, 19.0, 86.0, 120.0, 76.0, 262.0, 234.0])
(74.0, [141.0, 19.0, 86.0, 120.0, 76.0, 262.0, 234.0, 74.0])
(325.0, [141.0, 19.0, 86.0, 120.0, 76.0, 262.0, 234.0, 74.0, 325.0])

>g = lambda x:val(x)
l=[]
for x in range(1,10):

g(l)
183.0
33.0
315.0
244.0
308.0
168.0
146.0
378.0
297.0

>>

--
David C. Ullrich

[Terry Reedy]

David C. Ullrich wrote:

In article
<02**********************************@z72g2000hsb. googlegroups.com>,
ssecorp <ci**********@gmail.comwrote:

>I am never redefining the or reassigning the list when using validate
but since it spits the modified list back out that somehow means that
the modified list is part of the environment and not the old one.
i thought what happend inside a function stays inside a function
meaning what comes out is independent of what comes in.
Meaning if I want the list I send as a parameter to the function I
have to do x = func(x) and not just func(x) and x is magically what
comes out of func().

A function cannot modify the value of a global variable

Yes it can.

>>a=[]
def f():

a.append('yes I can')

>>f()
a

['yes I can']

(unless it specifies "global"). It doesn't reassign anything.

The statement 'global a' would allow f to *rebind* the global *name*
'a'. The word 'variable' should almost not be used in discussing Python
since it is often unclear whether it refers to a name (or collection
slot) or an object bound thereto.

But in the functions below you're not reassigning a variable,
you're _modifiying_ an object. A function _can_ modify an
object you pass to it:

It can modify any mutable object it can access.

>Doesnt Python have closure or that isnt what this is about?

Python does have closures. This is not about that.

>def validate(placed):
student = round(random.random()*401)
if student in placed:
return validate(placed)
else:
placed.append(student)
return student, placed

Delete this. It is redundant with the below.

>def val(placed):
student = round(random.random()*401)
if student in placed:
return validate(placed)
else:
placed.append(student)
return student

I believe this is equivalent to

def addval(placed):
while True:
student = round(random.random()*401)
if student not in placed:
break
placed.append(student)
return student

While this avoids the indefinite recursion depth problem, it does not
avoid the indefinite time problem. Use random.shuffle, or write your
own version if doing this for practice. Also consider removing the
return statement unless you actually directly use the added value. It
is easier to remember that addval mutates 'placed' without the return.

>>>>g = lambda x:validate(x)

This is doubly diseased.

First, never write a 'name = lambda...' statement since it is equivalent
to a def statement except that the resulting function object lacks a
proper .funcname attribute. The above only trivially abbreviates
def g(x): return validate(x)
by 3 characters. Another reason is that the lambda form somehow more
often tricks people into the next mistake .

Second, never write a function (with either def or lambda) that simply
returns a function of the argument(s). The wrapping does nothing! This
is a waste of time and space for both you and the interpreter. The
above is functionally equivalent to
g = validate
and if you want that, you could name the function 'g' when you define it.

>>>>l=[]

In some fonts, 'l' and '1' are nearly identical; please use something
else for public code, which you made this to be by posting it;-)

>>>>for x in range(1,10):

g(l)

As said, the 'g' wrapper is useless.
addval(l)

Hope this helps.

Terry Jan Reedy

[ssecorp]

ty very good answer. i know i shouldn't use lambda like that, i never
do i was just playing around there and then this happened which i
thought was weird.


On Jul 10, 8:09*pm, Terry Reedy <[e-mail]:

David C. Ullrich wrote:

In article
<[e-mail]>,
*ssecorp <[e-mail]:

I am never redefining the or reassigning the list when using validate
but since it spits the modified list back out that somehow means that
the modified list is part of the environment and not the old one.
i thought what happend inside a function stays inside a function
meaning what comes out is independent of what comes in.
Meaning if I want the list I send as a parameter to the function I
have to do x = func(x) and not just func(x) and x is magically what
comes out of func().

A function cannot modify the value of a global variable

Yes it can.
*>>a=[]
*>>def f():
* * * * a.append('yes I can')

*>>f()
*>>a
['yes I can']

(unless it specifies "global"). It doesn't reassign anything.

The statement 'global a' would allow f to *rebind* the global *name*
'a'. *The word 'variable' should almost not be used in discussing Python
since it is often unclear whether it refers to a name (or collection
slot) or an object bound thereto.

But in the functions below you're not reassigning a variable,
you're _modifiying_ an object. A function _can_ modify an
object you pass to it:

It can modify any mutable object it can access.

Doesnt Python have closure or that isnt what this is about?

Python does have closures. *This is not about that.

def validate(placed):
* * student = round(random.random()*401)
* * if student in placed:
* * * * return validate(placed)
* * else:
* * * * placed.append(student)
* * * * return student, placed

Delete this. It is redundant with the below.

def val(placed):
* * student = round(random.random()*401)
* * if student in placed:
* * * * return validate(placed)
* * else:
* * * * placed.append(student)
* * * * return student

I believe this is equivalent to

def addval(placed):
* *while True:
* * *student = round(random.random()*401)
* * *if student not in placed:
* * * *break
* *placed.append(student)
* *return student

While this avoids the indefinite recursion depth problem, it does not
avoid the indefinite time problem. *Use random.shuffle, or write your
own version if doing this for practice. *Also consider removing the
return statement unless you actually directly use the added value. *It
is easier to remember that addval mutates 'placed' without the return.

>>>g = lambda x:validate(x)

This is doubly diseased.

First, never write a 'name = lambda...' statement since it is equivalent
to a def statement except that the resulting function object lacks a
proper .funcname attribute. *The above only trivially abbreviates
* *def g(x): return validate(x)
by 3 characters. *Another reason is that the lambda form somehow more
often tricks people into the next mistake .

Second, never write a function (with either def or lambda) that simply
returns a function of the argument(s). *The wrapping does nothing! *This
is a waste of time and space for both you and the interpreter. *The
above is functionally equivalent to
* *g = validate
and if you want that, you could name the function 'g' when you define it.

>>>l=[]

In some fonts, 'l' and '1' are nearly identical; please use something
else for public code, which you made this to be by posting it;-)

>>>for x in range(1,10):
* * * *g(l)

As said, the 'g' wrapper is useless.
* * * * addval(l)

Hope this helps.

Terry Jan Reedy

[ssecorp]

>>def mod(x,y):
return x.append(y)

>>mod([1,2],3)
k=[1,2,3]
k

[1, 2, 3]

>>l = mod(k,4)
l
k

[1, 2, 3, 4]

>>l
k==l

False

>>mod(k,5)
k

[1, 2, 3, 4, 5]

>>mod(l,4)

Traceback (most recent call last):
File "<pyshell#29>", line 1, in <module>
mod(l,4)
File "<pyshell#18>", line 2, in mod
return x.append(y)
AttributeError: 'NoneType' object has no attribute 'append'

>>l
l=k
l

[1, 2, 3, 4, 5]

>>i=mod(k,1)
i

same stuff but i dont find this intuitive.

[MRAB]

On Jul 10, 9:46*pm, ssecorp <[e-mail]:

>def mod(x,y):

* * * * return x.append(y)

append adds y to list x and returns None, which is then returned by
mod.

>

>mod([1,2],3)
k=[1,2,3]
k

[1, 2, 3]

>l = mod(k,4)

4 has been appended to list k and mod has returned None, so l is now
None.

>l
k

[1, 2, 3, 4]

>l
k==l

False

Because k is [1, 2, 3, 4] and l is None.

>mod(k,5)
k

[1, 2, 3, 4, 5]

>mod(l,4)

Traceback (most recent call last):
* File "<pyshell#29>", line 1, in <module>
* * mod(l,4)
* File "<pyshell#18>", line 2, in mod
* * return x.append(y)
AttributeError: 'NoneType' object has no attribute 'append'

l is None, not a list, so it complains!

>l
l=k
l

[1, 2, 3, 4, 5]

>i=mod(k,1)
i

same stuff but i dont find this intuitive.

It's usual for methods which modify to return None.

[David C. Ullrich]

In article <ma**************************************@python.o rg>,
Terry Reedy <tj*****@udel.eduwrote:

David C. Ullrich wrote:

In article
<02**********************************@z72g2000hsb. googlegroups.com>,
ssecorp <ci**********@gmail.comwrote:

I am never redefining the or reassigning the list when using validate
but since it spits the modified list back out that somehow means that
the modified list is part of the environment and not the old one.
i thought what happend inside a function stays inside a function
meaning what comes out is independent of what comes in.
Meaning if I want the list I send as a parameter to the function I
have to do x = func(x) and not just func(x) and x is magically what
comes out of func().

A function cannot modify the value of a global variable

Yes it can.

>>a=[]
>>def f():

a.append('yes I can')

>>f()
>>a

['yes I can']

(unless it specifies "global"). It doesn't reassign anything.

The statement 'global a' would allow f to *rebind* the global *name*
'a'.

Aargh. That's exactly what I meant, sorry.

The word 'variable' should almost not be used in discussing Python
since it is often unclear whether it refers to a name (or collection
slot) or an object bound thereto.

Indeed. Which is exactly why I included those snippets of
code that you snipped, one of which does exactly what your
snippet above does... you're probably right, "variable" is
a bad idea, and "modify the value of a variable" is a very
bad idea.

The code doesn't "modify the value of the variable" in the
sense that the value of the variable is a certain object,
and after the function call the value is the same object.
It does "modify the value of the variable" in the sense
that the object which is the value of the variable has been
modified.

There's a problem here. Please note that this is not a criticism,
and I don't really know what anyone could do about the problem.
The problem is that if the reader is not accustomed to thinking
explicitly about what's going on under the hood when code is
executed he's going to have a hard time understanding the
difference between "assigning a value to a variable" and
"binding a name to an object". Once I realized that dicts
rule everything this became clear to me, but for some time
the discussions I saw on all this made no sense to me.

Which is why I think it's a good idea to include examples
illustrating what can and cannot be done, which is why I
did that. I tend to suspect that the OP is at least as
confused on the subtlties as I was back then (evidence below).

But in the functions below you're not reassigning a variable,
you're _modifiying_ an object. A function _can_ modify an
object you pass to it:

It can modify any mutable object it can access.

Doesnt Python have closure or that isnt what this is about?

Python does have closures. This is not about that.

This is why I suspect what I say I suspect. He's thought
that the word "closure" meant something like "local scope"...

def validate(placed):
student = round(random.random()*401)
if student in placed:
return validate(placed)
else:
placed.append(student)
return student, placed

Delete this. It is redundant with the below.

def val(placed):
student = round(random.random()*401)
if student in placed:
return validate(placed)
else:
placed.append(student)
return student

I believe this is equivalent to

def addval(placed):
while True:
student = round(random.random()*401)
if student not in placed:
break
placed.append(student)
return student

While this avoids the indefinite recursion depth problem, it does not
avoid the indefinite time problem. Use random.shuffle, or write your
own version if doing this for practice. Also consider removing the
return statement unless you actually directly use the added value. It
is easier to remember that addval mutates 'placed' without the return.

>>>g = lambda x:validate(x)

This is doubly diseased.

First, never write a 'name = lambda...' statement since it is equivalent
to a def statement except that the resulting function object lacks a
proper .funcname attribute. The above only trivially abbreviates
def g(x): return validate(x)
by 3 characters. Another reason is that the lambda form somehow more
often tricks people into the next mistake .

Second, never write a function (with either def or lambda) that simply
returns a function of the argument(s). The wrapping does nothing! This
is a waste of time and space for both you and the interpreter. The
above is functionally equivalent to
g = validate
and if you want that, you could name the function 'g' when you define it.

>>>l=[]

In some fonts, 'l' and '1' are nearly identical; please use something
else for public code, which you made this to be by posting it;-)

>>>for x in range(1,10):
g(l)

As said, the 'g' wrapper is useless.
addval(l)

Hope this helps.

Terry Jan Reedy

--
David C. Ullrich

[David C. Ullrich]

In article
<f4**********************************@t54g2000hsg. googlegroups.com>,
ssecorp <ci**********@gmail.comwrote:

>def mod(x,y):

return x.append(y)

>mod([1,2],3)
k=[1,2,3]
k

[1, 2, 3]

>l = mod(k,4)
l
k

[1, 2, 3, 4]

>l
k==l

False

>mod(k,5)
k

[1, 2, 3, 4, 5]

>mod(l,4)

Traceback (most recent call last):
File "<pyshell#29>", line 1, in <module>
mod(l,4)
File "<pyshell#18>", line 2, in mod
return x.append(y)
AttributeError: 'NoneType' object has no attribute 'append'

>l
l=k
l

[1, 2, 3, 4, 5]

>i=mod(k,1)
i

same stuff but i dont find this intuitive.

You need to read the docs. AList.append(x) does _not_
return AList with x appended. In fact it returns None,
because it wants to be a "procedure" that doesn't return
anything at all, but there is no such thing in Python;
functions and methods that do not explicitly contain
a "return" statement return None.

So when you say "return x.append(a)" you're saying
"return None", which explains the rest of it. You
noticed that the second line of

>l = mod(k,4)
l

didn't print anything? That's because the first line
set l to None. If you'd typed "print l" instead of just "l"
you would have seen

>>l = mod(k,4)
l
None

--
David C. Ullrich

[Steven D'Aprano]

On Thu, 10 Jul 2008 14:09:16 -0400, Terry Reedy wrote:

>>>>>g = lambda x:validate(x)

This is doubly diseased.

First, never write a 'name = lambda...' statement since it is equivalent
to a def statement except that the resulting function object lacks a
proper .funcname attribute.

Using lambda in this way is no more "diseased" than aliasing any other
object. It's a matter of personal preference not to bind a lambda to a
name. Functions, whether created by lambda or def, are first class
objects, and as such there's nothing wrong with binding them to names.

Admittedly, the lack of a func_name attribute can sometimes make
tracebacks harder to understand, especially if you've got many bound
lambdas. But that's no worse than this:

>>def factory(n):

.... def f():
.... return "spam"*n
.... return f
....

>>one_spam = factory(1)
two_spam = factory(2)
one_spam.func_name == two_spam.func_name

True

I'm sure that nobody would argue that using factory functions is
"diseased" because the functions have the same func_name attribute --
even if it does sometimes make it harder to interpret tracebacks:

>>one_spam()

'spam'

>>one_spam('arg')

Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: f() takes no arguments (1 given)
So by all means say that you personally don't like to bind lambdas to
names, but don't pretend that it is somehow wrong to do so.

Second, never write a function (with either def or lambda) that simply
returns a function of the argument(s).

I think you need to reconsider the way you say that. By definition,
*every* function simply returns a function of the arguments. That's what
functions do.

The wrapping does nothing! This
is a waste of time and space for both you and the interpreter.

When I was a newbie, I used to write:

def sin(x):
return math.sin(x)

That was before I learned to just do this:

sin = math.sin

But just to prove that you should never say never:

>>import timeit
T1 = timeit.Timer("f(1)", """def g(x):

.... return x+1
....
.... f = g
.... """)

>>T2 = timeit.Timer("f(1)", """def g(x):

.... return x+1
....
.... def f(x):
.... return g(x)
.... """)

>>>
T1.repeat()

[0.50848197937011719, 0.54573416709899902, 0.54767107963562012]

>>T2.repeat()

[0.88546991348266602, 0.89811587333679199, 0.95785093307495117]

So that's a good reason to wrap a function in another function -- to
measure the overhead of a function call.

*wink*
--
Steven

[Terry Reedy]

Steven D'Aprano wrote:

On Thu, 10 Jul 2008 14:09:16 -0400, Terry Reedy wrote:

>>>>>>g = lambda x:validate(x)

This is doubly diseased.

First, never write a 'name = lambda...' statement since it is equivalent
to a def statement except that the resulting function object lacks a
proper .funcname attribute.

Using lambda in this way is no more "diseased" than aliasing any other
object.

In the context of giving advice to a confused beginner, I disagree.
He must learn def statements. Lambda expressions are optional.

It's a matter of personal preference not to bind a lambda to a
name. Functions, whether created by lambda or def, are first class
objects, and as such there's nothing wrong with binding them to names.

When I brought this up on pydev, in the context of a style guide
addition, about 9 of 10 respondants agreed that this should be
discouraged. Alex Martelli reported his experience that this
construction more often leads people to the useless wrapping of function
calls, such as the OP posted, than the def statement equivalent does.

One of the major reasons people give for wanting lambda expressions kept
in Python and for using them is that they do not want to have to think
up a name for short expressions. If such a person then turns around and
binds the resulting function object to a name, then that rationale
disappears.

Over the years, people have written on c.l.p about 'lambdas' as if they
were a separate class of objects somehow different from def objects (and
not just an expression). I believe writing and reading both 'name =
lambda ...' and 'def name(...' engenders and reinforces this delusion,
especially for beginners.

Admittedly, the lack of a func_name attribute can sometimes make
tracebacks harder to understand, especially if you've got many bound
lambdas.

Is saving two keystrokes worth that disadvantage, and the confusions
mentioned above? To me, no. Hence my advice.

Terry Jan Reedy

[Steven D'Aprano]

On Sat, 12 Jul 2008 16:32:25 -0400, Terry Reedy wrote:

Steven D'Aprano wrote:

>On Thu, 10 Jul 2008 14:09:16 -0400, Terry Reedy wrote:

>>>>>>>g = lambda x:validate(x)
This is doubly diseased.

First, never write a 'name = lambda...' statement since it is
equivalent to a def statement except that the resulting function
object lacks a proper .funcname attribute.

Using lambda in this way is no more "diseased" than aliasing any other
object.

In the context of giving advice to a confused beginner, I disagree. He
must learn def statements. Lambda expressions are optional.

Lots of things are optional. sets and frozen sets and decorators and
classes and all sorts of things are optional, *and* capable of being
misused and/or abused. Would you describe properties as "diseased"
because some people might fill their classes with useless getters and
setters?

It's a matter of personal preference not to bind a lambda to a
name. Functions, whether created by lambda or def, are first class
objects, and as such there's nothing wrong with binding them to names.

When I brought this up on pydev, in the context of a style guide
addition, about 9 of 10 respondants agreed that this should be
discouraged. Alex Martelli reported his experience that this
construction more often leads people to the useless wrapping of function
calls, such as the OP posted, than the def statement equivalent does.

I have great respect for Alex, but anecdotes of that nature aren't proof.
Such anecdotal evidence is subject to confirmation bias. I'm sure that
Alex remembers the stupid pieces of code where people make such mistakes
but I doubt very much he remembers the cases where they don't.

But as a style guide... fine. Tell people it is discouraged. Tell them
that other Python developers will laugh at them if they do it, apart from
the ten percent who don't mind named lambdas. Point out that, like
factory functions, there's a debugging cost to having functions with a
unhelpful func_name attribute.

But don't cross that line from discouragement to telling people that it
is wrong to use named lambdas. That's what I object to.

One of the major reasons people give for wanting lambda expressions kept
in Python and for using them is that they do not want to have to think
up a name for short expressions. If such a person then turns around and
binds the resulting function object to a name, then that rationale
disappears.

That does not follow. The fact that I might choose to occasionally bind a
lambda to a name doesn't mean I never use unbound functions. Your
argument is like saying that the rationale for air conditioners ("cooling
in the hot weather") disappears because we can also use reverse-cycle air
conditioners for heating in the cold weather.

Lambdas are *necessary* if you want anonymous functions, just as we have
anonymous strings, anonymous integers and so forth. It would be a poor
language that forced people to write this:

x = 1
s = 'two'
def f(arg):
return arg+3

mylist = [x, s, f]

instead of the more sensible:

mylist = [1, 'two', lambda arg: arg+3]
That's why we need lambda. But once we have lambda, there is absolutely
no reason why we must prohibit other uses of it. Given the above
definition of mylist, would you then describe the following piece of code
as "diseased"?

def foo(mylist):
g = mylist[2]
for i in xrange(10000):
print g(i)

That's a perfectly good micro-optimization technique, and I would hope
you would not reject it merely because the function was anonymous. That
would be irrational.

I'm not defending the Original Poster's use of lambda in that specific
piece of code. You are right to criticize it *specifically*. What I
object to is that you generalize that criticism, and treat a personal
preference as an absolute rule. By all means think my code is ugly for
using named lambdas, and by all means tell me you think it is ugly, but
don't tell me I'm corrupting the youth of today with my filthy disease-
ridden code.
--
Steven