471,354 Members | 1,696 Online
Bytes | Software Development & Data Engineering Community
Post +

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 471,354 software developers and data experts.

Confused about closures and scoping rules

Hi all,

consider the following small example:

"""
Small test to try to understand a strange subtlety with closures
"""

def outer(nmax):

aa = []
for n in range(nmax):

def a(y):
return (y,n)
print 'Closure and cell id:',id(a.func_closure),\
id(a.func_closure[0])
aa.append(a)

return aa

print 'Closure creation.'
nmax = 3
aa = outer(nmax)

print
print 'Closure use.'
for n in range(nmax):
print '%s:%s' % (n,aa[n]('hello'))

################## EOF #################
If I run this, I get:

planck[test]python debug_closures.py
Closure creation.
Closure and cell id: 1075998828 1075618940
Closure and cell id: 1075999052 1075618940
Closure and cell id: 1075999084 1075618940

Closure use.
0:('hello', 2)
1:('hello', 2)
2:('hello', 2)
My confusion arises from the printout after 'closure use'. I was expecting that
each new function 'a' created inside the loop in 'outer' would capture the
value of n, therefore my expectation was to see a printout like:

0:('hello', 0)
1:('hello', 1)... etc.

However, what happens is a bit different. As can be seen from the printouts
of 'Closure and cell id', in each pass of the loop a new closure is created,
but it reuses the *same* cell object every time. For this reason, all the
closures end up sharing the scope with the values determined by the *last*
iteration of the loop.

This struck me as counterintuitive, but I couldn't find anything in the
official docs indicating what the expected behavior should be. Any
feedback/enlightenment would be welcome. This problem appeared deep inside a
complicated code and it took me almost two days to track down what was going
on...

Cheers,

f

Nov 7 '07 #1
2 1172
Fernando Perez schrieb:
Hi all,

consider the following small example:

"""
Small test to try to understand a strange subtlety with closures
"""

def outer(nmax):

aa = []
for n in range(nmax):

def a(y):
return (y,n)
print 'Closure and cell id:',id(a.func_closure),\
id(a.func_closure[0])
aa.append(a)

return aa

print 'Closure creation.'
nmax = 3
aa = outer(nmax)

print
print 'Closure use.'
for n in range(nmax):
print '%s:%s' % (n,aa[n]('hello'))

################## EOF #################
If I run this, I get:

planck[test]python debug_closures.py
Closure creation.
Closure and cell id: 1075998828 1075618940
Closure and cell id: 1075999052 1075618940
Closure and cell id: 1075999084 1075618940

Closure use.
0:('hello', 2)
1:('hello', 2)
2:('hello', 2)
My confusion arises from the printout after 'closure use'. I was expecting that
each new function 'a' created inside the loop in 'outer' would capture the
value of n, therefore my expectation was to see a printout like:

0:('hello', 0)
1:('hello', 1)... etc.

However, what happens is a bit different. As can be seen from the printouts
of 'Closure and cell id', in each pass of the loop a new closure is created,
but it reuses the *same* cell object every time. For this reason, all the
closures end up sharing the scope with the values determined by the *last*
iteration of the loop.

This struck me as counterintuitive, but I couldn't find anything in the
official docs indicating what the expected behavior should be. Any
feedback/enlightenment would be welcome. This problem appeared deep inside a
complicated code and it took me almost two days to track down what was going
on...
It's a FAQ. The reason is that the created closures don't capture the
_value_, but the _name_. Plus of course the locals()-dictionary outside
the function a to perform the lookup of that name. Which has the value
bound to it in the last iteration.

Common cure for this is to create an a-local name that shadows the outer
variable and is simultaneously bound to the desired value:

def outer(nmax):

aa = []
for n in range(nmax):
foo = 'bar'
def a(y,n=n):
bar = foo
return (y,n)
print 'Closure and cell id:',id(a.func_closure),\
id(a.func_closure[0])
aa.append(a)

return aa

print 'Closure creation.'
nmax = 3
aa = outer(nmax)

print
print 'Closure use.'
for n in range(nmax):
print '%s:%s' % (n,aa[n]('hello'))
Notice the foo/bar - that was necessary to actually create a closure at
all (to keep your printing working), as python statically checks if
there needs one to be.

Diez
Nov 7 '07 #2
Diez B. Roggisch wrote:

It's a FAQ. The reason is that the created closures don't capture the
_value_, but the _name_. Plus of course the locals()-dictionary outside
the function a to perform the lookup of that name. Which has the value
bound to it in the last iteration.

Common cure for this is to create an a-local name that shadows the outer
variable and is simultaneously bound to the desired value:
Many thanks (also to JP) for the clear explanation. Greatly appreciated.

Cheers,

f

Nov 7 '07 #3

This discussion thread is closed

Replies have been disabled for this discussion.

Similar topics

14 posts views Thread by Alexander May | last post: by
7 posts views Thread by Charles Krug | last post: by
9 posts views Thread by NevilleDNZ | last post: by
17 posts views Thread by Chad | last post: by
6 posts views Thread by stef mientki | last post: by
14 posts views Thread by Khookie | last post: by
2 posts views Thread by Joshua Kugler | last post: by
1 post views Thread by Jeff | last post: by
26 posts views Thread by Aaron \Castironpi\ Brady | last post: by
reply views Thread by XIAOLAOHU | last post: by

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.