473,372 Members | 980 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,372 software developers and data experts.

A gotcha: Python pain point?

Consider this example:
>>def funcs(x):
... for i in range(5):
... def g(): return x + i
... yield g

I would expect the value of x used in g to be that at the function
declaration time, as if you've pass g a (x=x) argument, especially
after reading this post: http://lua-users.org/wiki/LuaScopingDiscussion

But:
>>[ fun() for fun in list(funcs(1)) ]
[5, 5, 5, 5, 5]

Whereas:
>>[ fun() for fun in funcs(1) ]
[1, 2, 3, 4, 5]

This came up while discussing Python pain points at
http://intertwingly.net/blog/2007/06...ts#c1181602242

I can see how it works now, but I haven't found an easy-to-read
documentation on this.

I guess it's debatable if perhaps every i used in the loop shouldn't
be a different i. It had me fooled, anyways.

Rgds,
Bjorn

Jun 11 '07 #1
7 1282
Beorn wrote:
Consider this example:
>>def funcs(x):
... for i in range(5):
... def g(): return x + i
... yield g

I would expect the value of x used in g to be that at the function
declaration time, as if you've pass g a (x=x) argument, especially
after reading this post: http://lua-users.org/wiki/LuaScopingDiscussion

But:
>>[ fun() for fun in list(funcs(1)) ]
[5, 5, 5, 5, 5]

Whereas:
>>[ fun() for fun in funcs(1) ]
[1, 2, 3, 4, 5]

This came up while discussing Python pain points at
http://intertwingly.net/blog/2007/06...ts#c1181602242

I can see how it works now, but I haven't found an easy-to-read
documentation on this.

I guess it's debatable if perhaps every i used in the loop shouldn't
be a different i. It had me fooled, anyways.

Rgds,
Bjorn
If this isn't classified as a bug, then someone has some serious
explaining to do. Why would it be desirable for a generator to behave
differently in two different contexts. Should I import this to see how
many principles this behavior violates?

James
Jun 11 '07 #2

"Beorn" <bj****@gmail.comwrote in message
news:11**********************@z28g2000prd.googlegr oups.com...
| Consider this example:
|
| >>def funcs(x):
| ... for i in range(5):
| ... def g(): return x + i
| ... yield g
|
| I would expect the value of x used in g to be that at the function
| declaration time, as if you've pass g a (x=x) argument,

Since x is constant during the funcs call, does not matter. Perhaps you
meant i, which does vary? If so, put i=i in the header -- or perhaps ii=i
to give two different names to two things which are made to have the same
value.

In any case, the simple rule is that default argument expressions in the
*header* are evaluated at definition time while the *body* (past the doc
string, if any) is executed after a call.

Some people expect defaults to be executed every call; others expect part
of the body to be executed once. Both get in trouble. For nested
functions, outer call time is inner definition time and this can confuses.
Defining (but delaying the call of) multiple identical inner functions, as
you do in the first example below, also confuses.

| especially
| after reading this post: http://lua-users.org/wiki/LuaScopingDiscussion

Lua is not Python.

| But:
|
| >>[ fun() for fun in list(funcs(1)) ]
| [5, 5, 5, 5, 5]
|
| Whereas:
|
| >>[ fun() for fun in funcs(1) ]
| [1, 2, 3, 4, 5]

As Calderone explained, the simple rule works as long as one keeps track of
what is called when.

Terry Jan Reedy

Jun 12 '07 #3
James Stroud wrote:
Beorn wrote:
>Consider this example:
> >>def funcs(x):
... for i in range(5):
... def g(): return x + i
... yield g

I would expect the value of x used in g to be that at the function
declaration time, as if you've pass g a (x=x) argument, especially
after reading this post: http://lua-users.org/wiki/LuaScopingDiscussion

But:
> >>[ fun() for fun in list(funcs(1)) ]
[5, 5, 5, 5, 5]

Whereas:
> >>[ fun() for fun in funcs(1) ]
[1, 2, 3, 4, 5]

This came up while discussing Python pain points at
http://intertwingly.net/blog/2007/06...ts#c1181602242

I can see how it works now, but I haven't found an easy-to-read
documentation on this.

I guess it's debatable if perhaps every i used in the loop shouldn't
be a different i. It had me fooled, anyways.

Rgds,
Bjorn

If this isn't classified as a bug, then someone has some serious
explaining to do. Why would it be desirable for a generator to behave
differently in two different contexts. Should I import this to see how
many principles this behavior violates?
It's no bug. The generator behaves the same way in both cases. What is
different is the calling time of the function.

Certainly a surprising outcome. But no bug.

Diez
Jun 12 '07 #4

"James Stroud" <js*****@mbi.ucla.eduwrote in message
news:f4**********@zinnia.noc.ucla.edu...
| Beorn wrote:
| Consider this example:
| >
| >>def funcs(x):
| ... for i in range(5):
| ... def g(): return x + i
| ... yield g
| >
| >>[ fun() for fun in list(funcs(1)) ]
| [5, 5, 5, 5, 5]
| >
| Whereas:
| >
| >>[ fun() for fun in funcs(1) ]
| [1, 2, 3, 4, 5]

| If this isn't classified as a bug,

It is not, it is well-documented behavior. Still, suggestions for
improvement might be considered.

| Why would it be desirable for a generator to behave
| differently in two different contexts.

I have no idea.

Each call of the generator function funcs behaves the same. It returns a
generator that yields 5 identical copies of the inner function g. The
multiple copies are not needed and only serve to confuse the issue.
Changing funcs to return a generator that yields the *same* function (five
times) gives the same behavior.

def funcs(x):
def g(): return x + i
for i in range(5):
yield g

print [ fun() for fun in list(funcs(1)) ]
print [ fun() for fun in funcs(1) ]
>># when run
[5, 5, 5, 5, 5]
[1, 2, 3, 4, 5]

What matters is the value of g's nonlocal var i (funcs' local var i) when
the yielded function g is *called*.

The difference between returning versus yielding an inner closure such as g
is this. If g is returned, the outer function has terminated and the
enclosed variable(s), i in this case, is frozen at its final value. If g
is yielded, the enclosed i is *live* as long as the generator is, and its
values can change between calls, as in the second print statement.

| Should I import this to see how
| many principles this behavior violates?

???

If you meant 'report' (on SF), please do not.

Terry Jan Reedy

Jun 12 '07 #5
Beorn wrote:
Consider this example:
>>def funcs(x):
... for i in range(5):
... def g(): return x + i
... yield g

I would expect the value of x used in g to be that at the function
You mean i here, don't you?
declaration time, as if you've pass g a (x=x) argument, especially
after reading this post: http://lua-users.org/wiki/LuaScopingDiscussion

But:
>>[ fun() for fun in list(funcs(1)) ]
[5, 5, 5, 5, 5]

Whereas:
>>[ fun() for fun in funcs(1) ]
[1, 2, 3, 4, 5]

This came up while discussing Python pain points at
http://intertwingly.net/blog/2007/06...ts#c1181602242

I can see how it works now, but I haven't found an easy-to-read
documentation on this.
This has been discussed here very often. Python closures do capture the
names, not the values. If you want a value at a certain point, you need to
bind in the generated function scope by passing it as parameter. Like this:

def funcs(x):
for i in xrange(5):
def g(i=i): return x + i
yield g

Diez
Jun 12 '07 #6
In <ma***************************************@python. org>, Terry Reedy
wrote:
| Should I import this to see how
| many principles this behavior violates?

???

If you meant 'report' (on SF), please do not.
I think he meant ``import this`` at the Python interpreter.

Ciao,
Marc 'BlackJack' Rintsch
Jun 12 '07 #7
On Jun 12, 5:00 pm, "Diez B. Roggisch" <d...@nospam.web.dewrote:
Beorn wrote:
[...]
I can see how it works now, but I haven't found an easy-to-read
documentation on this.

This has been discussed here very often. Python closures do capture the
names, not the values. If you want a value at a certain point, you need to
bind in the generated function scope by passing it as parameter. Like this:

def funcs(x):
for i in xrange(5):
def g(i=i): return x + i
yield g
Well, I would hope to see this documented somewhere prominently;
looking at the PEPs didn't give me much of a clue. I'm not even sure
how to describe the problem, so formulating search queries becomes
hard(!).

How does other languages, like Ruby, work? Sam (see original link
about pain points) was very surprised that the for loop didn't
introduce a new scope (is that the right word?) for each iteration?

Rgds,
Bjorn

Jun 13 '07 #8

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

220
by: Brandon J. Van Every | last post by:
What's better about Ruby than Python? I'm sure there's something. What is it? This is not a troll. I'm language shopping and I want people's answers. I don't know beans about Ruby or have...
8
by: eScrewDotCom | last post by:
eScrew Welcome to eScrew! eScrew is eScrew and this is eScrew story. eScrew will tell you eScrew story if you promise eScrew to consider eScrew story as joke. eScrew story is very funny. eScrew...
112
by: mystilleef | last post by:
Hello, What is the Pythonic way of implementing getters and setters. I've heard people say the use of accessors is not Pythonic. But why? And what is the alternative? I refrain from using them...
1
by: CloudSolutions | last post by:
Introduction: For many beginners and individual users, requiring a credit card and email registration may pose a barrier when starting to use cloud servers. However, some cloud server providers now...
0
by: Faith0G | last post by:
I am starting a new it consulting business and it's been a while since I setup a new website. Is wordpress still the best web based software for hosting a 5 page website? The webpages will be...
0
isladogs
by: isladogs | last post by:
The next Access Europe User Group meeting will be on Wednesday 3 Apr 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM). In this session, we are pleased to welcome former...
0
by: taylorcarr | last post by:
A Canon printer is a smart device known for being advanced, efficient, and reliable. It is designed for home, office, and hybrid workspace use and can also be used for a variety of purposes. However,...
0
by: aa123db | last post by:
Variable and constants Use var or let for variables and const fror constants. Var foo ='bar'; Let foo ='bar';const baz ='bar'; Functions function $name$ ($parameters$) { } ...
0
by: ryjfgjl | last post by:
In our work, we often receive Excel tables with data in the same format. If we want to analyze these data, it can be difficult to analyze them because the data is spread across multiple Excel files...
0
BarryA
by: BarryA | last post by:
What are the essential steps and strategies outlined in the Data Structures and Algorithms (DSA) roadmap for aspiring data scientists? How can individuals effectively utilize this roadmap to progress...
1
by: nemocccc | last post by:
hello, everyone, I want to develop a software for my android phone for daily needs, any suggestions?
0
by: Hystou | last post by:
There are some requirements for setting up RAID: 1. The motherboard and BIOS support RAID configuration. 2. The motherboard has 2 or more available SATA protocol SSD/HDD slots (including MSATA, M.2...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.