455,587 Members | 1,677 Online
Need help? Post your question and get tips & solutions from a community of 455,587 IT Pros & Developers. It's quick & easy.

# Help me dig my way out of nested scoping

 P: n/a Hi everyone I'm new to Python, so forgive me if the solution to my question should have been obvious. I have a function, call it F(x), which asks for two other functions as arguments, say A(x) and B(x). A and B are most efficiently evaluated at once, since they share much of the same math, ie, A, B = AB(x), but F wants to call them independantly (it's part of a third party library, so I can't change this behaviour easily). My solution is to define a wrapper function FW(x), with two nested functions, AW(x) and BW(x), which only call AB(x) if x has changed. To make this all clear, here is my (failed) attempt: #------begin code --------- from ThirdPartyLibrary import F from MyOtherModule import AB def FW(x): lastX = None aLastX = None bLastX = None def AW(x): if x != lastX: lastX = x # ^ Here's the problem. this doesn't actually # change FW's lastX, but creates a new, local lastX aLastX, bLastX = AB(x) return aLastX def BW(x): if x != lastX: lastX = x # ^ Same problem aLastX, bLastX = AB(x) return bLastX #finally, call the third party function and return its result return F(AW, BW) #-------- end code --------- OK, here's my problem: How do I best store and change lastX, A(lastX) and B(lastX) in FW's scope? This seems like it should be easy, but I'm stuck. Any help would be appreciated! -Brendan -- Brendan Simons Jul 18 '05 #1
10 Replies

 P: n/a Brendan wrote: Hi everyone I'm new to Python, so forgive me if the solution to my question should have been obvious. .... Good question. For a thorough explanation see: http://www.python.org/dev/doc/devel/ref/naming.html Simple version follows: OK, here's my problem: How do I best store and change lastX, A(lastX) and B(lastX) in FW's scope? This seems like it should be easy, but I'm stuck. Any help would be appreciated! Assignments (i.e., binding names to objects) are always made in the local scope (unless you've used the 'global' declaration, which I don't think can help you here). So, for an even simpler demonstration of the problem see: def outer(): ... b = 1 ... def inner(): ... b += 1 ... print b ... inner() ... outer() Traceback (most recent call last): File "", line 1, in ? File "", line 6, in outer File "", line 4, in inner UnboundLocalError: local variable 'b' referenced before assignment The solution is not to re-bind the identifier from the enclosing scope, but rather to mutate the object that it references. This requires a mutable object, such as a list: def outer(): ... b = [1] # bind b to a mutable object ... def inner(): ... b[0] += 1 ... print b[0] ... inner() ... outer() 2 HTH Michael Jul 18 '05 #2

 P: n/a I wish I had time to dig into your specific problem because it looks interesting. But I think you might want to look at "python generators". I beleive there is no reason that they can't yield a function. http://www.python.org/peps/pep-0255.html http://docs.python.org/ref/yield.html http://linuxgazette.net/100/pramode.html James On Sunday 03 April 2005 02:12 pm, Brendan wrote: Hi everyone I'm new to Python, so forgive me if the solution to my question should have been obvious. I have a function, call it F(x), which asks for two other functions as arguments, say A(x) and B(x). A and B are most efficiently evaluated at once, since they share much of the same math, ie, A, B = AB(x), but F wants to call them independantly (it's part of a third party library, so I can't change this behaviour easily). My solution is to define a wrapper function FW(x), with two nested functions, AW(x) and BW(x), which only call AB(x) if x has changed. To make this all clear, here is my (failed) attempt: #------begin code --------- from ThirdPartyLibrary import F from MyOtherModule import AB def FW(x): lastX = None aLastX = None bLastX = None def AW(x): if x != lastX: lastX = x # ^ Here's the problem. this doesn't actually # change FW's lastX, but creates a new, local lastX aLastX, bLastX = AB(x) return aLastX def BW(x): if x != lastX: lastX = x # ^ Same problem aLastX, bLastX = AB(x) return bLastX #finally, call the third party function and return its result return F(AW, BW) #-------- end code --------- OK, here's my problem: How do I best store and change lastX, A(lastX) and B(lastX) in FW's scope? This seems like it should be easy, but I'm stuck. Any help would be appreciated! -Brendan -- Brendan Simons -- James Stroud, Ph.D. UCLA-DOE Institute for Genomics and Proteomics Box 951570 Los Angeles, CA 90095 http://www.jamesstroud.com/ Jul 18 '05 #3

 P: n/a On Sunday 03 April 2005 04:12 pm, Brendan wrote: from ThirdPartyLibrary import F from MyOtherModule import AB def FW(x): lastX = None aLastX = None bLastX = None I'm pretty sure your method will work if you just specify that these are global: def FW(x): global lastX = None global aLastX = None global bLastX = None OTOH, I'm biased against using module-level variables for this kind of purpose and I think things that retain state really ought to be class instances, so I'd probably replace AB(x) with a callable object, and define two wrappers to access it (untested): class AB: def __init__(self): self._last_x = None self._last_a = None self._last_b = None def __call__(self, x): if x == self._last_x: return self._last_a, self._last_b else: self._last_a, self._last_b = self.AB(x) return self._last_a, self._last_b def A(self, x): return self(x)[0] def B(self, x): return self(x)[1] def AB(self, x): """ This is where you compute your new values when needed. """ # something that computes a and b return a,b ab = AB() Then you actually pass the methods ab.A and ab.B to your library routine. This will usually work, though if it somehow insists on an actual function instead of a callable, you can always use wrapper functions. This also has the advantage that you *can* process more than one case at a time (i.e. if you have two different places where you need this function to be called and you aren't sure what order they'll be processed (or don't want to think about it), you can give them different instances of AB to work with, and they'll remember their previous calls separately. Cheers, Terry -- -- Terry Hancock ( hancock at anansispaceworks.com ) Anansi Spaceworks http://www.anansispaceworks.com Jul 18 '05 #4

 P: n/a On 3 Apr 2005 14:12:48 -0700, "Brendan" wrote: Hi everyoneI'm new to Python, so forgive me if the solution to my question shouldhave been obvious. I have a function, call it F(x), which asks for twoother functions as arguments, say A(x) and B(x). A and B are mostefficiently evaluated at once, since they share much of the same math,ie, A, B = AB(x), but F wants to call them independantly (it's part ofa third party library, so I can't change this behaviour easily). Mysolution is to define a wrapper function FW(x), with two nestedfunctions, AW(x) and BW(x), which only call AB(x) if x has changed. You have several easy choices, that would not require you modifying your program much. 1. Use the 'global' keyword to declare lastX, aLastX, and bLastX as globals, then all functions will have access to them. def FW(x): global lastX, aLastX, bLastX 2. Use function attributes, which are just names attached to the function using a '.'. def FW(x): # # Function body here # return F(AW, BW) FW.lastX = None FW.aLastX = None FW.bLastX = None result = FW(x) You will need to always include the FW. in front of those names. 3. Something else, that may help is you can return more than one value at a time. Python has this neat feature that you can have multiple items on either side of the '=' sign. a,b,c = 1,2,3 same as: a=1 b=2 c=3 And it also works with return statements so you can return multiple value. def abc(n): return n+1, n+2, n+3 a,b,c = abc(0) 5. Choice 5 and above is to rewrite your function as a class. Names in class's retain their values between calls and you can access those values the same way as accessing function attributes. Hope this helped. Cheers, Ron To make this all clear, here is my (failed) attempt:#------begin code ---------from ThirdPartyLibrary import Ffrom MyOtherModule import ABdef FW(x): lastX = None aLastX = None bLastX = None def AW(x): if x != lastX: lastX = x # ^ Here's the problem. this doesn't actually # change FW's lastX, but creates a new, local lastX aLastX, bLastX = AB(x) return aLastX def BW(x): if x != lastX: lastX = x # ^ Same problem aLastX, bLastX = AB(x) return bLastX #finally, call the third party function and return its result return F(AW, BW)#-------- end code ---------OK, here's my problem: How do I best store and change lastX, A(lastX)and B(lastX) in FW's scope? This seems like it should be easy, but I'mstuck. Any help would be appreciated! -Brendan Jul 18 '05 #5

 P: n/a "Brendan" wrote in message news:11**********************@l41g2000cwc.googlegr oups.com... I have a function, call it F(x), which asks for two other functions as arguments, say A(x) and B(x). ... If I understand this and the rest, a third party library whose code you cannot modify (easily) has a function F with (at least) three parameters: A, B, and x. During its operation, F calls A(x) and B(x). Because of code commonality for the particular A and B arg funcs you want to feed to F, you want avoid duplication by having the first call to either to calculate both return values. If F calls each of A and B exactly once and always in the same order and only for the value x you supply, the solution is pretty easy. A calls AB, stashes the B value away where it can be retrieved, and return the A value. B retrieves the B value and returns it. But your problem is the stash and retrieve part. Solutions: 1. global variable (easiest) - use global declaration in A; 2. closure variable - use mutable such as 1 element list (see below); 3. instance attribute - with A and B as methods. 2 is what you tried to do, but without knowing the mutable (list or dict) trick: def ABwrapper(): bsave = [None] def A(x): aval,bval = AB(x) bsave[0] = bval return aval def B(x): return bsave[0] return A,B This works because A does not try to *rebind* bsave to a new object. It only mutates the existing object. If the order of calling changes, you need more logic. If F calls A and B on multiple 'x's, as with, for instance, a derivative approximizer, then I would memoize A and/or B using the recipe posted here more than once and on the cookbook site and included in the new Python Cookbook v2 (and maybe v1, don't have it). Terry J. Reedy Jul 18 '05 #6

 P: n/a Thanks for the tips. Making FW a callable class (choice 5) seems to be a good (if verbose) solution. I might just wrap my temporary values in a list [lastX, lastA, lastB] and mutate them as Michael suggests. Thanks to Michael especially for the explanation of the name-binding process that's at the heart of the issue. The other choicess are not as helpful to me for the following reasons: choice 1: I don't want the temporary values of lastA and lastB to be global variables in my case as they are great big numeric arrays, and I'd like their memory to be reclaimed after FW is done. choice 2: I tried this without success. Using Micheal's example, I would assume you mean something like this: def outer(): b = 1 def inner(): outer.b += 1 print outer.b inner() outer() Which gives me: AttributeError: 'function' object has no attribute 'b' Perhaps I misapplied this method? choice 3: I know that Python can return multiple values in one line, but I don't think that applies here. My library function F, is looking for two separate function arguments Jul 18 '05 #7

 P: n/a F -is- in fact an iterative optimizer that minimizes A on x (B is the derivative of A). So yes, F will call A and B on mulitple 'x's. In that case, it seems the mutable object trick is the way to go. Thanks. I didn't follow your last sentence. What about the Python Cookbook? Jul 18 '05 #8

 P: n/a James Stroud Apr 3, 3:18 pm:I think you might want to look at "python generators". I've seen discussion of generators before, but haven't invested the time to understand them yet. This might be a good excuse. Jul 18 '05 #9

 P: n/a On 3 Apr 2005 16:21:10 -0700, "Brendan" wrote: Thanks for the tips. Making FW a callable class (choice 5) seems to bea good (if verbose) solution. I might just wrap my temporary values ina list [lastX, lastA, lastB] and mutate them as Michael suggests.Thanks to Michael especially for the explanation of the name-bindingprocess that's at the heart of the issue.The other choicess are not as helpful to me for the following reasons:choice 1: I don't want the temporary values of lastA and lastB to beglobal variables in my case as they are great big numeric arrays, andI'd like their memory to be reclaimed after FW is done. Generally global variables should be avoided in python if you are doing a large application. For smaller ones, they are ok, but they are just a little slower than local variables. You could use a classic class which is a good way to store a single group of data. The 'del' will unbind a name from an object so the objects can be garbage collected. class data: A = [] B = [] def countupdown(): for n in xrange(11): data.A.append(n) data.B.append(10-n) print data.A print data.B countupdown() # store data # Check out pickle module for this. del data choice 2: I tried this without success. Using Micheal's example, Iwould assume you mean something like this: def outer(): def inner(): outer.b += 1 print outer.b inner() outer.b = 1 # <-- initialize here after function of same name outer() # save data method here del outer # delete outer and it's attributes Jul 18 '05 #10

 P: n/a "Brendan" wrote in message news:11**********************@f14g2000cwb.googlegr oups.com...F -is- in fact an iterative optimizer that minimizes A on x (B is the derivative of A). So yes, F will call A and B on mulitple 'x's. In that case, it seems the mutable object trick is the way to go. Thanks. As long as it calls A and B in that same order, rather than A several times and B several times, then my code, fleshed out, will probably work. I didn't follow your last sentence. What about the Python Cookbook? It is a book that is one place to find memoizer code which allows one to save several x,f(x) pairs at once for later reuse. But you don't seem to need it since an optimizer should never return to the exact same x except by accident. Terry J. Reedy Jul 18 '05 #11

### This discussion thread is closed

Replies have been disabled for this discussion.