473,685 Members | 3,197 Online
Bytes | Software Development & Data Engineering Community
+ Post

Home Posts Topics Members FAQ

Dynamically defined functions via exec in imported module

Hello all,
I'm trying to write a function that will dynamically generate other
functions via exec. I then want to be able to import the file (module)
containing this function and use it in other modules, but for some
reason it only works using the "import <mod>" syntax, and not "from
<modimport *" syntax... i.e. in the latter case, the function is
dynamically generated, but not accessible from the importing module.
Any ideas on what I can do to be able to retain the second form of
import and still have the exec'd functions visible?

Here's the code... I have three files:

############### ############### #####
# modA.py

def dynamicdef(name , amt):
'''Dynamically defines a new function with the given name that
adds
the given amt to its argument and returns the result.'''
stm = "def %s(x):\n\tretur n x + %d" % (name, amt)
print stm
# exec stm # --- with this, 'name' is only accessible within
this fn
exec stm in globals() # --- this makes it global within this
module...

print eval(name)
dynamicdef('plu s5', 5)

print plus5(7)
############### ############### #####
# modB.py
# This uses the dynamicdef to dynamically define a new function, and
it
# works fine, with the newly defined function being accessible thru
the modA
# module...

import modA

modA.dynamicdef ('plus10', 10)

print help(modA.plus5 )
print help(modA.plus1 0)

print modA.plus5(20)
print modA.plus10(20)

############### ############### #####
# modC.py
# This uses the dynamicdef to dynamically define a new function, but
it
# doesn't work; seems like knowledge of the newly defined function in
the
# modA module is not propagated back to this context or something...?

from modA import *

dynamicdef('plu s10', 10)

print help(plus5)
#print help(plus10) # !!! Fails: NameError: name 'plus10' is not
defined

print plus5(20)
print plus10(20)
############### ############### #####
Thanks for any help/suggestions,
---nadeem

Aug 15 '08 #1
11 2568
On Aug 15, 7:26*pm, Nadeem <nadeemabdulha. ..@gmail.comwro te:
Hello all,
I'm trying to write a function that will dynamically generate other
functions via exec.
General tip: whenever you think you need to use exec (or eval), 99% of
the time you don't; most of the time there is a better (meaning, less
fragile and obscure) solution.
>I then want to be able to import the file (module)
containing this function and use it in other modules, but for some
reason it only works using the "import <mod>" syntax, and not "from
<modimport *" syntax... i.e. in the latter case, the function is
dynamically generated, but not accessible from the importing module.
Any ideas on what I can do to be able to retain the second form of
import and still have the exec'd functions visible?

Here's the code... I have three files:

############### ############### #####
# modA.py

def dynamicdef(name , amt):
* * '''Dynamically defines a new function with the given name that
adds
* * the given amt to its argument and returns the result.'''
* * stm = "def %s(x):\n\tretur n x + %d" % (name, amt)
* * print stm
* * # exec stm * * *# --- with this, 'name' is only accessible within
this fn
* * exec stm in globals() * # --- this makes it global within this
module...

* * print eval(name)

dynamicdef('plu s5', 5)

print plus5(7)
Unsurprisingly, there is indeed a better way, a closure:

def adder(amt):
def closure(x):
return x + amt
return closure
>>plus5 = adder(5)
plus5(7)
12

HTH,
George
Aug 16 '08 #2
I understand the 99% rule... the example I gave was to simplify the
issue. The full thing I'm working on is a library for an introductory
CS class I'm teaching. I'm trying, essentially, to build a library of
macros for students to use allowing them to define records (like
structs in C) with selector functions. In particular, I'm trying to
replicate some of the Scheme stuff from the HtDP project in Python
(http://www.htdp.org/2003-09-26/Book/curriculum-Z-
H-9.html#node_sec _6.3). I want to provide a function, called
defineStruct that is called like this:

defineStruct('p os', 'x', 'y')

The effect of this function will be to dynamically define several new
functions for working with structures:

makePos(x, y)
posX(p)
posY(p)
isPos(p)

I understand that all this can be done with classes and OO
programming, but the whole point of the HtDP curriculum is to
introduce students to programming in a pedagogically-effective way
using a functional approach instead of OO-first. They do it in Scheme,
which is primarily a f.p. language, and I'm trying to replicate a
similar approach in Python. The defineStruct thing is basically meant
to be a macro that introduces a set of functions for whatever
structure definition is needed.

So, for these reasons, I don't believe the closure example above is
helpful. I don't want to have to tell students anything about
closures, and certainly have them worrying about functions returning
functions, and function pointers, etc. I'm trying to bundle all that
up behind the scenes.

So, thinking about my problem again, an alternate question may be: Is
it possible, in a function called in a module, to access and update
the global definitions (dictionary or whatever) in the caller module.

--- nadeem
Aug 16 '08 #3
Well, I found one hack that seems to achieve this by accessing the
globals() dictionary of the outermost stack frame and adding an entry
to it for the newly created functions:

import inspect

def dynamicdef(name , amt):
'''Dynamically defines a new function with the given name that
adds
the given amt to its argument and returns the result.'''
stm = "def %s(x):\n\tretur n x + %d" % (name, amt)
print stm
exec stm in globals()

## ADDED THIS TO EXPORT THE NEW FUNCTION NAME TO THE TOP LEVEL...
inspect.stack()[-1][0].f_globals[name] = eval(name)
I guess I'll go with this unless someone suggests an alternate. Thanks
anyway, :)
--- nadeem
Aug 16 '08 #4
On Fri, Aug 15, 2008 at 10:48 PM, Nadeem <na************ **@gmail.comwro te:
I understand the 99% rule... the example I gave was to simplify the
issue. The full thing I'm working on is a library for an introductory
CS class I'm teaching. I'm trying, essentially, to build a library of
macros for students to use allowing them to define records (like
structs in C) with selector functions. In particular, I'm trying to
replicate some of the Scheme stuff from the HtDP project in Python
(http://www.htdp.org/2003-09-26/Book/curriculum-Z-
H-9.html#node_sec _6.3). I want to provide a function, called
defineStruct that is called like this:

defineStruct('p os', 'x', 'y')

The effect of this function will be to dynamically define several new
functions for working with structures:

makePos(x, y)
posX(p)
posY(p)
isPos(p)
def defineStruct(na me, *parameters):
class _struct:
def __init__(self, *init_parameter s):
for pname, pvalue in zip(parameters, init_parameters ):
setattr(self, pname, pvalue)
globals()["make" + name] = _struct
for parameter in parameters:
def getter(o, parameter=param eter):
return getattr(o, parameter)
globals()[name + parameter] = getter
globals()["is" + name] = lambda o: isinstance(o, _struct)

You might do other things, of course, like stepping up the frames from
sys._getframe() to inject the functions into the callers global scope.
There are some obvious optimizations you could make, too, as well as
other "methods" you might want to add.
I understand that all this can be done with classes and OO
programming, but the whole point of the HtDP curriculum is to
introduce students to programming in a pedagogically-effective way
using a functional approach instead of OO-first. They do it in Scheme,
which is primarily a f.p. language, and I'm trying to replicate a
similar approach in Python. The defineStruct thing is basically meant
to be a macro that introduces a set of functions for whatever
structure definition is needed.

So, for these reasons, I don't believe the closure example above is
helpful. I don't want to have to tell students anything about
closures, and certainly have them worrying about functions returning
functions, and function pointers, etc. I'm trying to bundle all that
up behind the scenes.

So, thinking about my problem again, an alternate question may be: Is
it possible, in a function called in a module, to access and update
the global definitions (dictionary or whatever) in the caller module.

--- nadeem
--
http://mail.python.org/mailman/listinfo/python-list


--
Read my blog! I depend on your acceptance of my opinion! I am interesting!
http://techblog.ironfroggy.com/
Follow me if you're into that sort of thing: http://www.twitter.com/ironfroggy
Aug 16 '08 #5
That's a really neat way of doing it, thanks a lot! I hadn't realized
how accessible all those globals() dictionaries were. Guess my example
still falls in the 99%... :)
--- nadeem

>
def defineStruct(na me, *parameters):
* class _struct:
* * def __init__(self, *init_parameter s):
* * * for pname, pvalue in zip(parameters, init_parameters ):
* * * * setattr(self, pname, pvalue)
* globals()["make" + name] = _struct
* for parameter in parameters:
* * def getter(o, parameter=param eter):
* * * return getattr(o, parameter)
* * globals()[name + parameter] = getter
* globals()["is" + name] = lambda o: isinstance(o, _struct)

You might do other things, of course, like stepping up the frames from
sys._getframe() to inject the functions into the callers global scope.
There are some obvious optimizations you could make, too, as well as
other "methods" you might want to add.
I understand that all this can be done with classes and OO
programming, but the whole point of the HtDP curriculum is to
introduce students to programming in a pedagogically-effective way
using a functional approach instead of OO-first. They do it in Scheme,
which is primarily a f.p. language, and I'm trying to replicate a
similar approach in Python. The defineStruct thing is basically meant
to be a macro that introduces a set of functions for whatever
structure definition is needed.
So, for these reasons, I don't believe the closure example above is
helpful. I don't want to have to tell students anything about
closures, and certainly have them worrying about functions returning
functions, and function pointers, etc. I'm trying to bundle all that
up behind the scenes.
So, thinking about my problem again, an alternate question may be: Is
it possible, in a function called in a module, to access and update
the global definitions (dictionary or whatever) in the caller module.
--- nadeem
--
http://mail.python.org/mailman/listinfo/python-list

--
Read my blog! I depend on your acceptance of my opinion! I am interesting!http://techblog.ironfroggy.com/
Follow me if you're into that sort of thing:http://www.twitter.com/ironfroggy
Aug 16 '08 #6
On Aug 16, 4:48*am, Nadeem <nadeemabdulha. ..@gmail.comwro te:
I understand the 99% rule... the example I gave was to simplify the
issue. The full thing I'm working on is a library for an introductory
CS class I'm teaching. I'm trying, essentially, to build a library of
macros for students to use allowing them to define records (like
structs in C) with selector functions.
The namedtuple recipe by Raymond Hetting does
exactly that and, guess what, it uses exec!
Also the doctest module in the standard library
uses exec at good effect. So, I would say it is
not a 99% rule, let's say it is a 98% rule ;)
BTW, I also write a paper on namedtuple which I
am going to translate in English soon or later
(for the moment there is Google Translator):
http://stacktrace.it/articoli/2008/0...cord-python-1/
The paper, just as the namedtuple recipe,
has a strong functional bias so it may be of interest
to people coming from Scheme.

Michele Simionato
Aug 16 '08 #7
On Aug 16, 12:50*am, Michele Simionato <michele.simion ...@gmail.com>
wrote:
On Aug 16, 4:48*am, Nadeem <nadeemabdulha. ..@gmail.comwro te:
I understand the 99% rule... the example I gave was to simplify the
issue. The full thing I'm working on is a library for an introductory
CS class I'm teaching. I'm trying, essentially, to build a library of
macros for students to use allowing them to define records (like
structs in C) with selector functions.

The namedtuple recipe by Raymond Hetting does
exactly that and, guess what, it uses exec!
I might be wrong, but the reason namedtuple uses exec is performance.
IIRC earlier versions of the recipe used a metaclass instead, so it's
not that it *has* to use exec, it's just an optimization, totally
justified in this case since namedtuples should be almost as fast as
plain tuples.

George
Aug 16 '08 #8
Nadeem wrote:
I understand that all this can be done with classes and OO
programming, but the whole point of the HtDP curriculum is to
introduce students to programming in a pedagogically-effective way
using a functional approach instead of OO-first.
And yet, one of the HtDP authors just posted a paper that argues that
it's time to abandon the whole idea of "programmin g language paradigms"
in teaching, and focus on language features instead.

Most books rigorously adhere to the sacred division of languages
into "functional ", "imperative ", "object-oriented", and "logic"
camps. I conjecture that this desire for taxonomy is an artifact
of our science-envy from the early days of our discipline: a
misguided attempt to follow the practice of science rather than
its spirit. We are, however, a science of the artificial. What
else to make of a language like Python, Ruby, or Perl? Their
designers have no patience for the niceties of these Linnaean
hierarchies; they borrow features as they wish, creating melanges
that utterly defy characterizatio n. How do we teach PL in this
post-Linnaean era?

http://www.cs.brown.edu/~sk/Publicat...post-linnaean/

Alright, his emphasis is on teaching programming language *design*
features (and there's still the usual tone of "Scheme is right all the
time and the others make mistakes all the time", despite the recent RS6S
brouhaha ;-), but I'd say you should think twice before exposing your
students to an absurdly artificial rendering of one language's model in
another language's syntax. It's trivial to show how Python's method
*syntax* is used to associate ordinary functions with "attribute
containers" without having to first introduce OO as a formal concept.

</F>

Aug 16 '08 #9
Le Saturday 16 August 2008 06:50:02 Michele Simionato, vous avez écrit*:
On Aug 16, 4:48*am, Nadeem <nadeemabdulha. ..@gmail.comwro te:
I understand the 99% rule... the example I gave was to simplify the
issue. The full thing I'm working on is a library for an introductory
CS class I'm teaching. I'm trying, essentially, to build a library of
macros for students to use allowing them to define records (like
structs in C) with selector functions.

The namedtuple recipe by Raymond Hetting does
exactly that and, guess what, it uses exec!
It uses exec, but could not, and IMO, should not, all of the cooking could be
done in the closure. I join version of it without exec. Of course the only
drawback is that you need to do the job of argument checking in __new__ and
can't rely on a dynamically compiled arglist. But it is a common idiom I
think.
Also the doctest module in the standard library
uses exec at good effect. So, I would say it is
not a 99% rule, let's say it is a 98% rule ;)
Yes, but in doctest, exec is used for what it is intended to be use, execute
arbitrary code provided by the user, this is legal and perfect use of exec
because here the user is a trusted one (the programer himself).

I'd say that everywhere exec/eval are used in a application/function/lib that
doesn't mean to interpret arbitrary and user provided python code, it is a
bad usage.


--
_____________

Maric Michaud

Aug 16 '08 #10

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

3
19314
by: Fernando Rodriguez | last post by:
Hi, How can I get the list of all the functions defined in the current module?
4
1568
by: Joerg Schuster | last post by:
Hello, I want to refer to all functions of a module, say test2.py, without having to manually enumerate them. The following code comes closest to what I am trying to do. It does not work, because dir() does not return functions, but function names as strings. ##### test2.py #####
6
2031
by: Kamilche | last post by:
I can't figure out how to list all functions from an imported module. I searched Google, but all the answers I found didn't work. Did something change in Python 2.2, perhaps there's a new method of doing it?
5
1569
by: James Tauber | last post by:
Had a question from a colleague that I embarrassingly couldn't answer. He has a script, foo.py with a global. He wants to import bar.py and needs that global available in bar.py The following obviously doesn't work: # foo.py my_global = "hello" print globals().keys()
5
15698
by: Ed Havelaar | last post by:
I have a cool function that I want to use as a default value for a column in my table. But I can't because apparently Access doesn't allow user defined functions in expressions for default values. It's very annoying. Is there any way around it? What I need is something like AutoNumber, *except* I want the values to be unique across six different tables. Any ideas? Unknown function <name> in validation expression or default value...
2
5480
by: David Emme | last post by:
Access 97 I have a number of SELECT statements which contain references to user-defined VBA functions. These typically work as expected, but occasionally, on one user's machine or another, produce a "function not defined" message when the SELECT statement is executed, even having previously worked on that machine. What can I do to correct this difficulty when it occurs? What do I need to know about user-defined functions in SELECT...
4
1296
by: Paul McGuire | last post by:
I have a new enhancement to pyparsing that doubles the parse speed (using a technique called "packrat parsing"), but which is not suitable for all parsers, specifically those that have complex parse actions. I don't want to just enable this feature by default - I think there is too much risk of it breaking existing code. Actually, I think the chance is <10%, but I think I'm better off leaving the feature dormant unless explicitly enabled,...
0
1058
by: Mike | last post by:
Hi. I have a vb.net/asp.net application, originally created in VS2003. I'm upgrading it to VS2005, and have found that Public Functions in a module are no longer accessable in my application. My code looks like this - and in this example I cannot call Function 123 within my application. --- Public Module ABC Public Function 123() as Boolean ...
1
1132
by: anil.pundoor | last post by:
hi all, i have following code if <condition> from SIPT.xml_param_mapping import MESSAGE_PARAMETER_MAPPING else: from SIPT.msg_param_mapping import MESSAGE_PARAMETER_MAPPING
0
8569
marktang
by: marktang | last post by:
ONU (Optical Network Unit) is one of the key components for providing high-speed Internet services. Its primary function is to act as an endpoint device located at the user's premises. However, people are often confused as to whether an ONU can Work As a Router. In this blog post, we’ll explore What is ONU, What Is Router, ONU & Router’s main usage, and What is the difference between ONU and Router. Let’s take a closer look ! Part I. Meaning of...
0
8504
by: Hystou | last post by:
Most computers default to English, but sometimes we require a different language, especially when relocating. Forgot to request a specific language before your computer shipped? No problem! You can effortlessly switch the default language on Windows 10 without reinstalling. I'll walk you through it. First, let's disable language synchronization. With a Microsoft account, language settings sync across devices. To prevent any complications,...
1
8758
by: Hystou | last post by:
Overview: Windows 11 and 10 have less user interface control over operating system update behaviour than previous versions of Windows. In Windows 11 and 10, there is no way to turn off the Windows Update option using the Control Panel or Settings app; it automatically checks for updates and installs any it finds, whether you like it or not. For most users, this new feature is actually very convenient. If you want to control the update process,...
0
8772
tracyyun
by: tracyyun | last post by:
Dear forum friends, With the development of smart home technology, a variety of wireless communication protocols have appeared on the market, such as Zigbee, Z-Wave, Wi-Fi, Bluetooth, etc. Each protocol has its own unique characteristics and advantages, but as a user who is planning to build a smart home system, I am a bit confused by the choice of these technologies. I'm particularly interested in Zigbee because I've heard it does some...
0
7584
agi2029
by: agi2029 | last post by:
Let's talk about the concept of autonomous AI software engineers and no-code agents. These AIs are designed to manage the entire lifecycle of a software development project—planning, coding, testing, and deployment—without human intervention. Imagine an AI that can take a project description, break it down, write the code, debug it, and then launch it, all on its own.... Now, this would greatly impact the work of software developers. The idea...
0
4299
by: TSSRALBI | last post by:
Hello I'm a network technician in training and I need your help. I am currently learning how to create and manage the different types of VPNs and I have a question about LAN-to-LAN VPNs. The last exercise I practiced was to create a LAN-to-LAN VPN between two Pfsense firewalls, by using IPSEC protocols. I succeeded, with both firewalls in the same network. But I'm wondering if it's possible to do the same thing, with 2 Pfsense firewalls...
0
4521
by: adsilva | last post by:
A Windows Forms form does not have the event Unload, like VB6. What one acts like?
1
2935
by: 6302768590 | last post by:
Hai team i want code for transfer the data from one system to another through IP address by using C# our system has to for every 5mins then we have to update the data what the data is updated we have to send another system
3
1921
bsmnconsultancy
by: bsmnconsultancy | last post by:
In today's digital era, a well-designed website is crucial for businesses looking to succeed. Whether you're a small business owner or a large corporation in Toronto, having a strong online presence can significantly impact your brand's success. BSMN Consultancy, a leader in Website Development in Toronto offers valuable insights into creating effective websites that not only look great but also perform exceptionally well. In this comprehensive...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.