473,721 Members | 2,060 Online
Bytes | Software Development & Data Engineering Community
+ Post

Home Posts Topics Members FAQ

specifying constants for a function (WAS: generator expressions:per formance anomaly?)

I wrote:
If you really want locals that don't contribute to arguments, I'd be
much happier with something like a decorator, e.g.[1]:

@with_consts(i= 1, deftime=time.ct ime())
def foo(x, y=123, *args, **kw):
return x*y, kw.get('which_t ime')=='now' and time.ctime() or deftime

Then you don't have to mix parameter declarations with locals
definitions.

Steve

[1] I have no idea how implementable such a decorator would be. I'd
just like to see function constants declared separate from arguments
since they mean such different things.


I played around with this, and I think it's basically implementable:

py> import new
py> class with_consts(obj ect):
.... def __init__(self, **consts):
.... self.consts = consts
.... def __call__(self, func):
.... return new.function(fu nc.func_code,
.... dict(globals(), **self.consts))
....
py> @with_consts(y= 123)
.... def f(x):
.... return x*y, str
....
py> f(2)
(246, <type 'str'>)

I just update the function globals with the keywords passed to the
decorator. The only problem is that updates to globals() aren't
reflected in the function's globals:

py> str = 5
py> f(2)
(246, <type 'str'>)

Steve
Jul 18 '05 #1
6 1177
Steven Bethard wrote:
I wrote:
> If you really want locals that don't contribute to arguments, I'd be
> much happier with something like a decorator, e.g.[1]:
>
> @with_consts(i= 1, deftime=time.ct ime())
> def foo(x, y=123, *args, **kw):
> return x*y, kw.get('which_t ime')=='now' and time.ctime() or deftime
>
> Then you don't have to mix parameter declarations with locals
> definitions.
>
> Steve
>
> [1] I have no idea how implementable such a decorator would be. I'd
> just like to see function constants declared separate from arguments
> since they mean such different things.


I played around with this, and I think it's basically implementable:


Raymond's constant binding decorator is probably a good model for how to do it:
http://aspn.activestate.com/ASPN/Coo.../Recipe/277940

Cheers,
Nick.

--
Nick Coghlan | nc******@email. com | Brisbane, Australia
---------------------------------------------------------------
http://boredomandlaziness.skystorm.net
Jul 18 '05 #2
On Sun, 23 Jan 2005 13:14:10 -0700, Steven Bethard <st************ @gmail.com> wrote:
Bengt Richter wrote:
On Sat, 22 Jan 2005 16:22:33 +1000, Nick Coghlan <nc******@iinet .net.au> wrote:

[Steven Bethard]
> If you really want locals that don't contribute to arguments, I'd be
> much happier with something like a decorator, e.g.[1]:
>
> @with_consts(i= 1, deftime=time.ct ime())
> def foo(x, y=123, *args, **kw):
> return x*y, kw.get('which_t ime')=='now' and time.ctime() or deftime
>
> Then you don't have to mix parameter declarations with locals
> definitions. [...]

[Nick Coghlan]Raymond's constant binding decorator is probably a good model for how to do it:
http://aspn.activestate.com/ASPN/Coo.../Recipe/277940

[Bengt Richter]
I thought so too. I modified it to accept **presets as a keyword argument
and generate constants and assignments from its values matching assignment names
when the rhs was __frompresets__ , e.g.,
>>> from makeconstpre import make_constants as pre
>>> import time
>>> @pre(verbose=Tr ue, deftime=time.ct ime(), a=1, b=2, c=3, pi=__import__(' math').pi)

... def foo():
... deftime = __frompresets__
... b, a = __frompresets__
... c = __frompresets__
... pi = __frompresets__
... return locals()
...
__frompresets__ : deftime --> Sat Jan 22 20:18:09 2005
__frompresets__ : ('b', 'a') --> (2, 1)
__frompresets__ : c --> 3
__frompresets__ : pi --> 3.14159265359
locals --> <built-in function locals>

[Steven Bethard]Hmm... Having to state
deftime = __frompresets__
when you've already stated
deftime=time.ct ime()
seems a little redundant to me...

Yeah, but as a quick hack, it made the byte code twiddling all changing byte codes in-place.
I didn't want to verify that byte code wouldn't need relocation fixups if I inserted something,
(unless of course within a suite with branches or loops) but hopefully it's easily so at the
beginning.

BTW, unlike default arguments, varname = __frompresets__ will work anywhere in the code,
(for the current decorator version) and the generated byte code will have a load of
an appropriate constant (the same one if the same name is being assigned). This would
be tricky to get right if done in a hidden just-in-time manner only in branches that
used the values, but hidden initialization of the whole presets set at the beginning
should be relatively easy if there are no relocation problems.
So, e.g., for
presets = dict(a=1, b=2, deftime=__impor t__('time').cti me())
in the decorator args, the next version will act as if the decorated
function had the source code
print '%s = __frompresets__ ' % ', '.join(sorted(p resets))

a, b, deftime = __frompresets__

for the current version, except that it will be hidden.

Regards,
Bengt Richter
Jul 18 '05 #3
Bengt Richter wrote:
So, e.g., for
>>> presets = dict(a=1, b=2, deftime=__impor t__('time').cti me())
in the decorator args, the next version will act as if the decorated
function had the source code
>>> print '%s = __frompresets__ ' % ', '.join(sorted(p resets))

a, b, deftime = __frompresets__

for the current version, except that it will be hidden.


Cool! Keep us posted. I assume you'll put this into the Cookbook?

Steve
Jul 18 '05 #4
On Mon, 24 Jan 2005 00:31:17 -0700, Steven Bethard <st************ @gmail.com> wrote:
Bengt Richter wrote:
So, e.g., for
>>> presets = dict(a=1, b=2, deftime=__impor t__('time').cti me())


in the decorator args, the next version will act as if the decorated
function had the source code
>>> print '%s = __frompresets__ ' % ', '.join(sorted(p resets))

a, b, deftime = __frompresets__

for the current version, except that it will be hidden.


Cool! Keep us posted. I assume you'll put this into the Cookbook?

Not ready for prime time, I guess, but I made a module that does presets
and also adjusts parameters and count to do currying without nested function
calling in the curried function.
from presets import presets, curry
@presets(verbos e=True, a=1, b=2, deftime=__impor t__('time').cti me()) ... def foo():
... print (a, b)
... print deftime
...

presets: -- "name(?)" means name may be unused
a = 1
b = 2
deftime = 'Mon Jan 24 12:16:07 2005'
foo() (1, 2)
Mon Jan 24 12:16:07 2005 @curry(verbose= True, y=444) ... def bar(x=3, y=4):
... return x*y
...

presets: -- "name(?)" means name may be unused
y = 444
bar(2) 888 bar(2, 3) Traceback (most recent call last):
File "<stdin>", line 1, in ?
TypeError: bar() takes at most 1 argument (2 given) bar() 1332 import dis
dis.dis(bar) 1 0 LOAD_CONST 1 (444)
3 STORE_FAST 1 (y)

3 6 LOAD_FAST 0 (x)
9 LOAD_FAST 1 (y)
12 BINARY_MULTIPLY
13 RETURN_VALUE dis.dis(foo) 1 0 LOAD_CONST 1 ((1, 2, 'Mon Jan 24 12:16:07 2005'))
3 UNPACK_SEQUENCE 3
6 STORE_FAST 0 (a)
9 STORE_FAST 1 (b)
12 STORE_FAST 2 (deftime)

3 15 LOAD_FAST 0 (a)
18 LOAD_FAST 1 (b)
21 BUILD_TUPLE 2
24 PRINT_ITEM
25 PRINT_NEWLINE

4 26 LOAD_FAST 2 (deftime)
29 PRINT_ITEM
30 PRINT_NEWLINE
31 LOAD_CONST 0 (None)
34 RETURN_VALUE

Now I have to see whether I can optimize the result with Raymond's make_constants
as two decorators in a row... (pre without keyword args is an alias for Raymond's make_constants,
just handy for the moment)
from pre import pre
@pre(verbose=Tr ue) ... @presets(verbos e=True, a=1, b=2, deftime=__impor t__('time').cti me())
... def foo():
... print (a, b)
... print deftime
... print __name__
...

presets: -- "name(?)" means name may be unused
a = 1
b = 2
deftime = 'Mon Jan 24 12:29:21 2005'

__name__ --> __main__ foo() (1, 2)
Mon Jan 24 12:29:21 2005
__main__ dis.dis(foo)

1 0 LOAD_CONST 1 ((1, 2, 'Mon Jan 24 12:29:21 2005'))
3 UNPACK_SEQUENCE 3
6 STORE_FAST 0 (a)
9 STORE_FAST 1 (b)
12 STORE_FAST 2 (deftime)

3 15 LOAD_FAST 0 (a)
18 LOAD_FAST 1 (b)
21 BUILD_TUPLE 2
24 PRINT_ITEM
25 PRINT_NEWLINE

4 26 LOAD_FAST 2 (deftime)
29 PRINT_ITEM
30 PRINT_NEWLINE

5 31 LOAD_CONST 2 ('__main__')
34 PRINT_ITEM
35 PRINT_NEWLINE
36 LOAD_CONST 0 (None)
39 RETURN_VALUE

Regards,
Bengt Richter
Jul 18 '05 #5
On Mon, 24 Jan 2005 20:35:09 GMT, bo**@oz.net (Bengt Richter) wrote:
On Mon, 24 Jan 2005 00:31:17 -0700, Steven Bethard <st************ @gmail.com> wrote:
Bengt Richter wrote:
So, e.g., for

>>> presets = dict(a=1, b=2, deftime=__impor t__('time').cti me())

in the decorator args, the next version will act as if the decorated
function had the source code

>>> print '%s = __frompresets__ ' % ', '.join(sorted(p resets))
a, b, deftime = __frompresets__

for the current version, except that it will be hidden.


Cool! Keep us posted. I assume you'll put this into the Cookbook?

Not ready for prime time, I guess, but I made a module that does presets
and also adjusts parameters and count to do currying without nested function
calling in the curried function.
from presets import presets, curry
@presets(verbos e=True, a=1, b=2, deftime=__impor t__('time').cti me()) ... def foo():
... print (a, b)
... print deftime
...

presets: -- "name(?)" means name may be unused
a = 1
b = 2
deftime = 'Mon Jan 24 12:16:07 2005'
foo() (1, 2)
Mon Jan 24 12:16:07 2005 @curry(verbose= True, y=444) ... def bar(x=3, y=4):
... return x*y
...

presets: -- "name(?)" means name may be unused
y = 444
bar(2) 888 bar(2, 3) Traceback (most recent call last):
File "<stdin>", line 1, in ?
TypeError: bar() takes at most 1 argument (2 given) bar()

1332


Not much reaction, so I guess I won't see it if any, since I am going off line for a while.
BTW, this exercise makes me realize that you could do some other interesting things too,
if you put your mind to it. E.g., it might be interesting to inject properties into
the local namespace of a function, so that e.g., x=1 would would trigger an effect
like someobj.x = 1, where type(someobj).x was a property. It wouldn't be
too hard to make

@addprop(verbos e=True, x=property(getx , setx, delx))
def foo():
x = 123
return x

etc. Maybe could be used to monitor access to specific locals for debug,
or other uses. Haven't even begun to think about that.

Another thing is to turn selected local defs into constants, since executing
the aggregation-of-parts code that accomplished the def may well be unchanging.
This would reduce the execution cost of nested functions. I guess that is
a specific case of a more general sticky-evaluate-rhs-once kind of directive
for locals that are only assigned in one place. special sentinels could
be used in the decorator keyword values to indicate this treatment of the
associated name, e.g.
from presets import presets
@presets(x=pres ets.STICKY, y=123)
def foo(y):
x = initx() # done once on the first call, making x references constant after that
return x*y

Another thing that could be done is to inject pre and post things like atexit stuff.
Also maybe some adaptation things that is just a buzzword so far that I haven't explored.

BTW, is there another op that needs relocation besides JUMP_ABSOLUTE?

I was wondering about doing code mods by messing with an ast, but a decorator would
have to chase down the source and reparse I guess, in order to do that. It would be
interesting to be able to hook into ast stuff with some kind of metacompile hook (hand waving ;-)

Bye for a while.

Regards,
Bengt Richter
Jul 18 '05 #6
Bengt Richter wrote:
On Mon, 24 Jan 2005 20:35:09 GMT, bo**@oz.net (Bengt Richter) wrote:
>from presets import presets, curry
>@presets(v erbose=True, a=1, b=2, deftime=__impor t__('time').cti me())


... def foo():
... print (a, b)
... print deftime
...

presets: -- "name(?)" means name may be unused
a = 1
b = 2
deftime = 'Mon Jan 24 12:16:07 2005'

>foo()


(1, 2)
Mon Jan 24 12:16:07 2005


Not much reaction


Sorry, it's definitely cool -- I was just waiting for you to post the
Cookbook link so I could see how you did it. ;)

Steve
Jul 18 '05 #7

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

72
4398
by: Raymond Hettinger | last post by:
Peter Norvig's creative thinking triggered renewed interest in PEP 289. That led to a number of contributors helping to re-work the pep details into a form that has been well received on the python-dev list: http://www.python.org/peps/pep-0289.html In brief, the PEP proposes a list comprehension style syntax for creating fast, memory efficient generator expressions on the fly: sum(x*x for x in roots)
10
1905
by: John Machin | last post by:
Please consider the timings below, where a generator expression starts out slower than the equivalent list comprehension, and gets worse: >python -m timeit -s "orig=range(100000)" "lst=orig;lst=(x for x in orig)" 10 loops, best of 3: 6.84e+004 usec per loop >python -m timeit -s "orig=range(200000)" "lst=orig;lst=(x for x in orig)" 10 loops, best of 3: 5.22e+005 usec per loop
2
1291
by: Johannes Ahl mann | last post by:
hi, i am in the process of profiling an application and noticed how much time my depth first generator for walking a tree data structure took. i wrote the same thing as a recursive function producing an array, and this non-generator version is nearly 10 times faster! any ideas why this is, what i did wrong... for some reason the generator version appears as being called much more
0
306
by: guy | last post by:
has anyone useful info on running winforms apps under citrix, with approx 10-20 users? (oracle back end on separate server) how good was performance? cheers guy
3
2819
by: Alexandre H. Guerra | last post by:
Hello I need to process a SQL monitoring log stored in a table to group the statements that change just the constants in it. Ex: select a,b,c from table where (a = 'xyz' and b = 123 and c != '2004-10-10') or ( a like 'def%' ) select a,b,c from table where (a = 'zyx' and b = 321 and c != '2004-01-01') or ( a like '%fed' )
5
5428
by: James Dean | last post by:
I wanted to use regular expressions but unfortunetely it is too slow.....Should they be so slow or am i doing something wrong. I am reading in bytes from a file then converting them to char then making a string out of each of the individual bytes. I check if its in the correct format...and take out the various paretres i need. It looked nice and neat so i am not happy that i may have to use another method.....any alternative solutions?.
11
1676
by: Christof Nordiek | last post by:
Hi all, Can there be a performance difference, if i use readonly fields instead of constants? e.g.: const int n = 100; vs. static readonly int = 100;
5
1634
by: Bruce | last post by:
Hello I am building a C# app that creates anywhere from 10 to 100 connections to a specified server and sends 1000s of TCP requests and processes the responses. (it is a stress tool) I planned to create a Socket object for each connection and do a BeginReceive on each socket to handle the responses. (it is clean and I can avoid the hassles of managing my own threads) My question is, will this scale well? Or is there a more efficient...
13
1330
by: William McBrine | last post by:
Hi all, I'm pretty new to Python (a little over a month). I was wondering -- is something like this: s = re.compile('whatever') def t(whatnot): return s.search(whatnot)
0
8736
by: Hystou | last post by:
Most computers default to English, but sometimes we require a different language, especially when relocating. Forgot to request a specific language before your computer shipped? No problem! You can effortlessly switch the default language on Windows 10 without reinstalling. I'll walk you through it. First, let's disable language synchronization. With a Microsoft account, language settings sync across devices. To prevent any complications,...
0
9373
Oralloy
by: Oralloy | last post by:
Hello folks, I am unable to find appropriate documentation on the type promotion of bit-fields when using the generalised comparison operator "<=>". The problem is that using the GNU compilers, it seems that the internal comparison operator "<=>" tries to promote arguments from unsigned to signed. This is as boiled down as I can make it. Here is my compilation command: g++-12 -std=c++20 -Wnarrowing bit_field.cpp Here is the code in...
0
9077
tracyyun
by: tracyyun | last post by:
Dear forum friends, With the development of smart home technology, a variety of wireless communication protocols have appeared on the market, such as Zigbee, Z-Wave, Wi-Fi, Bluetooth, etc. Each protocol has its own unique characteristics and advantages, but as a user who is planning to build a smart home system, I am a bit confused by the choice of these technologies. I'm particularly interested in Zigbee because I've heard it does some...
0
8019
agi2029
by: agi2029 | last post by:
Let's talk about the concept of autonomous AI software engineers and no-code agents. These AIs are designed to manage the entire lifecycle of a software development project—planning, coding, testing, and deployment—without human intervention. Imagine an AI that can take a project description, break it down, write the code, debug it, and then launch it, all on its own.... Now, this would greatly impact the work of software developers. The idea...
1
6676
isladogs
by: isladogs | last post by:
The next Access Europe User Group meeting will be on Wednesday 1 May 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM). In this session, we are pleased to welcome a new presenter, Adolph Dupré who will be discussing some powerful techniques for using class modules. He will explain when you may want to use classes instead of User Defined Types (UDT). For example, to manage the data in unbound forms. Adolph will...
0
5992
by: conductexam | last post by:
I have .net C# application in which I am extracting data from word file and save it in database particularly. To store word all data as it is I am converting the whole word file firstly in HTML and then checking html paragraph one by one. At the time of converting from word file to html my equations which are in the word document file was convert into image. Globals.ThisAddIn.Application.ActiveDocument.Select();...
0
4497
by: TSSRALBI | last post by:
Hello I'm a network technician in training and I need your help. I am currently learning how to create and manage the different types of VPNs and I have a question about LAN-to-LAN VPNs. The last exercise I practiced was to create a LAN-to-LAN VPN between two Pfsense firewalls, by using IPSEC protocols. I succeeded, with both firewalls in the same network. But I'm wondering if it's possible to do the same thing, with 2 Pfsense firewalls...
2
2588
muto222
by: muto222 | last post by:
How can i add a mobile payment intergratation into php mysql website.
3
2137
bsmnconsultancy
by: bsmnconsultancy | last post by:
In today's digital era, a well-designed website is crucial for businesses looking to succeed. Whether you're a small business owner or a large corporation in Toronto, having a strong online presence can significantly impact your brand's success. BSMN Consultancy, a leader in Website Development in Toronto offers valuable insights into creating effective websites that not only look great but also perform exceptionally well. In this comprehensive...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.