Bizarre additional calling overhead.

[Chris Mellon]

>>def test_func():
.... pass
....

>>import new
test_func2 = new.function(test_func.func_code, {}, "test_func2")
test_func2

<function test_func2 at 0x01B8C2F0>

>>test_func

<function test_func at 0x01B8C270>

>>import timeit
tf = timeit.Timer("test_func()", "from __main__ import test_func")
tf.repeat()

[0.2183461704377247, 0.18068215314489791, 0.17978585841498085]

>>tf2 = timeit.Timer("test_func2()", "from __main__ import test_func2")
tf2.repeat()

[0.40015390239890891, 0.35893452879396648, 0.36034628133737456]

>>>

Why almost twice the calling overhead for a dynamic function?

[Matimus]

On Nov 2, 3:08 pm, "Chris Mellon" <[e-mail]:

>def test_func():

... pass
...>>import new

>test_func2 = new.function(test_func.func_code, {}, "test_func2")
test_func2

<function test_func2 at 0x01B8C2F0>>>test_func

<function test_func at 0x01B8C270>>>import timeit

>tf = timeit.Timer("test_func()", "from __main__ import test_func")
tf.repeat()

[0.2183461704377247, 0.18068215314489791, 0.17978585841498085]>>tf2 = timeit.Timer("test_func2()", "from __main__ import test_func2")

>tf2.repeat()

[0.40015390239890891, 0.35893452879396648, 0.36034628133737456]

Why almost twice the calling overhead for a dynamic function?

So, I don't have an official explanation for why it takes twice as
long, but the only difference between the two functions I could find
was that test_func.func_globals was set to globals() and
test_func2.func_globals was an empty dict. When I re-created
test_func2 with globals set to globals() it ran just as fast as
test_func.

>>test_func2 = new.function(test_func.func_code, globals(), "test_func")
tf2 = timeit.Timer("test_func2()", "from __main__ import test_func2")
tf2.repeat()

[0.18119118491313202, 0.18396220748718406, 0.18722407456812107]

>>tf.repeat()

[0.18125124841299112, 0.17981251807145782, 0.18517996002287873]

Matt

[Steven D'Aprano]

On Fri, 02 Nov 2007 17:08:06 -0500, Chris Mellon wrote:

>>>def test_func():

... pass
...

>>>import new
test_func2 = new.function(test_func.func_code, {}, "test_func2")

[snip results of timeit]

Why almost twice the calling overhead for a dynamic function?

When I time the functions, I get a lot of variability. The minimum value
is probably the best one to look at:

>>min(timeit.Timer("test_func()",

.... "from __main__ import test_func").repeat(20))
0.35664010047912598

>>min(timeit.Timer("test_func2()",

.... "from __main__ import test_func2").repeat(20))
0.68138217926025391
Disassembling the code does not give any hints:

>>import dis
dis.dis(test_func)

2 0 LOAD_CONST 0 (None)
3 RETURN_VALUE

>>dis.dis(test_func2)

2 0 LOAD_CONST 0 (None)
3 RETURN_VALUE

Which is what we should expect, because both functions have the same code:

>>test_func.func_code is test_func2.func_code

True
But try this:

>>test_func3 = new.function(test_func.func_code,

.... globals(), 'test_func3')

>>min(timeit.Timer("test_func3()",

.... "from __main__ import test_func3").repeat(20))
0.35772204399108887
Hmmm... it looks like the difference in speed has to do with the globals,
not the fact that it is created dynamically. I wonder why?


--
Steven.