469,354 Members | 2,046 Online
Bytes | Developer Community
New Post

Home Posts Topics Members FAQ

Post your question to a community of 469,354 developers. It's quick & easy.

Timing out arbitrary functions

I have a problem and I don't know where to start looking for a solution.

I have a class that needs to call an arbitrary function and wait for a
result. The function, being completely arbitrary and not under my control,
may be very time consuming and possibly may not even halt. My class needs
to be able to give up waiting for a result after a specified amount of
time.

I'm thinking something conceptually like this:

# pseudo-code:
set time out to 30 seconds
try:
result = somefunction()
except TimeOut:
# 30 second time out happened
print "somefunction() timed out without returning"
else:
print "somefunction() returned %s" % result
The easy (for some definition of easy) solution would be to code
somefunction() so that it raised an exception if it hadn't returned a
result within a certain time. Unfortunately, I can't do rely on that -- I
only have control over the calling code, not the called somefunction(),
which may be any arbitrary function.

How do others handle something like this? What should I be looking for?
I'm after a lightweight solution, if any such thing exists.

Thanks,
--
Steven.

Dec 24 '05 #1
5 1353
Steven D'Aprano <st***@REMOVETHIScyber.com.au> writes:
How do others handle something like this? What should I be looking for?
I'm after a lightweight solution, if any such thing exists.


Is something stopping you from using sigalarm?
Dec 24 '05 #2
Steven D'Aprano wrote:
I have a problem and I don't know where to start looking for a solution.

I have a class that needs to call an arbitrary function and wait for a
result. The function, being completely arbitrary and not under my control,
may be very time consuming and possibly may not even halt. My class needs
to be able to give up waiting for a result after a specified amount of
time.

I'm thinking something conceptually like this:

# pseudo-code:
set time out to 30 seconds
try:
result = somefunction()
except TimeOut:
# 30 second time out happened
print "somefunction() timed out without returning"
else:
print "somefunction() returned %s" % result
The easy (for some definition of easy) solution would be to code
somefunction() so that it raised an exception if it hadn't returned a
result within a certain time. Unfortunately, I can't do rely on that -- I
only have control over the calling code, not the called somefunction(),
which may be any arbitrary function.

How do others handle something like this? What should I be looking for?
I'm after a lightweight solution, if any such thing exists.


For simple cases, I would use signal.alarm() with a SIGALARM handler
that raises a TimeOut exception. However, this is by no means
foolproof; you have to rely on the called function not to mess with
your signal handler. Plus, if your alarm occurs within a try-except
block that catches the TimeOut, it'll still be dropped. And to the best
of my knowledge, you can't otherwise forcibly terminate the execution
of a Python thread or block of code.

If you're going to be running untrusted code, I would use the
subprocess module to invoke a separate Python instance which takes the
code to be executed on stdin, and returns a pickled copy of the return
value on stdout. Then you can start it running, wait 30 seconds, and
then kill it if it hasn't already returned.

-- David

Dec 24 '05 #3
On Sat, 24 Dec 2005 04:47:34 -0800, Paul Rubin wrote:
Steven D'Aprano <st***@REMOVETHIScyber.com.au> writes:
How do others handle something like this? What should I be looking for?
I'm after a lightweight solution, if any such thing exists.


Is something stopping you from using sigalarm?


Pure ignorance of its existence.
Thanks, I'll check it out.
--
Steven.

Dec 24 '05 #4
Steven D'Aprano <st***@REMOVETHIScyber.com.au> writes:
Is something stopping you from using sigalarm?


Pure ignorance of its existence.
Thanks, I'll check it out.


Two things to keep in mind:

- You can have only ONE alarm pending for the whole process. If
different things in the program need timeouts of their own, you have
to manage that yourself, maybe with heapq. And the thing you're
trying to time out may itself mess with the alarm or its handler.

- The alarm raises an exception in the main thread. If you want a
timeout in some other thread, you're more or less out of luck. Antoon
Pardon has posted a couple times about a ctypes-dependent hack that
raises asynchronous exceptions in arbitrary threads, that might be
worth looking into if you have to. I haven't done so for now.

Besides sigalarm you might be able to concoct something with SIGIO
(have another thread sleep til the timeout then send a character back
to the main process through a pipe) or some other signal (use
os.kill). The same issues would apply as with sigalarm.
Dec 24 '05 #5
AOP would be a quite elegant way set timeouts for functions, in my
opinion. The nice thing in it is that, in principle, you can write a
single timeout advice code and then wrap it over any function you want
to timeout.

I wrote timeout_advice.py to demonstrate this a couple of years ago
(see http://www.cs.tut.fi/~ask/aspects/aspects.html). It may not
directly solve the problem at hand because it is thought to be used
with a wrap_around implementation that wraps methods in classes rather
than ordinary functions in modules. urllib.URLopener.open is used as an
example in the code. Unfortunately, I still have not implemented the
wrapping for ordinary functions, although it should be straight-forward
with the same idea that is explained in the web page.

Of course, in the real life, timeouts are tricky and dangerous. The
consequences of interrupting a function that is not designed to be
interrupted, or leaving it running in the background after the timeout
(which is what timeout_advice.py does) may be surprising.

-- Antti Kervinen

Dec 28 '05 #6

This discussion thread is closed

Replies have been disabled for this discussion.

Similar topics

8 posts views Thread by Steve Neill | last post: by
3 posts views Thread by Randy Yates | last post: by
2 posts views Thread by Steven D'Aprano | last post: by
1 post views Thread by Gary Coutts | last post: by
28 posts views Thread by walterbyrd | last post: by
1 post views Thread by MartyFromIreland | last post: by
reply views Thread by Daniel Fetchinson | last post: by
reply views Thread by zhoujie | last post: by
1 post views Thread by Marylou17 | last post: by
By using this site, you agree to our Privacy Policy and Terms of Use.