returning a value from a thread

On Wed, 14 Jul 2004, Ken Godee wrote:

I thought there was some sort of memory type container
I could use...

ie. store value in memory from module3 and be able to
read it from module1.

The closest I can think of is to pass a queue reference
and stor values from module3 in the queue and then read them
back while in module1.

Actually, any kind of container object would do: you could pass a list,
dictionary, or class to the thread, and the thread could store its results
in it.

A hackish method would be to pass the dictionary returned by locals() in
module1 to the thread. This way the thread could set a value in module1
directly.

Just don't forget to .join() on the thread in module1 before accessing the
container object! (Assuming you're using threading) I just wish Python's
..join() could return values, like pthread_join can.

Christopher T King <sq******@WPI.E DU> writes:

Just don't forget to .join() on the thread in module1 before accessing the
container object! (Assuming you're using threading) I just wish Python's
.join() could return values, like pthread_join can.

Although if you can join on the thread, then you have to have a
reference to the thread object, at which point you can do anything you
want in terms of permitting state to be interrogated on that object
(direct attribute access, getters, etc...), which is even better than
having join return an object.

-- David

Christopher T King wrote:

On Wed, 14 Jul 2004, Ken Godee wrote:

I thought there was some sort of memory type container
I could use...

ie. store value in memory from module3 and be able to
read it from module1.

The closest I can think of is to pass a queue reference
and stor values from module3 in the queue and then read them
back while in module1.

Actually, any kind of container object would do: you could pass a list,
dictionary, or class to the thread, and the thread could store its results
in it.

A hackish method would be to pass the dictionary returned by locals() in
module1 to the thread. This way the thread could set a value in module1
directly.

Just don't forget to .join() on the thread in module1 before accessing the
container object! (Assuming you're using threading) I just wish Python's
.join() could return values, like pthread_join can.

This may work if the worker thread will perform a relatively short task
and then die *before* you access the result. But lists and dictionaries
are not thread-safe -- if they are potentially accessed by multiple
threads concurrently, then the behavior will be unpredictable. (Think
of a case where thread B starts to update a dictionary, inserts a key
but is interrupted before it can attach a value to that key, and then
while thread B is interrupted thread A looks at the dictionary and finds
the key it's looking for, but with no valid reference as its value...
[Disclaimer: I don't know the inner workings of dictionaries well enough
to know if this exact situation is possible, but I do know that dicts
are not threadsafe, so something *similar* is possible...])

Unless you're certain that your worker thread can die before you get the
results, and you're able to have your main thread potentially sit around
doing nothing until that happens (which is what join() does), you need
to use a threadsafe method of passing data back and forth. The simplest
such method is indeed to use a queue. You *can* use some other
container, and protect access to that container through the use of
locks.... but that's exactly what a queue does, so why reinvent the wheel?

Jeff Shannon
Technician/Programmer
Credit International

On Wed, 14 Jul 2004, Jeff Shannon wrote:

Christopher T King wrote:

Just don't forget to .join() on the thread in module1 before accessing the
container object! (Assuming you're using threading) I just wish Python's
.join() could return values, like pthread_join can.

This may work if the worker thread will perform a relatively short task
and then die *before* you access the result. But lists and dictionaries
are not thread-safe -- if they are potentially accessed by multiple
threads concurrently, then the behavior will be unpredictable.

That's where the .join() comes in handy (it blocks until the thread dies)
;)

Ken Godee wrote:

[detailed version of the question in the subject line]

This question is asked often enough that it maybe should
become a FAQ, but in any case it was asked a few weeks
ago and I posted sample code with the idiomatic approach,
as I recall. A Google Groups search with your keywords
and my name should find it pretty quick.

-Peter

Peter Hansen wrote:

Ken Godee wrote:

[detailed version of the question in the subject line]

This question is asked often enough that it maybe should
become a FAQ, but in any case it was asked a few weeks
ago and I posted sample code with the idiomatic approach,
as I recall. A Google Groups search with your keywords
and my name should find it pretty quick.

Oops, doesn't look like I'm remembering the right thread,
since I didn't post in it. Anand Pillai, however, did
post sample code:

http://groups.google.ca/groups?hl=en...9006da0&rnum=1

-Peter

Peter Hansen wrote:

Peter Hansen wrote:

Ken Godee wrote:

[detailed version of the question in the subject line]

This question is asked often enough that it maybe should
become a FAQ, but in any case it was asked a few weeks
ago and I posted sample code with the idiomatic approach,
as I recall. A Google Groups search with your keywords
and my name should find it pretty quick.

Oops, doesn't look like I'm remembering the right thread,
since I didn't post in it. Anand Pillai, however, did
post sample code:

Actually, my memory is better than my search skills today:

http://groups.google.ca/groups?selm=...40powergate.ca

Christopher T King wrote:

On Wed, 14 Jul 2004, Jeff Shannon wrote:

Christopher T King wrote:

Just don't forget to .join() on the thread in module1 before accessing the
container object! (Assuming you're using threading) I just wish Python's
.join() could return values, like pthread_join can.

This may work if the worker thread will perform a relatively short task
and then die *before* you access the result. But lists and dictionaries
are not thread-safe -- if they are potentially accessed by multiple
threads concurrently, then the behavior will be unpredictable.

That's where the .join() comes in handy (it blocks until the thread dies)
;)

True, but quite frankly, I don't see much value in starting a new thread
if you're only going to be sitting around waiting for that thread to
finish. At that point, where's the gain over simply calling a
function? This may be a matter of style, but I see two major uses for
worker threads. In one case, you're doing a little bit of work here, a
little bit of work there, a little bit here again... back and forth.
The other is the case where you have a long-running task, but want your
user interface to remain responsive while that task running. (For
instance, most GUI frameworks will have problems if their event queues
aren't serviced regularly, and a lengthy calculation can prevent that
unless you put it in another thread.) This would also include the case
of a service/daemon which must remain responsive to new requests, so it
fires up a separate thread to handle each incoming request. In the
first case, you need to synchronize execution between the threads at
multiple points; in the second, the threads each go about their own
thing until the worker is finished, at which point it must notify the
main thread. In neither case is it practical to join() the worker
thread, because the whole point is that the main thread can't just sit
and do nothing.

If you *are* going to join(), then the advantages of concurrent
execution get thrown away while sitting at that join(). I see no
practical advantage of this:

workerthread = threading.Threa d(target=somefu nc)
workerthread.ru n()
somestuff()
someotherstuff( )
workerthread.jo in()

vs this:

somestuff()
someotherstuff( )
somefunc()

The only way you could possibly get a speed advantage from using a
thread is on a multiprocessor machine, and I don't believe that Python
currently makes use of multiple processors anyhow (though I think that
it's possible for C extension modules to do so). The flow-control
advantage of using a thread, that your program flow doesn't have to wait
for the task to finish, is lost when you are waiting to join(). All
you're left with is the added complexity. Now, if the worker thread
needs to finish some subtask before, say, someotherstuff( ) is called,
then you've got a potential for flow-control advantage... but you also
need synchronization that's a lot more advanced than just waiting for
the thread to die. If you don't need thread-safe communication between
the threads, then you don't need threads, at least IMO.

Jeff Shannon
Technician/Programmer
Credit International

>>ie. store value in memory from module3 and be able to

read it from module1.

The closest I can think of is to pass a queue reference
and stor values from module3 in the queue and then read them
back while in module1.

Actually, any kind of container object would do: you could pass a list,
dictionary, or class to the thread, and the thread could store its results
in it.

This was my orginal problem, being the middle man (module2) that spins
the thread(calling function in module3) has returned to module1 before
the thread result was available. So I couldn't figure a way to get the
result from module3 back to a global var in module1, since middle man
was gone.

A hackish method would be to pass the dictionary returned by locals() in
module1 to the thread. This way the thread could set a value in module1
directly.
Hhmmm, interesting, didn't know you could do that. Might have to try it.
Just don't forget to .join() on the thread in module1 before accessing the
container object! (Assuming you're using threading) I just wish Python's
.join() could return values, like pthread_join can.

Using thread here.
But I really have nothing against using queue, I've used queues before
but more for a kind of stream of data and was thinking I was missing
something because I only want to pass back a single value, so Queue(1)
would probally be alright.

Thanks for all the responses.

Ken