473,379 Members | 1,530 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,379 software developers and data experts.

Persistent Threads & Synchronisation

I appear to be having some problems with the isAlive() method of
detecting if a thread is alive/active/running or not. I'd be grateful
for any advice.

I have a visualisation program (which uses PyGame Extended [1]) that
presents content to the user and is meant to download the next batch of
content whilst the current one is being displayed. To achieve this, I
created a thread to perform the downloading in the background (a class
based on threading.Thread).

They way I want it to work is this: The downloading thread, when
spawned, stays alive for the duration of the program. Occasionally the
main program will call a function in it to download the data and save it
as files on disk. Then, these files are loaded by the main thread.
When this has happened, the main thread calls another function in the
download thread to delete the temporary files. The same thing happens
next time a download is needed, when the user is looking at some other
content.

My problem is this: the downloading thread only seems to execute code
in a separate thread as long as the run() function (called by the
start() function) is running. This is as per the documentation, of
course. I am performing the download in the run() function, but the
file cleanup is still done with a separate call. This actually does
work, after the download is over, and run() has terminated, but I
believe it isn't happening in a separate thread anymore (as previously I
made a mistake and called run() directly, instead of start() and it
blocked the main program).

I have been using join() to wait until the download is complete at the
point in the main program where it is absolutely necessary for the
download to have finished. I'm trying to use the following code to
restart the downloading thread when I next need to use it:

if not self.preview_thread.isAlive():
self.preview_thread.start()

Unfortunately, isAlive() sometimes returns False when the thread is
actually still running. This means that my code crashes out, on the
assertion in threading.Thread that self.__started is False. The
documentation [2] explains that the issue of threads being ``alive''
has ``intentionally been left vague'', which doesn't inspire confidence
:-S.

To work around this problem, I was thinking of doing this:

class PreviewThread(threading.Thread):
. . .
def run(self):
self.download_data()
while not self.temp_files_cleaned:
pass
return
. . .
def cleanup(self):
. . .
self.temp_files_cleaned = True

This should allow the thread to remain alive until after the download
*and* cleanup have been completed, so that I can be sure it's safe to
restart it when I next need to download more data.

My ultimate question: is this the right way to do things? I don't like
the idea of making a loop such as that in the proposed run() function
above, it seems inefficient. I am also concerned that it won't let me
call other functions in the thread, perhaps.

I did wonder about using a Condition, but that seems to be more suited
for synchronising between threads, which isn't really the issue here
(and thus it seems like overkill for solving the problem of replacing
that loop with something more efficient, though could be a possibility,
I suppose).

It seems I'm missing something big and possibly obvious about how to
make threads persistent. I would like to know ``the right way'' to do
it and would really appreciate some advice on the subject.

Many thanks in advance!
[1] http://codereactor.net/projects/pygext/
[2] http://docs.python.org/lib/thread-objects.html
--
Matthew Tylee Atkinson <ma*****@agrip.org.uk>
Nov 26 '06 #1
2 2191
On Sun, 26 Nov 2006 19:38:20 +0000, Dennis Lee Bieber wrote:
Methods defined in a thread class but called from outside the
running thread run in the environment of the caller, not as part of the
thread (and if the method manipulates state you may run into conflicts).
Thanks very much; I'd arrived at this conclusion, but didn't know how to fix the
situation.
You CAN NOT restart threads... You have to recreate the whole thread
object. .start() can only be called ONCE on any thread object.
That clears a lot up.
clean(), next(), and quit() will be called by the main thread as
needed. The "preview thread" does nothing except wait for a command on
the queue, then performs the operation requested. The main thread will
have to ensure it calls the next() and clean() methods in proper
sequence.
Excellent; thanks for the example.
(How the main thread is informed of completion is something I
don't know from your post).
Currently, I am using locks (as in threading.Lock()).
If you /need/ to have overlapped downloads and cleanups, you will
need TWO threads, and much more complex interlocking.
I don't, but thanks for the advice.

best regards,
--
Matthew Tylee Atkinson <ma*****@agrip.org.uk>
Nov 27 '06 #2
"Matthew Tylee Atkinson" <ma*****@agrip.org.ukwrote in message
news:ek**********@news.freedom2surf.net...
>I appear to be having some problems with the isAlive() method of
detecting if a thread is alive/active/running or not. I'd be grateful
for any advice.
Your comments about restartable threads got me thinking about generators.
While this does not strictly do threading, this little example uses a list
of generators, which save their state and pick up where they left off.
(Also, look into simpy, which has a similar concept, but much more
infrastructure support).

-- Paul
class Processor(object):
def __init__(self, id_):
self.id = id_
self.finished = False

def __str__(self):
return "Processor: %s" % self.id

def run(self):
def runImpl_(self):
runGen = self.runImpl()
while not self.finished:
try:
yield self.id,runGen.next()
except StopIteration:
self.finished = True
return runImpl_(self)

def runImpl(self):
times = 0
while times < self.id:
times += 1
yield times
import random

class RandomProcessor(Processor):
# just implement runImpl in subclass
def runImpl(self):
times = 0
while times < self.id:
times += 1
yield random.random()

def main():
# create list of processors
procList =[ (random.choice([True,False]) and
Processor(i) or RandomProcessor(i))
for i in range(10)]
procs = [ (p,p.run()) for p in procList ]

# context switch loop
while procs:

# cycle through all processors
for p in procs:
try:
ret = p[1].next()
print ret
except StopIteration:
pass

# remove any processors that have finished
procs = [ p for p in procs if not p[0].finished ]

main()

Nov 27 '06 #3

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

2
by: grahamo | last post by:
Hi, I realise that c++ knows nothing about threads however my question is related to an (excellent) article I was reading about threads and C++. For all intents and purposes we can forget the...
0
by: Gomaw Beoyr | last post by:
Hello The book "C# Black Book" chapter about Threads says that a "background thread" cannot communicate directly with a visual element, e.g. a label, and thus has to use the BeginInvoke method,...
2
by: Alex | last post by:
Dear colleagues I am programming an application which starts a lot of threads for some calculations. The problem that I have is that when I start up to 100 threads then everything works fine....
2
by: Fekkai | last post by:
I have a question about threads: In the TCPClient help page I find the following explanation "Thread Safety Any public static (Shared in Visual Basic) members of this type are safe for...
2
by: Matthew Tylee Atkinson | last post by:
I appear to be having some problems with the isAlive() method of detecting if a thread is alive/active/running or not. I'd be grateful for any advice. I have a visualisation program (which uses...
41
by: Carl J. Van Arsdall | last post by:
Hey everyone, I have a question about python threads. Before anyone goes further, this is not a debate about threads vs. processes, just a question. With that, are python threads reliable? Or...
7
by: =?Utf-8?B?Q2hyaXM=?= | last post by:
I have a win app. this a form with a button. after pressing the button a new form is shown. when looading the form is being filled by data taken from a database. the problem is that it takes a few...
4
by: =?Utf-8?B?bG91aXM=?= | last post by:
Can someone please discuss "Synclock" or other methods to use for making a class 'thread safe". I need to have about 4 processes running (probably on backgroundworker threads) and all will need to...
167
by: darren | last post by:
Hi I have to write a multi-threaded program. I decided to take an OO approach to it. I had the idea to wrap up all of the thread functions in a mix-in class called Threadable. Then when an...
0
by: Faith0G | last post by:
I am starting a new it consulting business and it's been a while since I setup a new website. Is wordpress still the best web based software for hosting a 5 page website? The webpages will be...
0
isladogs
by: isladogs | last post by:
The next Access Europe User Group meeting will be on Wednesday 3 Apr 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM). In this session, we are pleased to welcome former...
0
by: ryjfgjl | last post by:
In our work, we often need to import Excel data into databases (such as MySQL, SQL Server, Oracle) for data analysis and processing. Usually, we use database tools like Navicat or the Excel import...
0
by: taylorcarr | last post by:
A Canon printer is a smart device known for being advanced, efficient, and reliable. It is designed for home, office, and hybrid workspace use and can also be used for a variety of purposes. However,...
0
by: ryjfgjl | last post by:
If we have dozens or hundreds of excel to import into the database, if we use the excel import function provided by database editors such as navicat, it will be extremely tedious and time-consuming...
0
by: ryjfgjl | last post by:
In our work, we often receive Excel tables with data in the same format. If we want to analyze these data, it can be difficult to analyze them because the data is spread across multiple Excel files...
0
by: emmanuelkatto | last post by:
Hi All, I am Emmanuel katto from Uganda. I want to ask what challenges you've faced while migrating a website to cloud. Please let me know. Thanks! Emmanuel
0
BarryA
by: BarryA | last post by:
What are the essential steps and strategies outlined in the Data Structures and Algorithms (DSA) roadmap for aspiring data scientists? How can individuals effectively utilize this roadmap to progress...
1
by: nemocccc | last post by:
hello, everyone, I want to develop a software for my android phone for daily needs, any suggestions?

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.