Hi all,
What's the best way to queue up and wait for number of threads to complete.
This problem was trivial in VC++ 6 but I'm finding it rather hard to solve
in VB.NET. My calculations run about 2 mins each (on a 2.8 Ghz Xeon and the
server range from 2 to 8 processors) (give or take about 15 secs). I have 8
sets of calculations to do (in another app I have 121) all the same calc
just different data. But the rest of the processing cannot continue until
ALL of the calculations are done.
so here's the algorithm I want to use (written from the C++ implentation
standpoint) {for ease of reading i have left out any/all discussion about
the queue ops used to schedule the runs which can change to what ever is
appropriate}
------------------------------------------------------------------
get user pref on number of concurrent threads...
for loop (to number of concurrent threads)
start a thread (with data)
add to wait list
while true
WaitForMultipleObjects(...threads...)
if more threads to process
launch thread with data
add to wait list
else if all threads finished
break
------------------------------------------------------------------
The problem I'm having is finding the .NET equivalent to
WaitForMultipleObjects. The threads might not complete in order or might
complete at the same time. I was toying with the idea of a delegate that
would basically have the functionality of the while loop, but seemed like
spaghetti coding to me.
Any ideas or examples you can think of would be greatly appreciated.
Thanks in advance,
Mike
----------------------------------------------------------
Mike Elledge 10 3015
<"MikeE" <mikeDOTelledgeATattbiDOTcom>> wrote: What's the best way to queue up and wait for number of threads to complete. This problem was trivial in VC++ 6 but I'm finding it rather hard to solve in VB.NET. My calculations run about 2 mins each (on a 2.8 Ghz Xeon and the server range from 2 to 8 processors) (give or take about 15 secs). I have 8 sets of calculations to do (in another app I have 121) all the same calc just different data. But the rest of the processing cannot continue until ALL of the calculations are done.
That's easy then - just call Join() on all the threads.
--
Jon Skeet - <sk***@pobox.com> http://www.pobox.com/~skeet
If replying to the group, please do not mail me too
Jon,
Doesn't Join() block until that particular thread is done (the same as
WaitForSingleObject)? If so, then that won't give me the proper scheduling
right? When any of the current threads end, I need to schedule the next in
the queue, and I can't schedule (i.e., start()) the next one UNTIL one of
the MAX # finishes.
Please correct me if i'm wrong about Join().
Thanks,
ME
"Jon Skeet [C# MVP]" <sk***@pobox.com> wrote in message
news:MP************************@msnews.microsoft.c om... <"MikeE" <mikeDOTelledgeATattbiDOTcom>> wrote: What's the best way to queue up and wait for number of threads to complete. This problem was trivial in VC++ 6 but I'm finding it rather hard to solve in VB.NET. My calculations run about 2 mins each (on a 2.8 Ghz Xeon and the server range from 2 to 8 processors) (give or take about 15 secs). I have 8 sets of calculations to do (in another app I have 121) all the same calc just different data. But the rest of the processing cannot continue until ALL of the calculations are done.
That's easy then - just call Join() on all the threads.
-- Jon Skeet - <sk***@pobox.com> http://www.pobox.com/~skeet If replying to the group, please do not mail me too
<"MikeE" <mikeDOTelledgeATattbiDOTcom>> wrote: Doesn't Join() block until that particular thread is done (the same as WaitForSingleObject)?
Yes.
If so, then that won't give me the proper scheduling right?
Nope.
When any of the current threads end, I need to schedule the next in the queue, and I can't schedule (i.e., start()) the next one UNTIL one of the MAX # finishes.
Please correct me if i'm wrong about Join().
Maybe I've misunderstood you. I thought you had created a bunch of
threads, started them running, and then wanted to know when they'd all
finished. In that case, calling Join() on each of them in turn does
exactly what you want.
If that's not what you're doing, you'll need to give a bit more
information about what you're doing.
--
Jon Skeet - <sk***@pobox.com> http://www.pobox.com/~skeet
If replying to the group, please do not mail me too
Jon,
I'll try to restate then...
I have X number of tasks (unknown run time) that must be completed before
additional work is done. The USER specifies that the system can use N
threads to do the processing in parallel. How do we get this done in
VB.NET? The VC++ implentation is as follows...
1) Queue all work data (X items)
2) For i = 0; i < N; i++
2a) Get work data from queue (i.e., remove one item)
2b) Create Thread with work Data
2c) Start Thread
2d) Add Thread to watch list
3) while (true)
3a) WaitForMultipleObjects(watch list) <--- this will
block until ANY of the threads return
3b) remove finished thread from watch list
3c) If work data queue NOT empty
3c1) Get work data from queue (i.e., remove one item)
3c2) Create Thread with work Data
3c3) Start Thread
3c4) Add Thread to watch list
3d) else if all threads done (watch list empty)
3d1) break; <--
break the while loop when ALL data is processed and ALL threads are done
// else still processing threads...
4) additional work...
So another way to state my question is...
How do I block on multiple threads in VB.NET? Are WaitHandles my only
option?
Thanks,
"Jon Skeet [C# MVP]" <sk***@pobox.com> wrote in message
news:MP************************@msnews.microsoft.c om... <"MikeE" <mikeDOTelledgeATattbiDOTcom>> wrote: Doesn't Join() block until that particular thread is done (the same as WaitForSingleObject)?
Yes.
If so, then that won't give me the proper scheduling right?
Nope.
When any of the current threads end, I need to schedule the next in the queue, and I can't schedule (i.e., start()) the next one UNTIL one of the MAX # finishes.
Please correct me if i'm wrong about Join().
Maybe I've misunderstood you. I thought you had created a bunch of threads, started them running, and then wanted to know when they'd all finished. In that case, calling Join() on each of them in turn does exactly what you want.
If that's not what you're doing, you'll need to give a bit more information about what you're doing.
-- Jon Skeet - <sk***@pobox.com> http://www.pobox.com/~skeet If replying to the group, please do not mail me too
Mike,
It sounds like you are wanting to write your own thread pool.
Any reason you are not using System.Threading.ThreadPool itself?
If ThreadPool was not quite what I needed, instead of creating a distinct
thread for each request I would simply create N threads. I would place all
the requests into a single System.Collections.Queue, each thread would read
a request from the queue, process it, then get another request from the
queue. Of course the Queue itself would need to be wrapped in thread safe
code such as via Queue.Synchronized. You may need to implement something to
let each thread know when it should stop processing requests...
Hope this helps
Jay
"MikeE" <mikeDOTelledgeATattbiDOTcom> wrote in message
news:%2****************@TK2MSFTNGP12.phx.gbl... Jon,
I'll try to restate then...
I have X number of tasks (unknown run time) that must be completed before additional work is done. The USER specifies that the system can use N threads to do the processing in parallel. How do we get this done in VB.NET? The VC++ implentation is as follows...
1) Queue all work data (X items) 2) For i = 0; i < N; i++ 2a) Get work data from queue (i.e., remove one item) 2b) Create Thread with work Data 2c) Start Thread 2d) Add Thread to watch list 3) while (true) 3a) WaitForMultipleObjects(watch list) <--- this will block until ANY of the threads return 3b) remove finished thread from watch list 3c) If work data queue NOT empty 3c1) Get work data from queue (i.e., remove one item) 3c2) Create Thread with work Data 3c3) Start Thread 3c4) Add Thread to watch list 3d) else if all threads done (watch list empty) 3d1) break; <-- break the while loop when ALL data is processed and ALL threads are done // else still processing threads... 4) additional work...
So another way to state my question is... How do I block on multiple threads in VB.NET? Are WaitHandles my only option?
Thanks, "Jon Skeet [C# MVP]" <sk***@pobox.com> wrote in message news:MP************************@msnews.microsoft.c om... <"MikeE" <mikeDOTelledgeATattbiDOTcom>> wrote: Doesn't Join() block until that particular thread is done (the same as WaitForSingleObject)?
Yes.
If so, then that won't give me the proper scheduling right?
Nope.
When any of the current threads end, I need to schedule the next in the queue, and I can't schedule (i.e., start()) the next one UNTIL one of the MAX # finishes.
Please correct me if i'm wrong about Join().
Maybe I've misunderstood you. I thought you had created a bunch of threads, started them running, and then wanted to know when they'd all finished. In that case, calling Join() on each of them in turn does exactly what you want.
If that's not what you're doing, you'll need to give a bit more information about what you're doing.
-- Jon Skeet - <sk***@pobox.com> http://www.pobox.com/~skeet If replying to the group, please do not mail me too
<"MikeE" <mikeDOTelledgeATattbiDOTcom>> wrote: I'll try to restate then...
I have X number of tasks (unknown run time) that must be completed before additional work is done. The USER specifies that the system can use N threads to do the processing in parallel. How do we get this done in VB.NET? The VC++ implentation is as follows...
That sounds exactly like a thread pool. If the system thread pool
doesn't work for you (and there are various reasons why you might not
want to use it) you may want to look at the implementation I've got at http://www.pobox.com/~skeet/csharp/miscutil
It's not been tested much, but you're welcome to it...
--
Jon Skeet - <sk***@pobox.com> http://www.pobox.com/~skeet
If replying to the group, please do not mail me too
John/Jay,
Yes! what I want is so close to the threadpool that it is irritating. The
ONE MAJOR missing part for a thread pool is the MAX number of threads. See,
this processing is some rather lengthy numerical calculations, VERY CPU
intensive. We do not run it on any machine that does not have at least 2
processors (or one hyper threaded one, and then just for testing).
Otherwise it locks the system up. But we have some servers that have up to
8 CPUs in them. So we need the number of concurrent threads to be variable.
If we just lanched them all at once, we would QUICKLY run out of cpu
bandwith for the calcs and the system. (back to locked systems). we just
have to be able to control the number of "active" threads at one time.
What would your thoughts/feelings be towards using "WaitForMultipleObjects".
We of course could import it (which ive done) but having trouble setting up
the array of Thread handles to be watched. Any ideas? (just a reminder...i
kinda need this in vb.net)
On another note.. since i do have to get something working soon.. i've
started down the spaghetti code method, where the threads fire events to
which i have a re-enterable function that can access the remaining queue,
and when finally empty and done fires another event that is handled by a
delegate that does "the rest of the stuff" Unfortunatly this is looking
REALLY ugly, so i'm trying to come up with a class to wrap the thread and
that contains a wait handle which is montitored.. but still trying to flush
this out.
Again, what gets me, is that this takes less than 50 lines of code in VC++,
with error checking, and was rock solid. I'm really having a tough time
coming to grips with the fact that we can't do it EVEN EASIER in .NET like
so many of our other upgrades.
And john, thanks for the link.. i'll take a look at miscutil tomorrow!
Thanks again for your time on this,
ME
"Jon Skeet [C# MVP]" <sk***@pobox.com> wrote in message
news:MP************************@msnews.microsoft.c om... <"MikeE" <mikeDOTelledgeATattbiDOTcom>> wrote: I'll try to restate then...
I have X number of tasks (unknown run time) that must be completed before additional work is done. The USER specifies that the system can use N threads to do the processing in parallel. How do we get this done in VB.NET? The VC++ implentation is as follows...
That sounds exactly like a thread pool. If the system thread pool doesn't work for you (and there are various reasons why you might not want to use it) you may want to look at the implementation I've got at http://www.pobox.com/~skeet/csharp/miscutil
It's not been tested much, but you're welcome to it...
-- Jon Skeet - <sk***@pobox.com> http://www.pobox.com/~skeet If replying to the group, please do not mail me too
<"Mike E" <mikeDOTelledgeATattbiDOTcom>> wrote: Yes! what I want is so close to the threadpool that it is irritating. The ONE MAJOR missing part for a thread pool is the MAX number of threads. See, this processing is some rather lengthy numerical calculations, VERY CPU intensive. We do not run it on any machine that does not have at least 2 processors (or one hyper threaded one, and then just for testing). Otherwise it locks the system up. But we have some servers that have up to 8 CPUs in them. So we need the number of concurrent threads to be variable. If we just lanched them all at once, we would QUICKLY run out of cpu bandwith for the calcs and the system. (back to locked systems). we just have to be able to control the number of "active" threads at one time.
What would your thoughts/feelings be towards using "WaitForMultipleObjects". We of course could import it (which ive done) but having trouble setting up the array of Thread handles to be watched. Any ideas? (just a reminder...i kinda need this in vb.net)
Well, you don't really want to wait for other threads to actually
finish, because you can't reuse them. Doing a thread pool of *some*
description (which could just be several threads with a
producer/consumer queue - see http://www.pobox.com/~skeet/csharp/t...eadlocks.shtml for some
sample code) is a much better answer here.
Alternatively, using Auto/ManualResetEvents, one per thread, would
allow you to wait for any of the threads to finish without being too
hard. Or just have a single monitor which the main thread waits on and
which other threads pulse.
On another note.. since i do have to get something working soon.. i've started down the spaghetti code method, where the threads fire events to which i have a re-enterable function that can access the remaining queue, and when finally empty and done fires another event that is handled by a delegate that does "the rest of the stuff" Unfortunatly this is looking REALLY ugly, so i'm trying to come up with a class to wrap the thread and that contains a wait handle which is montitored.. but still trying to flush this out.
Again, what gets me, is that this takes less than 50 lines of code in VC++, with error checking, and was rock solid. I'm really having a tough time coming to grips with the fact that we can't do it EVEN EASIER in .NET like so many of our other upgrades.
You may well find it's easier in .NET, once you change the approach
slightly. There are lots of things which are hard to do if you try
using the "old" approach, but for which there are new approaches which
make things easier.
--
Jon Skeet - <sk***@pobox.com> http://www.pobox.com/~skeet
If replying to the group, please do not mail me too
Jon,
Thanks, I'll look into the single monitor option... seems promising.
One clarification though.. I HAVE to wait for all the threads to finish, not
because I want to reuse the thread, but because I need the "ANSWER". A math
problem is not finished until all the parts are there. In our program the
final result can NOT be calculated until ALL the threads have returned.
Thanks again,
ME
"Jon Skeet [C# MVP]" <sk***@pobox.com> wrote in message
news:MP************************@msnews.microsoft.c om... <"Mike E" <mikeDOTelledgeATattbiDOTcom>> wrote: Yes! what I want is so close to the threadpool that it is irritating. The ONE MAJOR missing part for a thread pool is the MAX number of threads. See, this processing is some rather lengthy numerical calculations, VERY CPU intensive. We do not run it on any machine that does not have at least 2 processors (or one hyper threaded one, and then just for testing). Otherwise it locks the system up. But we have some servers that have up to 8 CPUs in them. So we need the number of concurrent threads to be variable. If we just lanched them all at once, we would QUICKLY run out of cpu bandwith for the calcs and the system. (back to locked systems). we just have to be able to control the number of "active" threads at one time.
What would your thoughts/feelings be towards using "WaitForMultipleObjects". We of course could import it (which ive done) but having trouble setting up the array of Thread handles to be watched. Any ideas? (just a reminder...i kinda need this in vb.net)
Well, you don't really want to wait for other threads to actually finish, because you can't reuse them. Doing a thread pool of *some* description (which could just be several threads with a producer/consumer queue - see http://www.pobox.com/~skeet/csharp/t...eadlocks.shtml for some sample code) is a much better answer here.
Alternatively, using Auto/ManualResetEvents, one per thread, would allow you to wait for any of the threads to finish without being too hard. Or just have a single monitor which the main thread waits on and which other threads pulse.
On another note.. since i do have to get something working soon.. i've started down the spaghetti code method, where the threads fire events to which i have a re-enterable function that can access the remaining queue, and when finally empty and done fires another event that is handled by a delegate that does "the rest of the stuff" Unfortunatly this is looking REALLY ugly, so i'm trying to come up with a class to wrap the thread and that contains a wait handle which is montitored.. but still trying to flush this out.
Again, what gets me, is that this takes less than 50 lines of code in VC++, with error checking, and was rock solid. I'm really having a tough time coming to grips with the fact that we can't do it EVEN EASIER in .NET like so many of our other upgrades.
You may well find it's easier in .NET, once you change the approach slightly. There are lots of things which are hard to do if you try using the "old" approach, but for which there are new approaches which make things easier.
-- Jon Skeet - <sk***@pobox.com> http://www.pobox.com/~skeet If replying to the group, please do not mail me too
MikeE,
Why not use two queues?
One queue for the questions (requests) to the N threads in the pool and a
second queue for the answers (to the main thread). The main thread can
simply wait for answers on the second queue, once it has all the answers it
can figure the final result.
Hope this helps
Jay
"MikeE" <mikeDOTelledgeATattbiDOTcom> wrote in message
news:e4**************@TK2MSFTNGP15.phx.gbl... Jon,
Thanks, I'll look into the single monitor option... seems promising.
One clarification though.. I HAVE to wait for all the threads to finish, not because I want to reuse the thread, but because I need the "ANSWER". A math problem is not finished until all the parts are there. In our program the final result can NOT be calculated until ALL the threads have returned.
Thanks again, ME
"Jon Skeet [C# MVP]" <sk***@pobox.com> wrote in message news:MP************************@msnews.microsoft.c om... <"Mike E" <mikeDOTelledgeATattbiDOTcom>> wrote: Yes! what I want is so close to the threadpool that it is irritating. The ONE MAJOR missing part for a thread pool is the MAX number of threads. See, this processing is some rather lengthy numerical calculations, VERY CPU intensive. We do not run it on any machine that does not have at least 2 processors (or one hyper threaded one, and then just for testing). Otherwise it locks the system up. But we have some servers that have up to 8 CPUs in them. So we need the number of concurrent threads to be variable. If we just lanched them all at once, we would QUICKLY run out of cpu bandwith for the calcs and the system. (back to locked systems). we just have to be able to control the number of "active" threads at one time.
What would your thoughts/feelings be towards using "WaitForMultipleObjects". We of course could import it (which ive done) but having trouble setting up the array of Thread handles to be watched. Any ideas? (just a reminder...i kinda need this in vb.net)
Well, you don't really want to wait for other threads to actually finish, because you can't reuse them. Doing a thread pool of *some* description (which could just be several threads with a producer/consumer queue - see http://www.pobox.com/~skeet/csharp/t...eadlocks.shtml for some sample code) is a much better answer here.
Alternatively, using Auto/ManualResetEvents, one per thread, would allow you to wait for any of the threads to finish without being too hard. Or just have a single monitor which the main thread waits on and which other threads pulse.
On another note.. since i do have to get something working soon.. i've started down the spaghetti code method, where the threads fire events to which i have a re-enterable function that can access the remaining queue, and when finally empty and done fires another event that is handled by a delegate that does "the rest of the stuff" Unfortunatly this is looking REALLY ugly, so i'm trying to come up with a class to wrap the thread and that contains a wait handle which is montitored.. but still trying to flush this out.
Again, what gets me, is that this takes less than 50 lines of code in VC++, with error checking, and was rock solid. I'm really having a tough time coming to grips with the fact that we can't do it EVEN EASIER in .NET like so many of our other upgrades.
You may well find it's easier in .NET, once you change the approach slightly. There are lots of things which are hard to do if you try using the "old" approach, but for which there are new approaches which make things easier.
-- Jon Skeet - <sk***@pobox.com> http://www.pobox.com/~skeet If replying to the group, please do not mail me too
This thread has been closed and replies have been disabled. Please start a new discussion. Similar topics
by: bughunter |
last post by:
Hi,
Consider this code:
----
Monitor.Pulse(oLock);
Monitor.Exit(oLock);
----
If a thread was waiting on oLock then will the current thread
|
by: Douwe |
last post by:
I'm trying to write a piece of C code that has two threads: The first
thread waits for a socket connection. The second thread is in a
continues cycle waiting for an period of time (lets say 500 ms)...
|
by: mphanke |
last post by:
Hi,
I would like to implement a windows service which forks a configurable
number of threads which all siten to the same port and reply to requests
of different clients.
How would I implement...
|
by: Elhurzen |
last post by:
X-No-Archive: Yes
>From what I understand, if multiple threads are waiting on a
ManualResetEvent after calling WaitOne(), only one thread is guaranteed
to be signaled when Set() is called on that...
|
by: Tumurbaatar S. |
last post by:
ASP.NET QuickStart Tutorial says that:
....
ASP.NET maintains a pool of HttpApplication instances over the course of a
Web application's lifetime. ASP.NET automatically assigns one of these...
|
by: MikeE |
last post by:
Hi all,
What's the best way to queue up and wait for number of threads to complete.
This problem was trivial in VC++ 6 but I'm finding it rather hard to solve
in VB.NET. My calculations run...
|
by: Bruce Wood |
last post by:
Maybe it's just late in my day, but I'm reading Jon's article on
threading, in particular how to use Monitor.Wait() and Monitor.Pulse(),
and there's something that's not sinking in. The code in...
|
by: jankhana |
last post by:
Hi all,
I'm having an application in that i use Sql Compact 3.5 with VS2008. I'm running multiple threads in my application which contacts the compact database and accesses the row. It...
|
by: Charles Arthur |
last post by:
How do i turn on java script on a villaon, callus and itel keypad mobile phone
|
by: emmanuelkatto |
last post by:
Hi All, I am Emmanuel katto from Uganda. I want to ask what challenges you've faced while migrating a website to cloud.
Please let me know.
Thanks!
Emmanuel
|
by: BarryA |
last post by:
What are the essential steps and strategies outlined in the Data Structures and Algorithms (DSA) roadmap for aspiring data scientists? How can individuals effectively utilize this roadmap to progress...
|
by: nemocccc |
last post by:
hello, everyone, I want to develop a software for my android phone for daily needs, any suggestions?
|
by: Hystou |
last post by:
There are some requirements for setting up RAID:
1. The motherboard and BIOS support RAID configuration.
2. The motherboard has 2 or more available SATA protocol SSD/HDD slots (including MSATA, M.2...
|
by: Hystou |
last post by:
Most computers default to English, but sometimes we require a different language, especially when relocating. Forgot to request a specific language before your computer shipped? No problem! You can...
|
by: jinu1996 |
last post by:
In today's digital age, having a compelling online presence is paramount for businesses aiming to thrive in a competitive landscape. At the heart of this digital strategy lies an intricately woven...
|
by: Hystou |
last post by:
Overview:
Windows 11 and 10 have less user interface control over operating system update behaviour than previous versions of Windows. In Windows 11 and 10, there is no way to turn off the Windows...
|
by: isladogs |
last post by:
The next Access Europe User Group meeting will be on Wednesday 1 May 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM).
In this session, we are pleased to welcome a new...
| |