Hi...
I'm doing a program that analise a web page for a string, if found it
register that page, and if it found any links on that page, it follows the
link and searsh it for the string, and so on.
The problem is that I want to delimit this process to only 5 pages to be
processed at a time. I think of using a queue, and when the one of the 5
treads end processing the page, pick another link on the queue and start
processing again.
But how can I do that??? I need some mechanism that tells me when a thread
finished so I can start another...
Do you all have any idea that can help me???
[]s...
4  1548 
Well, your threads could check the queue when they are finished doing an
item, and if there is still data, process it.
Or you could write your own full threadpool.
However, why you want to limit to five threads? Why don't you use the
default System.Threading.ThreadPool?
-mike
MVP
"Ricardo" <dr*******@terra.com.br> wrote in message
news:OQ**************@TK2MSFTNGP10.phx.gbl... Hi...
I'm doing a program that analise a web page for a string, if found it
register that page, and if it found any links on that page, it follows the
link and searsh it for the string, and so on.
The problem is that I want to delimit this process to only 5 pages to be
processed at a time. I think of using a queue, and when the one of the 5
treads end processing the page, pick another link on the queue and start
processing again.
But how can I do that??? I need some mechanism that tells me when a thread
finished so I can start another...
Do you all have any idea that can help me???
[]s...
how this threadpool works??
"Michael Giagnocavo [MVP]" <mg*******@Atrevido.net> wrote in message
news:el**************@TK2MSFTNGP10.phx.gbl... Well, your threads could check the queue when they are finished doing an
item, and if there is still data, process it.
Or you could write your own full threadpool.
However, why you want to limit to five threads? Why don't you use the
default System.Threading.ThreadPool?
-mike
MVP
"Ricardo" <dr*******@terra.com.br> wrote in message
news:OQ**************@TK2MSFTNGP10.phx.gbl... Hi...
I'm doing a program that analise a web page for a string, if found it
register that page, and if it found any links on that page, it follows
the link and searsh it for the string, and so on.
The problem is that I want to delimit this process to only 5 pages to be
processed at a time. I think of using a queue, and when the one of the 5
treads end processing the page, pick another link on the queue and start
processing again.
But how can I do that??? I need some mechanism that tells me when a
thread finished so I can start another...
Do you all have any idea that can help me???
[]s...
The ThreadPool works by maintaining it's own internal queue of work items
that need to be completed. You can add as many items as you would like and
they will be processed as threads become available for them. MS has done
loads of performance testing on the optimum number of threads to run in the
default pool, and so the default pool should work fine for any general solution
thread based computing problems (such as the one you proposed).
--
Justin Rogers
DigiTec Web Consultants, LLC.
Blog: http://weblogs.asp.net/justin_rogers
"Ricardo" <dr*******@terra.com.br> wrote in message
news:OJ**************@TK2MSFTNGP11.phx.gbl... how this threadpool works??
"Michael Giagnocavo [MVP]" <mg*******@Atrevido.net> wrote in message
news:el**************@TK2MSFTNGP10.phx.gbl... Well, your threads could check the queue when they are finished doing an
item, and if there is still data, process it.
Or you could write your own full threadpool.
However, why you want to limit to five threads? Why don't you use the
default System.Threading.ThreadPool?
-mike
MVP
"Ricardo" <dr*******@terra.com.br> wrote in message
news:OQ**************@TK2MSFTNGP10.phx.gbl... Hi...
I'm doing a program that analise a web page for a string, if found it
register that page, and if it found any links on that page, it follows the link and searsh it for the string, and so on.
The problem is that I want to delimit this process to only 5 pages to be
processed at a time. I think of using a queue, and when the one of the 5
treads end processing the page, pick another link on the queue and start
processing again.
But how can I do that??? I need some mechanism that tells me when a thread finished so I can start another...
Do you all have any idea that can help me???
[]s...
Here is one way...
1) create a class that contains a thread. The class may have a public
..Start() method.
2) the thread start calls a private method that is your loop that reads a
shared queue. The queue is sync'd.
3) create 5 instances of this class and add to a collection (i.e. array,
array list, etc.) This is in your main control object.
4) Enum the list and start all threads.
5) You probably have another thread that is your producer to put items into
the queue. This may also be your control object.
5) put 5 stop objects into the queue when you want them to shutdown.
Cheers.
--
William Stacey, MVP
"Ricardo" <dr*******@terra.com.br> wrote in message
news:OQ**************@TK2MSFTNGP10.phx.gbl... Hi...
I'm doing a program that analise a web page for a string, if found it
register that page, and if it found any links on that page, it follows the
link and searsh it for the string, and so on.
The problem is that I want to delimit this process to only 5 pages to be
processed at a time. I think of using a queue, and when the one of the 5
treads end processing the page, pick another link on the queue and start
processing again.
But how can I do that??? I need some mechanism that tells me when a thread
finished so I can start another...
Do you all have any idea that can help me???
[]s...
This thread has been closed and replies have been disabled. Please start a new discussion. Similar topics
by: AlanR |
last post by:
Hi,
I have to implement(or possibly acquire) a queue that manages
priorities. Low priority elements cannot get lost in the queue
indefinitely if there is a sustained, constant flow of high...
|
by: Anand Pillai |
last post by:
The standard Python Queue module, allows to generate queues that
have no size limit, by passing the size argument as <= 0.
q = Queue(0)
In a multithreaded application, these queues could be...
|
by: Aki Niimura |
last post by:
Hello everyone.
I'm trying to control a program from a Python program using IPC.
Although using a socket is a common choice for such applications, I
would like
to use SysV message queues...
|
by: badtemper |
last post by:
Is there any way to use perl or any other programming language to
perform a "qchk -A" from a web page and see the listing?
My problem is that I manage the network for a medical facility that is...
|
by: saleem |
last post by:
Dear friends,
I am working on the problem which deals with the data management
of requests and how should we match the responses for the requests to
compare the correct Response.
my problem...
|
by: aakil |
last post by:
Need help in trying to determine the cause. Everything will work fine
sometimes minutes sometime days and then one start to get "Message Queuing
has not been installed on this computer" under...
|
by: Safalra |
last post by:
A long time ago when I used a browser that didn't support the shift or
unshift methods of the array object, I came up with an implementation of
queues that guaranteed amortised constant time...
|
by: brianbass |
last post by:
I am having a perplexing issue and am wondering if anyone can see the
flaw in my setup.
I have 3 windows services and 3 message queues. Each service resides
in the same directory and they all...
|
by: ericstein81 |
last post by:
I am trying to write a program that uses three queues for inputs of numbers. The problem i am having is I am not quite sure how to designate the seperate queues. can they be named as queue(a),...
|
by: DCC700 |
last post by:
Here are a couple of errors I have found in the event log
system.typeinitialization NIL
system.messaging.messagequeue NIL
These occur when trying to send a message through ms queues to a...
|
by: taylorcarr |
last post by:
A Canon printer is a smart device known for being advanced, efficient, and reliable. It is designed for home, office, and hybrid workspace use and can also be used for a variety of purposes. However,...
|
by: Charles Arthur |
last post by:
How do i turn on java script on a villaon, callus and itel keypad mobile phone
|
by: ryjfgjl |
last post by:
If we have dozens or hundreds of excel to import into the database, if we use the excel import function provided by database editors such as navicat, it will be extremely tedious and time-consuming...
|
by: BarryA |
last post by:
What are the essential steps and strategies outlined in the Data Structures and Algorithms (DSA) roadmap for aspiring data scientists? How can individuals effectively utilize this roadmap to progress...
|
by: nemocccc |
last post by:
hello, everyone, I want to develop a software for my android phone for daily needs, any suggestions?
|
by: marktang |
last post by:
ONU (Optical Network Unit) is one of the key components for providing high-speed Internet services. Its primary function is to act as an endpoint device located at the user's premises. However,...
|
by: Hystou |
last post by:
Most computers default to English, but sometimes we require a different language, especially when relocating. Forgot to request a specific language before your computer shipped? No problem! You can...
|
by: Oralloy |
last post by:
Hello folks,
I am unable to find appropriate documentation on the type promotion of bit-fields when using the generalised comparison operator "<=>".
The problem is that using the GNU compilers,...
|
by: jinu1996 |
last post by:
In today's digital age, having a compelling online presence is paramount for businesses aiming to thrive in a competitive landscape. At the heart of this digital strategy lies an intricately woven...
| |
