473,769 Members | 6,831 Online
Bytes | Software Development & Data Engineering Community
+ Post

Home Posts Topics Members FAQ

possible to use Threadpool

Hi all

I have written a app that gets emails from a DB and checks if it's a
correct email (connects to the remote mailserver and checks the
response).
I would like to improve the performance with using multiple threads.
can I use a threadpool for this? and how do I have to do this.
For starting i have all emailaddresses in a Array.

thanks for your help.

daniel

Nov 11 '06 #1
11 2440
Hello Daniel,

DI have written a app that gets emails from a DB and checks if it's a
Dcorrect email (connects to the remote mailserver and checks the
Dresponse).
DI would like to improve the performance with using multiple threads.
Dcan I use a threadpool for this?

You could, but it can be not the very good solution for the 1.1 fw if u have
a big number of long runned tasks, because the number of TP's threads there
are limited by 25 thread, and in case if you have numbers of long-runned
operations u got the blocked threads which wait till one of 25 threads complete
its work.

In 2.0 u can set the desired number of threads, but take into account that
25 is the general optimal number of threads, and overcoming have negative
effect due to context switching

Dand how do I have to do this.
DFor starting i have all emailaddresses in a Array.

The samples of using ThreadPool class are into MSDN

---
WBR,
Michael Nemtsev :: blog: http://spaces.live.com/laflour

"At times one remains faithful to a cause only because its opponents do not
cease to be insipid." (c) Friedrich Nietzsche
Nov 11 '06 #2

Michael Nemtsev schrieb:
Hello Daniel,

DI have written a app that gets emails from a DB and checks if it's a
Dcorrect email (connects to the remote mailserver and checks the
Dresponse).
DI would like to improve the performance with using multiple threads.
Dcan I use a threadpool for this?

You could, but it can be not the very good solution for the 1.1 fw if u have
a big number of long runned tasks, because the number of TP's threads there
are limited by 25 thread, and in case if you have numbers of long-runned
operations u got the blocked threads which wait till one of 25 threads complete
its work.

In 2.0 u can set the desired number of threads, but take into account that
25 is the general optimal number of threads, and overcoming have negative
effect due to context switching

Dand how do I have to do this.
DFor starting i have all emailaddresses in a Array.

The samples of using ThreadPool class are into MSDN

---
WBR,
Michael Nemtsev :: blog: http://spaces.live.com/laflour

"At times one remains faithful to a cause only because its opponents do not
cease to be insipid." (c) Friedrich Nietzsche

Thanks for the answer. I use the 2.0 FW. So it should work fine.
As far as I could read in the docs, the Pool is using max. 25 threads
at the time. Is there a way to show in a label or so which thread is
working on with emailaddress? So I could have a view of the process.
I have to write back the not existing emailaddresses to the database.
when i do it from the thread, it will open the connection write it back
and close the connection. right? but this could produce a lot of work
for the db. is there a way to open the conecction ones and then write
back all addresses and then close the connection? or is this not so
important?

Nov 11 '06 #3
Hello Daniel,

DThanks for the answer. I use the 2.0 FW. So it should work fine.
DAs far as I could read in the docs, the Pool is using max. 25 threads
Dat the time.

Yep, but it's per process per processor, and your can change it.

DIs there a way to show in a label or so which thread is
Dworking on with emailaddress?

see ThreadPool.GetA vailableThreads method

So I could have a view of the process.
DI have to write back the not existing emailaddresses to the database.
Dwhen i do it from the thread, it will open the connection write it
Dback
Dand close the connection. right? but this could produce a lot of work
Dfor the db. is there a way to open the conecction ones and then write
Dback all addresses and then close the connection? or is this not so
Dimportant?

In MSSQL server your DB connections are pooled by defauilt (in others
DB they are not), so don't mind it

---
WBR,
Michael Nemtsev :: blog: http://spaces.live.com/laflour

"At times one remains faithful to a cause only because its opponents do not
cease to be insipid." (c) Friedrich Nietzsche
Nov 11 '06 #4
Daniel <da****@danielf .chwrote:
I have written a app that gets emails from a DB and checks if it's a
correct email (connects to the remote mailserver and checks the
response).
I would like to improve the performance with using multiple threads.
can I use a threadpool for this? and how do I have to do this.
For starting i have all emailaddresses in a Array.
I would suggest not using the thread pool for this - you could easily
swamp it, which would make other things which try to use it less
responsive.

I would suggest you use a producer/consumer queue, and then you can
"manually" create as many threads as you want to be consumers of the
queue. This would be a sort of threadpool in itself, *just* for this
purpose. You'd have a lot more control over things that way.

See http://www.pobox.com/~skeet/csharp/t...eadlocks.shtml (half
way down) for some sample producer/consumer code.

--
Jon Skeet - <sk***@pobox.co m>
http://www.pobox.com/~skeet Blog: http://www.msmvps.com/jon.skeet
If replying to the group, please do not mail me too
Nov 11 '06 #5
Jon schrieb:
Daniel <da****@danielf .chwrote:
I have written a app that gets emails from a DB and checks if it's a
correct email (connects to the remote mailserver and checks the
response).
I would like to improve the performance with using multiple threads.
can I use a threadpool for this? and how do I have to do this.
For starting i have all emailaddresses in a Array.

I would suggest not using the thread pool for this - you could easily
swamp it, which would make other things which try to use it less
responsive.

I would suggest you use a producer/consumer queue, and then you can
"manually" create as many threads as you want to be consumers of the
queue. This would be a sort of threadpool in itself, *just* for this
purpose. You'd have a lot more control over things that way.

See http://www.pobox.com/~skeet/csharp/t...eadlocks.shtml (half
way down) for some sample producer/consumer code.

--
Jon Skeet - <sk***@pobox.co m>
http://www.pobox.com/~skeet Blog: http://www.msmvps.com/jon.skeet
If replying to the group, please do not mail me too
Thanks for the answer.

I was now just playing around with this. Not sure if I understad it
right.
Here What I have:

using System;
using System.Collecti ons;
using System.Threadin g;

public class Test
{
static CheckEmail queue;
public static int rows = 100;

static void Main()
{
queue = new CheckEmail();
new Thread(new ThreadStart(Ema ilChecker)).Sta rt();
new Thread(new ThreadStart(Ema ilChecker)).Sta rt();
new Thread(new ThreadStart(Ema ilChecker)).Sta rt();
new Thread(new ThreadStart(Ema ilChecker)).Sta rt();
new Thread(new ThreadStart(Ema ilChecker)).Sta rt();

Random rng = new Random(0);
for (int i = 0; i < rows; i++)
{
Console.WriteLi ne("Producing {0}", i);
queue.Produce(" Daniel " + i.ToString());
}

while (queue.Count 0)
{
Console.WriteLi ne("There are {0} emails Waiting",
queue.Count);
Thread.Sleep(20 00);
}
//Here I would like to go ahead when all EMails are processed.
}

static void EmailChecker()
{
// Make sure we get a different random seed from the
// first thread
Random rng = new Random(2);
// We happen to know we've only got 10
// items to receive
for (int i = 0; i < rows; i++)
{
object o = queue.EmailChec k();
//Process Email
Console.WriteLi ne("\t\t\t\tCon suming {0}", o);
Thread.Sleep(rn g.Next(1000));
}
}
}

public class CheckEmail
{
readonly object listLock = new object();
Queue queue = new Queue();
private string Recipient;

public int Count
{
get { return queue.Count; }
}

public void Produce(object o)
{
lock (listLock)
{
queue.Enqueue(o );
Recipient = o.ToString();

// We always need to pulse, even if the queue wasn't
// empty before. Otherwise, if we add several items
// in quick succession, we may only pulse once, waking
// a single thread up, even if there are multiple threads
// waiting for items.
Monitor.Pulse(l istLock);
}
}

public object EmailCheck()
{
lock (listLock)
{
// If the queue is empty, wait for an item to be added
// Note that this is a while loop, as we may be pulsed
// but not wake up before another thread has come in and
// consumed the newly added object. In that case, we'll
// have to wait for another pulse.
while (queue.Count == 0)
{
// This releases listLock, only reacquiring it
// after being woken up by a call to Pulse
Monitor.Wait(li stLock);
}
return queue.Dequeue() ;
}
}
}

So I produce the queue with all emailaddresses (200'000) and then walk
trought with 5 Threads (in this exemple).
Then I wait till the que is empty. How do I know if the last Thread has
finished the work?

So I have to Write the emailaddresses in the Function EmailChecker()
back to the database right?
I also have to write the Logfile in the function EmailChecker().

How can I close the Queue to exit correctly?

I want to run this only if somone requests it by pressing the button.

Thanks for your help

Daniel

Nov 13 '06 #6
Daniel <da****@danielf .chwrote:

<snip>
So I produce the queue with all emailaddresses (200'000) and then walk
trought with 5 Threads (in this exemple).
Then I wait till the que is empty. How do I know if the last Thread has
finished the work?
One way would be to have a special value (eg null) which means "all
work is complete". At the end of adding all the "real" values to the
queue, add the same number of nulls as there are threads. Then change
the threading code to exit the loop when they see a null. Finally, from
the main loop, call Join on each of the worker threads in turn.
So I have to Write the emailaddresses in the Function EmailChecker()
back to the database right?
Presumably - it's not entirely obvious from your original description
what you need to write back.
I also have to write the Logfile in the function EmailChecker().
I'd suggest having a single thread which writes log entries, to make
sure you don't end up trying to write one log entry in the middle of
another one. This is sort of the reverse of the producer/consumer queue
you've got - there you'd have several producers and one consumer.
Alternatively, you could just synchronise writing out the log entries.
How can I close the Queue to exit correctly?
You could have a separate flag that is visible to all threads which
indicates that they should quit "early" even if there's more work to
do.
I want to run this only if somone requests it by pressing the button.
That's just a matter of responding to an appropriate button click
event.

--
Jon Skeet - <sk***@pobox.co m>
http://www.pobox.com/~skeet Blog: http://www.msmvps.com/jon.skeet
If replying to the group, please do not mail me too
Nov 13 '06 #7

Jon schrieb:
Daniel <da****@danielf .chwrote:

<snip>
So I produce the queue with all emailaddresses (200'000) and then walk
trought with 5 Threads (in this exemple).
Then I wait till the que is empty. How do I know if the last Thread has
finished the work?

One way would be to have a special value (eg null) which means "all
work is complete". At the end of adding all the "real" values to the
queue, add the same number of nulls as there are threads. Then change
the threading code to exit the loop when they see a null. Finally, from
the main loop, call Join on each of the worker threads in turn.
So I have to Write the emailaddresses in the Function EmailChecker()
back to the database right?

Presumably - it's not entirely obvious from your original description
what you need to write back.
I also have to write the Logfile in the function EmailChecker().

I'd suggest having a single thread which writes log entries, to make
sure you don't end up trying to write one log entry in the middle of
another one. This is sort of the reverse of the producer/consumer queue
you've got - there you'd have several producers and one consumer.
Alternatively, you could just synchronise writing out the log entries.
How can I close the Queue to exit correctly?

You could have a separate flag that is visible to all threads which
indicates that they should quit "early" even if there's more work to
do.
I want to run this only if somone requests it by pressing the button.

That's just a matter of responding to an appropriate button click
event.

--
Jon Skeet - <sk***@pobox.co m>
http://www.pobox.com/~skeet Blog: http://www.msmvps.com/jon.skeet
If replying to the group, please do not mail me too
Hi Jon

Sorry it's me again. I don't really understand what you mean.

So I did a class called CheckedEmail:

public class CheckedEmail
{
private bool bolBounce;
private string strEmail;
private string strLog;
public CheckedEmail(bo ol Spam, string Email, string Log)
{
bolBounce = Bounce;
strEmail = Email;
strLog = Log;
}

public bool Bounce
{
set { bolBounce = value; }
get { return bolBounce; }
}

public string Email
{
set { strEmail = value; }
get { return strEmail; }
}

public string Log
{
set { strLog = value; }
get { return strLog; }
}
}

Then I fill the Queue

for (int i = 0; i < 10; i++)
{
objEmail = new CheckedEmail(fa lse, "Daniel " + i, "");
Console.WriteLi ne("Producing {0}", objEmail.Email) ;
queue.Produce(o bjEmail);
}

So in the EmailChecker() i have:

for (int i = 0; i < 10; i++)
{
CheckedEmail o = (CheckedEmail) queue.EmailChec k();
Console.WriteLi ne("\t\t\t\tCon suming {0}", o.Email);

//Check Emailaddress
objEmail.Bounce = true;
objEmail.Log = "Connect\r\nHel o OK+\r\nsender not allowed";
Thread.Sleep(rn g.Next(1000));

/*
if (objEmail.Bounc e)
{
Add objEmail.Email to the DB
Add objEmail.Log to a Logfile
}
else
//Do nothing
*/
}

How can I put the objEmail in a queue to update the DB and write the
Log when objEmail.Bounce is true?

Also with the finish Flag I don't understand what you mean. In witch
loop do I have to put what? After filling the Queue with emailaddresses
I added 5 object's with objEmail.Bounce =true. So I can check if this
value is true. If so I can quit the Thread. But I don't know how and
where.

Thanks for your Help and your patience

Daniel

Nov 13 '06 #8

Jon schrieb:
Daniel <da****@danielf .chwrote:

<snip>
So I produce the queue with all emailaddresses (200'000) and then walk
trought with 5 Threads (in this exemple).
Then I wait till the que is empty. How do I know if the last Thread has
finished the work?

One way would be to have a special value (eg null) which means "all
work is complete". At the end of adding all the "real" values to the
queue, add the same number of nulls as there are threads. Then change
the threading code to exit the loop when they see a null. Finally, from
the main loop, call Join on each of the worker threads in turn.
So I have to Write the emailaddresses in the Function EmailChecker()
back to the database right?

Presumably - it's not entirely obvious from your original description
what you need to write back.
I also have to write the Logfile in the function EmailChecker().

I'd suggest having a single thread which writes log entries, to make
sure you don't end up trying to write one log entry in the middle of
another one. This is sort of the reverse of the producer/consumer queue
you've got - there you'd have several producers and one consumer.
Alternatively, you could just synchronise writing out the log entries.
How can I close the Queue to exit correctly?

You could have a separate flag that is visible to all threads which
indicates that they should quit "early" even if there's more work to
do.
I want to run this only if somone requests it by pressing the button.

That's just a matter of responding to an appropriate button click
event.

--
Jon Skeet - <sk***@pobox.co m>
http://www.pobox.com/~skeet Blog: http://www.msmvps.com/jon.skeet
If replying to the group, please do not mail me too


Hi Jon

Sorry it's me again. I don't really understand what you mean.
So I did a class called CheckedEmail:
public class CheckedEmail
{
private bool bolBounce;
private string strEmail;
private string strLog;
public CheckedEmail(bo ol Spam, string Email, string Log)
{
bolBounce = Bounce;
strEmail = Email;
strLog = Log;
}
public bool Bounce
{
set { bolBounce = value; }
get { return bolBounce; }
}
public string Email
{
set { strEmail = value; }
get { return strEmail; }
}
public string Log
{
set { strLog = value; }
get { return strLog; }
}

}
Then I fill the Queue

for (int i = 0; i < 10; i++)
{
objEmail = new CheckedEmail(fa lse, "Daniel " + i, "");
Console.WriteLi ne("Producing {0}", objEmail.Email) ;
queue.Produce(o bjEmail);
}
So in the EmailChecker() i have:
for (int i = 0; i < 10; i++)
{
CheckedEmail o = (CheckedEmail) queue.EmailChec k();
Console.WriteLi ne("\t\t\t\tCon suming {0}", o.Email);
//Check Emailaddress
o.Bounce = true;
o.Log = "Connect\r\nHel o OK+\r\nsender not allowed";
Thread.Sleep(rn g.Next(1000));
/*
if (o.Bounce)
{
Add objEmail.Email to the DB
Add objEmail.Log to a Logfile
}
else
//Do nothing
*/
}
How can I put the objEmail in a queue to update the DB and write the
Log when objEmail.Bounce is true?
Also with the finish Flag I don't understand what you mean. In witch
loop do I have to put what? After filling the Queue with emailaddresses

I added 5 object's with objEmail.Bounce =true. So I can check if this
value is true. If so I can quit the Thread. But I don't know how and
where.
Thanks for your Help and your patience
Daniel

Nov 13 '06 #9
Daniel <da****@danielf .chwrote:
So in the EmailChecker() i have:

for (int i = 0; i < 10; i++)
{
CheckedEmail o = (CheckedEmail) queue.EmailChec k();
It's not very clear why this is called EmailCheck rather than just
Consume - the queue can be a reusable class which can handle any class,
not just CheckedEmail objects.
How can I put the objEmail in a queue to update the DB and write the
Log when objEmail.Bounce is true?
Well, you can have another queue, like the first one, which all the
worker threads add to. You'd have:

Main thread (produces email addresses to check)

/ | \
Thread 1 Thread 2 Thread 3 etc (checking mails)
\ | /

Logging/DB thread (logging and writing to the database)
Also with the finish Flag I don't understand what you mean. In witch
loop do I have to put what? After filling the Queue with emailaddresses

I added 5 object's with objEmail.Bounce =true. So I can check if this
value is true. If so I can quit the Thread. But I don't know how and
where.
I don't think that's the right way of signalling things - it's better
to have a different type of object, or null, or something else to make
it clear that it means you've finished. Either way though, just after
you've consumed the item, you check whether it means "end of queue" and
just return if so.

--
Jon Skeet - <sk***@pobox.co m>
http://www.pobox.com/~skeet Blog: http://www.msmvps.com/jon.skeet
If replying to the group, please do not mail me too
Nov 14 '06 #10

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

5
12345
by: Dan Battagin | last post by:
Is there a known bug with the interaction between the HttpWebRequest and the ThreadPool? I current spawn several HttpWebRequest's using BeginGetResponse, and they work for a while, using worker threads from the ThreadPool. However, eventually (relatively quickly) there become fewer and fewer available worker threads in the pool, until there are 0 and a System.InvalidOperationException occurs with the message: "There were not enough free...
6
2758
by: Max Adams | last post by:
Threads and ThreadPool's If I use a ThreadPool how can I tell when a thead in the threadpool has exited? I don't want to set a global member variable I would much rather be able to act on an event Also (failing this ThreadPool issue) is it possible to create an array of Worker Threads, can anyone illustrate with code please Thanks
1
1397
by: doudou-shen | last post by:
I will use threadpool do some work with threadpool . but I haven't any information about it . who can help me! thank a lot
13
2522
by: orekin | last post by:
Hi There I have been programming C# for a couple of months and am trying to master Threading. I understand that ThreadPool uses background threads (see code example in MSDN page titled 'ThreadPool Class ') .... however I would like to ensure that all my ThreadPool threads have completed before I exit my Main function.
10
8780
by: Lenn | last post by:
Hello, I have always used a certain design pattern for multithreaded Windows app; Start new worker thread from UI thread, use events to notify UI threads when something happens, update UI controls using delegates through .BeginInvoke. I came across some code samples recently where ThreadPool class is used to start new threads. My questions; 1. what's the difference? 2. Any performance benefits?
3
2310
by: Kevin | last post by:
Using this: http://msdn2.microsoft.com/en-us/library/3dasc8as(VS.80).aspx as an example I have a question concerning the reuse of objects. In the example 10 instances of the Fibonacci class are made and then all put in the threadpool at once, which is well within the limits of the threadpool. However, what happens if you want to do 10,000 Fibonacci calculations
5
2567
by: =?Utf-8?B?RkxEYXZlTQ==?= | last post by:
I'm developing an application that gets data from 100 sources (via telnet connections, but you can think stock quotes from a webservice if you like). I was planning on using the thread pool (25 at a time). I know I would start all 100 at once and as threads finish, a new thread would become available and the next one would start. However, I need to do this over and over (using an app that will run for days or longer). How can I tell...
3
2498
by: UltimateBanoffee | last post by:
Hi, I'm using asp.net 2.0 and I have an understanding issue here! I don't quite understand when the available threads in the ThreadPool are ever used. The application I have running doesn't use ThreadPool.QueueWorkItem, and doesn't create any other Threads. So say I have 100 users access my application at the exact same time and they all call on an aspx that takes a couple of seconds to process. I gather 100 sessions are active, but...
7
7204
by: =?Utf-8?B?cmJEZXZlbG9wZXI=?= | last post by:
The following is from a simple Windows application in VS2005, which has button1 and textbox1 dragged onto a form. In StartThreads(), I call ThreadPool.QueueUserWorkItem(), then call WaitOne(). My expectation is that I would see the text generated in WasteTime() before seeing the "Hey" printout that comes after WaitOne(). Instead, I'm seeing the "Hey" as the first thing to print out in the text box. Any thoughts on why WaitOne() isn't...
0
9589
marktang
by: marktang | last post by:
ONU (Optical Network Unit) is one of the key components for providing high-speed Internet services. Its primary function is to act as an endpoint device located at the user's premises. However, people are often confused as to whether an ONU can Work As a Router. In this blog post, we’ll explore What is ONU, What Is Router, ONU & Router’s main usage, and What is the difference between ONU and Router. Let’s take a closer look ! Part I. Meaning of...
0
10222
Oralloy
by: Oralloy | last post by:
Hello folks, I am unable to find appropriate documentation on the type promotion of bit-fields when using the generalised comparison operator "<=>". The problem is that using the GNU compilers, it seems that the internal comparison operator "<=>" tries to promote arguments from unsigned to signed. This is as boiled down as I can make it. Here is my compilation command: g++-12 -std=c++20 -Wnarrowing bit_field.cpp Here is the code in...
1
9999
by: Hystou | last post by:
Overview: Windows 11 and 10 have less user interface control over operating system update behaviour than previous versions of Windows. In Windows 11 and 10, there is no way to turn off the Windows Update option using the Control Panel or Settings app; it automatically checks for updates and installs any it finds, whether you like it or not. For most users, this new feature is actually very convenient. If you want to control the update process,...
1
7413
isladogs
by: isladogs | last post by:
The next Access Europe User Group meeting will be on Wednesday 1 May 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM). In this session, we are pleased to welcome a new presenter, Adolph Dupré who will be discussing some powerful techniques for using class modules. He will explain when you may want to use classes instead of User Defined Types (UDT). For example, to manage the data in unbound forms. Adolph will...
0
6675
by: conductexam | last post by:
I have .net C# application in which I am extracting data from word file and save it in database particularly. To store word all data as it is I am converting the whole word file firstly in HTML and then checking html paragraph one by one. At the time of converting from word file to html my equations which are in the word document file was convert into image. Globals.ThisAddIn.Application.ActiveDocument.Select();...
0
5310
by: TSSRALBI | last post by:
Hello I'm a network technician in training and I need your help. I am currently learning how to create and manage the different types of VPNs and I have a question about LAN-to-LAN VPNs. The last exercise I practiced was to create a LAN-to-LAN VPN between two Pfsense firewalls, by using IPSEC protocols. I succeeded, with both firewalls in the same network. But I'm wondering if it's possible to do the same thing, with 2 Pfsense firewalls...
1
3967
by: 6302768590 | last post by:
Hai team i want code for transfer the data from one system to another through IP address by using C# our system has to for every 5mins then we have to update the data what the data is updated we have to send another system
2
3570
muto222
by: muto222 | last post by:
How can i add a mobile payment intergratation into php mysql website.
3
2815
bsmnconsultancy
by: bsmnconsultancy | last post by:
In today's digital era, a well-designed website is crucial for businesses looking to succeed. Whether you're a small business owner or a large corporation in Toronto, having a strong online presence can significantly impact your brand's success. BSMN Consultancy, a leader in Website Development in Toronto offers valuable insights into creating effective websites that not only look great but also perform exceptionally well. In this comprehensive...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.