Everywhere in documentation there are recommendations to use threads from
thread pooling for relatively short tasks. As I understand, fetching a page
or multiple pages (sometimes up to 50 but not tipical) from the Internet and
doing some processing on those would be considered to be a big/long task for
a thread from a pool. In our app it is possible to break the task into some
small ones (thread per fetch and processing thereafter or event smaller), but
the more threads we create the more context switches we get. Where is the
ballance point? What penalties do we incur for big/long tasks?
One more thing. There seems to be a bug with large numbers of asynchronous
IO where threads deadlock and timeout (it is all over the developers
communities). Any info on when the fix is going to be out?
Thank you,
Alexei
31  2449 
You may want to be real careful about using the ThreadPool for downloading
stuff using WebRequest classes as they use the ThreadPool too. I had a lot
of pain figuring this out - not fun!
--
Sriram Krishnan http://www.dotnetjunkies.com/weblog/sriram
"AlexeiOst" <Al*******@disc ussions.microso ft.com> wrote in message
news:79******** *************** ***********@mic rosoft.com... Everywhere in documentation there are recommendations to use threads from
thread pooling for relatively short tasks. As I understand, fetching a
page
or multiple pages (sometimes up to 50 but not tipical) from the Internet
and
doing some processing on those would be considered to be a big/long task
for
a thread from a pool. In our app it is possible to break the task into
some
small ones (thread per fetch and processing thereafter or event smaller),
but
the more threads we create the more context switches we get. Where is the
ballance point? What penalties do we incur for big/long tasks?
One more thing. There seems to be a bug with large numbers of asynchronous
IO where threads deadlock and timeout (it is all over the developers
communities). Any info on when the fix is going to be out?
Thank you,
Alexei
Thank you for your warning. Unfortunatelly, I bumped into this problem
already as I mentioned in the 'One more thing' part of the message. Or is
there something else that you saw?
Thank you,
Alexei
"Sriram Krishnan" wrote: You may want to be real careful about using the ThreadPool for downloading
stuff using WebRequest classes as they use the ThreadPool too. I had a lot
of pain figuring this out - not fun!
--
Sriram Krishnan
http://www.dotnetjunkies.com/weblog/sriram
"AlexeiOst" <Al*******@disc ussions.microso ft.com> wrote in message
news:79******** *************** ***********@mic rosoft.com... Everywhere in documentation there are recommendations to use threads from
thread pooling for relatively short tasks. As I understand, fetching a
page
or multiple pages (sometimes up to 50 but not tipical) from the Internet
and
doing some processing on those would be considered to be a big/long task
for
a thread from a pool. In our app it is possible to break the task into
some
small ones (thread per fetch and processing thereafter or event smaller),
but
the more threads we create the more context switches we get. Where is the
ballance point? What penalties do we incur for big/long tasks?
One more thing. There seems to be a bug with large numbers of asynchronous
IO where threads deadlock and timeout (it is all over the developers
communities). Any info on when the fix is going to be out?
Thank you,
Alexei
Unfortunately - you consider it a bug while Microsoft says that is 'by
design'. So you're stuck I'm afraid. You might want to check out http://www.bearcanyon.com/dotnet/#tpcontrol for an example of how to
increase the pool thread limit
--
Sriram Krishnan http://www.dotnetjunkies.com/weblog/sriram
"AlexeiOst" <Al*******@disc ussions.microso ft.com> wrote in message
news:3D******** *************** ***********@mic rosoft.com... Thank you for your warning. Unfortunatelly, I bumped into this problem
already as I mentioned in the 'One more thing' part of the message. Or is
there something else that you saw?
Thank you,
Alexei
"Sriram Krishnan" wrote:
You may want to be real careful about using the ThreadPool for
downloading
stuff using WebRequest classes as they use the ThreadPool too. I had a
lot
of pain figuring this out - not fun!
--
Sriram Krishnan
http://www.dotnetjunkies.com/weblog/sriram
"AlexeiOst" <Al*******@disc ussions.microso ft.com> wrote in message
news:79******** *************** ***********@mic rosoft.com... > Everywhere in documentation there are recommendations to use threads
> from
> thread pooling for relatively short tasks. As I understand, fetching a
> page
> or multiple pages (sometimes up to 50 but not tipical) from the
> Internet
> and
> doing some processing on those would be considered to be a big/long
> task
> for
> a thread from a pool. In our app it is possible to break the task into
> some
> small ones (thread per fetch and processing thereafter or event
> smaller),
> but
> the more threads we create the more context switches we get. Where is
> the
> ballance point? What penalties do we incur for big/long tasks?
>
> One more thing. There seems to be a bug with large numbers of
> asynchronous
> IO where threads deadlock and timeout (it is all over the developers
> communities). Any info on when the fix is going to be out?
>
> Thank you,
>
> Alexei
The threadpool is a system wide resource that by default provides 25
threads. The BCL uses this internally for many of its operations so if you
are making heavy use of it yourself in long-running tasks there is a danger
that you will exhaust all available threads.
However, it does sound like your processing requirements are good candidates
for a threadpool. There are plenty of opensource threadpool classes
available that you can use for those tasks; use google to locate one. This
will give you complete control over the number of threads in the threadpool
and how they behave, and you can customize it for your own purposes (e.g.
tasks of different priorities, etc).
"AlexeiOst" <Al*******@disc ussions.microso ft.com> wrote in message
news:79******** *************** ***********@mic rosoft.com... Everywhere in documentation there are recommendations to use threads from
thread pooling for relatively short tasks. As I understand, fetching a
page
or multiple pages (sometimes up to 50 but not tipical) from the Internet
and
doing some processing on those would be considered to be a big/long task
for
a thread from a pool. In our app it is possible to break the task into
some
small ones (thread per fetch and processing thereafter or event smaller),
but
the more threads we create the more context switches we get. Where is the
ballance point? What penalties do we incur for big/long tasks?
One more thing. There seems to be a bug with large numbers of asynchronous
IO where threads deadlock and timeout (it is all over the developers
communities). Any info on when the fix is going to be out?
Thank you,
Alexei
David Levine <no************ ****@wi.rr.com> wrote: The threadpool is a system wide resource that by default provides 25
threads. The BCL uses this internally for many of its operations so if you
are making heavy use of it yourself in long-running tasks there is a danger
that you will exhaust all available threads.
However, it does sound like your processing requirements are good candidates
for a threadpool. There are plenty of opensource threadpool classes
available that you can use for those tasks; use google to locate one. This
will give you complete control over the number of threads in the threadpool
and how they behave, and you can customize it for your own purposes (e.g.
tasks of different priorities, etc).
I have one which you're welcome to use, for example. It's part of my
MiscUtil library available at http://www.pobox.com/~skeet/csharp/miscutil
--
Jon Skeet - <sk***@pobox.co m> http://www.pobox.com/~skeet
If replying to the group, please do not mail me too
The threadpool is not a system wide resource, it's a CLR (aka. per process)
managed pool of:
1) max. 25 threads per CPU plus
2) a pool of max 1000 IOCP threads per process.
Willy.
"David Levine" <no************ ****@wi.rr.com> wrote in message
news:%2******** ********@tk2msf tngp13.phx.gbl. .. The threadpool is a system wide resource that by default provides 25
threads. The BCL uses this internally for many of its operations so if you
are making heavy use of it yourself in long-running tasks there is a
danger that you will exhaust all available threads.
However, it does sound like your processing requirements are good
candidates for a threadpool. There are plenty of opensource threadpool
classes available that you can use for those tasks; use google to locate
one. This will give you complete control over the number of threads in
the threadpool and how they behave, and you can customize it for your own
purposes (e.g. tasks of different priorities, etc).
"AlexeiOst" <Al*******@disc ussions.microso ft.com> wrote in message
news:79******** *************** ***********@mic rosoft.com... Everywhere in documentation there are recommendations to use threads from
thread pooling for relatively short tasks. As I understand, fetching a
page
or multiple pages (sometimes up to 50 but not tipical) from the Internet
and
doing some processing on those would be considered to be a big/long task
for
a thread from a pool. In our app it is possible to break the task into
some
small ones (thread per fetch and processing thereafter or event smaller),
but
the more threads we create the more context switches we get. Where is the
ballance point? What penalties do we incur for big/long tasks?
One more thing. There seems to be a bug with large numbers of
asynchronous
IO where threads deadlock and timeout (it is all over the developers
communities). Any info on when the fix is going to be out?
Thank you,
Alexei
I stated it badly...by "system" I was referring to the managed process. I
was unaware of the 1000 IOCP threads/process; is this a Windows limit or a
CLR limit?
"Willy Denoyette [MVP]" <wi************ *@pandora.be> wrote in message
news:eS******** ********@TK2MSF TNGP14.phx.gbl. .. The threadpool is not a system wide resource, it's a CLR (aka. per
process) managed pool of:
1) max. 25 threads per CPU plus
2) a pool of max 1000 IOCP threads per process.
Willy.
"David Levine" <no************ ****@wi.rr.com> wrote in message
news:%2******** ********@tk2msf tngp13.phx.gbl. .. The threadpool is a system wide resource that by default provides 25
threads. The BCL uses this internally for many of its operations so if
you are making heavy use of it yourself in long-running tasks there is a
danger that you will exhaust all available threads.
However, it does sound like your processing requirements are good
candidates for a threadpool. There are plenty of opensource threadpool
classes available that you can use for those tasks; use google to locate
one. This will give you complete control over the number of threads in
the threadpool and how they behave, and you can customize it for your own
purposes (e.g. tasks of different priorities, etc).
"AlexeiOst" <Al*******@disc ussions.microso ft.com> wrote in message
news:79******** *************** ***********@mic rosoft.com... Everywhere in documentation there are recommendations to use threads
from
thread pooling for relatively short tasks. As I understand, fetching a
page
or multiple pages (sometimes up to 50 but not tipical) from the Internet
and
doing some processing on those would be considered to be a big/long task
for
a thread from a pool. In our app it is possible to break the task into
some
small ones (thread per fetch and processing thereafter or event
smaller), but
the more threads we create the more context switches we get. Where is
the
ballance point? What penalties do we incur for big/long tasks?
One more thing. There seems to be a bug with large numbers of
asynchronous
IO where threads deadlock and timeout (it is all over the developers
communities). Any info on when the fix is going to be out?
Thank you,
Alexei
> I have one which you're welcome to use, for example. It's part of my
MiscUtil library available at
http://www.pobox.com/~skeet/csharp/miscutil
I'll take a look at it - I'm always on the lookout for good open source
libs. Thanks.
David Levine <no************ ****@wi.rr.com> wrote: I have one which you're welcome to use, for example. It's part of my
MiscUtil library available at
http://www.pobox.com/~skeet/csharp/miscutil
I'll take a look at it - I'm always on the lookout for good open source
libs. Thanks.
Goodo - it hasn't had much use yet, so although I'm fairly sure it's
thread-safe, there may be other features which would be useful to add.
--
Jon Skeet - <sk***@pobox.co m> http://www.pobox.com/~skeet
If replying to the group, please do not mail me too This thread has been closed and replies have been disabled. Please start a new discussion. Similar topics | |
by: Andreas Suurkuusk |
last post by:
Hi,
I just noticed your post in the "C# memory problem: no end for our problem?"
thread.
In the post you implied that I do not how the garbage collector works and
that I mislead people. Since the thread is over a month old, I decided to
start a new one with my response.
Please see my comments inline.
|
by: David Sworder |
last post by:
This message was already cross-posted to C# and ADO.NET, but I forgot to
post to this "general" group... sorry about that. It just occured to me
after my first post that the "general" group readers might have some
thoughts on this perplexing .NET blocking issue.
(see below)
=====
Hi,
|
by: sleepyant |
last post by:
Hi, I have an application that will download something from a remote server
which might take a long time while downloading. To enable my form to be
'responsive' and able to push a 'Cancel' button on the main form to
terminate the download, I decided to use thread.
After a few research, I find that there are actually different ways to do...
|
by: David Sworder |
last post by:
Hi,
I'm developing an application that will support several thousand
simultaneous connections on the server-side. I'm trying to maximize
throughput. The client (WinForms) and server communicate via a socket
connection (no remoting, no ASP.NET). The client sends a message to the
server that contains some instructions and the server responds...
|
by: AlexeiOst |
last post by:
Everywhere in documentation there are recommendations to use threads from
thread pooling for relatively short tasks. As I understand, fetching a page
or multiple pages (sometimes up to 50 but not tipical) from the Internet and
doing some processing on those would be considered to be a big/long task for
a thread from a pool. In our app it is...
| | |
by: Jim Stools |
last post by:
Newbie question: Hopefully I worded this correctly...
Does each instance of an aspx page spawn its own thread? If so are the
threads protected? I assume each aspx page is processed in a synchronized
method and each instance has to finish processing before the next instance
is queued? If an aspx page has a class that requires lengthly...
|
by: Carl Heller |
last post by:
If I'm creating a class to do some work that I want threaded out, where's
the best location to call ThreadStart? Or does it depend on the nature of
the work?
a. Call it outside the class, giving it the starting method of the class?
b. Have the class create the thread itself?
ie:
x = new WorkerClass();
ioThread = new Thread(new...
|
by: salberts |
last post by:
Hi,
I am writing an application that has its UI and Logic layers. My
initial idea was to launch the two layers on two different threads and
manage calls made by the UI to the Logic manually. Which means
handling the Sleep/Wakeup of the logic thread all by myself. This
looks reasonable to me in the way that the UI and the Logic run...
|
by: =?GB2312?B?0rvK18qr?= |
last post by:
Hi all,
Recently I had a new coworker. There is some dispute between us.
The last company he worked for has a special networking programming
model. They split the business logic into different modules, and have
a dedicated thread for the each module. Modules exchanged info
through a in-memory message queue.
In my opinion, such a...
|
by: Hystou |
last post by:
Most computers default to English, but sometimes we require a different language, especially when relocating. Forgot to request a specific language before your computer shipped? No problem! You can effortlessly switch the default language on Windows 10 without reinstalling. I'll walk you through it.
First, let's disable language...
|
by: Oralloy |
last post by:
Hello folks,
I am unable to find appropriate documentation on the type promotion of bit-fields when using the generalised comparison operator "<=>".
The problem is that using the GNU compilers, it seems that the internal comparison operator "<=>" tries to promote arguments from unsigned to signed.
This is as boiled down as I can make it. ...
| | |
by: jinu1996 |
last post by:
In today's digital age, having a compelling online presence is paramount for businesses aiming to thrive in a competitive landscape. At the heart of this digital strategy lies an intricately woven tapestry of website design and digital marketing. It's not merely about having a website; it's about crafting an immersive digital experience that...
|
by: Hystou |
last post by:
Overview:
Windows 11 and 10 have less user interface control over operating system update behaviour than previous versions of Windows. In Windows 11 and 10, there is no way to turn off the Windows Update option using the Control Panel or Settings app; it automatically checks for updates and installs any it finds, whether you like it or not. For...
|
by: tracyyun |
last post by:
Dear forum friends,
With the development of smart home technology, a variety of wireless communication protocols have appeared on the market, such as Zigbee, Z-Wave, Wi-Fi, Bluetooth, etc. Each protocol has its own unique characteristics and advantages, but as a user who is planning to build a smart home system, I am a bit confused by the...
|
by: isladogs |
last post by:
The next Access Europe User Group meeting will be on Wednesday 1 May 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM).
In this session, we are pleased to welcome a new presenter, Adolph Dupré who will be discussing some powerful techniques for using class modules.
He will explain when you may want to use classes...
|
by: adsilva |
last post by:
A Windows Forms form does not have the event Unload, like VB6. What one acts like?
|
by: 6302768590 |
last post by:
Hai team
i want code for transfer the data from one system to another through IP address by using C# our system has to for every 5mins then we have to update the data what the data is updated we have to send another system
| | |
by: bsmnconsultancy |
last post by:
In today's digital era, a well-designed website is crucial for businesses looking to succeed. Whether you're a small business owner or a large corporation in Toronto, having a strong online presence can significantly impact your brand's success. BSMN Consultancy, a leader in Website Development in Toronto offers valuable insights into creating...
| |
