473,327 Members | 2,025 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,327 software developers and data experts.

Threading Question

Dear:

I need to know what should I do for the following scenario:

1. Application will continously listen to a folder waiting a files to be
copied
2. so for each file copies should be a thread to process the file then
deliver the reuslts
3. the expected number of file that maybe copied at same time is more than
20,000 files.
so is using ThreadPool in this case is right solution???
if not what should I use to manage the threads?>
Nov 16 '05 #1
8 3418
Raed Sawalha <Ra*********@discussions.microsoft.com> wrote:
I need to know what should I do for the following scenario:

1. Application will continously listen to a folder waiting a files to be
copied
2. so for each file copies should be a thread to process the file then
deliver the reuslts
3. the expected number of file that maybe copied at same time is more than
20,000 files.
so is using ThreadPool in this case is right solution???
if not what should I use to manage the threads?>


Personally I'd suggest avoiding the system threadpool, as it's easy to
accidentally deadlock it. You can easily create a custom threadpool
though - I have one you can use in my MiscUtil library.

See http://www.pobox.com/~skeet/csharp/miscutil

--
Jon Skeet - <sk***@pobox.com>
http://www.pobox.com/~skeet
If replying to the group, please do not mail me too
Nov 16 '05 #2
I downloading Threading DLL, so can I use it without care about thread
managing inside my program.

"Jon Skeet [C# MVP]" wrote:
Raed Sawalha <Ra*********@discussions.microsoft.com> wrote:
I need to know what should I do for the following scenario:

1. Application will continously listen to a folder waiting a files to be
copied
2. so for each file copies should be a thread to process the file then
deliver the reuslts
3. the expected number of file that maybe copied at same time is more than
20,000 files.
so is using ThreadPool in this case is right solution???
if not what should I use to manage the threads?>


Personally I'd suggest avoiding the system threadpool, as it's easy to
accidentally deadlock it. You can easily create a custom threadpool
though - I have one you can use in my MiscUtil library.

See http://www.pobox.com/~skeet/csharp/miscutil

--
Jon Skeet - <sk***@pobox.com>
http://www.pobox.com/~skeet
If replying to the group, please do not mail me too

Nov 16 '05 #3
Raed Sawalha <Ra*********@discussions.microsoft.com> wrote:
I downloading Threading DLL, so can I use it without care about thread
managing inside my program.


What do you mean by "threading DLL"?

You pretty much *always* need to care about thread management in your
program.

--
Jon Skeet - <sk***@pobox.com>
http://www.pobox.com/~skeet
If replying to the group, please do not mail me too
Nov 16 '05 #4
Oh! I just meant MiscUtil.dll, but realy I ougth to use in Managed C++ so I
did like
this
#using "C:\Utilities\MiscUtil.dll"
using namespace MiscUtil::Threading;

but when application loaded it throws exception that file MiscUtil.dll not
found, do u know the write way to import the DLL , because I used to import
..NET DLLs like this
#using <mscorlib.dll>
using namespace System;
#using "System.dll"
#using "System.Data.dll"
.....etc

but with MiscUtil.dll does not work ,why?

"Jon Skeet [C# MVP]" wrote:
Raed Sawalha <Ra*********@discussions.microsoft.com> wrote:
I downloading Threading DLL, so can I use it without care about thread
managing inside my program.


What do you mean by "threading DLL"?

You pretty much *always* need to care about thread management in your
program.

--
Jon Skeet - <sk***@pobox.com>
http://www.pobox.com/~skeet
If replying to the group, please do not mail me too

Nov 16 '05 #5
Raed Sawalha <Ra*********@discussions.microsoft.com> wrote:
Oh! I just meant MiscUtil.dll, but realy I ougth to use in Managed C++ so I
did like
this
#using "C:\Utilities\MiscUtil.dll"
using namespace MiscUtil::Threading;

but when application loaded it throws exception that file MiscUtil.dll not
found, do u know the write way to import the DLL , because I used to import
.NET DLLs like this
#using <mscorlib.dll>
using namespace System;
#using "System.dll"
#using "System.Data.dll"
....etc

but with MiscUtil.dll does not work ,why?


I don't know how you do it in Managed C++, I'm afraid - in C# you just
add a reference to the DLL in the project.

--
Jon Skeet - <sk***@pobox.com>
http://www.pobox.com/~skeet
If replying to the group, please do not mail me too
Nov 16 '05 #6
moving 20,000 files at a time from one location to another. Adding threads
won't help very much (it will help a little, but not much). The reason is
that you are not thread-bound or even processor-bound... you are disk-bound.
In other words, you can't go any faster than the time it takes to move
blocks of data off of your disk (and either back onto the disk or out your
network port).

After more than a small number of threads (<12), there will be no advantage
to adding more threads... so plan to queue the requests and run through the
queue on each thread as fast as you can.

--
--- Nick Malik [Microsoft]
MCSD, CFPS, Certified Scrummaster
http://blogs.msdn.com/nickmalik

Disclaimer: Opinions expressed in this forum are my own, and not
representative of my employer.
I do not answer questions on behalf of my employer. I'm just a
programmer helping programmers.
--
"Raed Sawalha" <Ra*********@discussions.microsoft.com> wrote in message
news:8D**********************************@microsof t.com...
Dear:

I need to know what should I do for the following scenario:

1. Application will continously listen to a folder waiting a files to be
copied
2. so for each file copies should be a thread to process the file then
deliver the reuslts
3. the expected number of file that maybe copied at same time is more than 20,000 files.
so is using ThreadPool in this case is right solution???
if not what should I use to manage the threads?>

Nov 16 '05 #7
Nick,

However it can give a nice sound hearing that diskhead going from left to
right all the time?

:-))

Cor
Nov 16 '05 #8
I agree Nick. I would only add that adding threads would probably not even
help a little and would only slow it down. Would need to test
implementation to be sure.

--
William Stacey, MVP
http://mvp.support.microsoft.com

"Nick Malik [Microsoft]" <ni*******@hotmail.nospam.com> wrote in message
news:xcWAd.257662$5K2.166205@attbi_s03...
moving 20,000 files at a time from one location to another. Adding threads won't help very much (it will help a little, but not much). The reason is
that you are not thread-bound or even processor-bound... you are disk-bound. In other words, you can't go any faster than the time it takes to move
blocks of data off of your disk (and either back onto the disk or out your
network port).

After more than a small number of threads (<12), there will be no advantage to adding more threads... so plan to queue the requests and run through the queue on each thread as fast as you can.

--
--- Nick Malik [Microsoft]
MCSD, CFPS, Certified Scrummaster
http://blogs.msdn.com/nickmalik

Disclaimer: Opinions expressed in this forum are my own, and not
representative of my employer.
I do not answer questions on behalf of my employer. I'm just a
programmer helping programmers.
--
"Raed Sawalha" <Ra*********@discussions.microsoft.com> wrote in message
news:8D**********************************@microsof t.com...
Dear:

I need to know what should I do for the following scenario:

1. Application will continously listen to a folder waiting a files to be
copied
2. so for each file copies should be a thread to process the file then
deliver the reuslts
3. the expected number of file that maybe copied at same time is more

than
20,000 files.
so is using ThreadPool in this case is right solution???
if not what should I use to manage the threads?>



Nov 16 '05 #9

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

65
by: Anthony_Barker | last post by:
I have been reading a book about the evolution of the Basic programming language. The author states that Basic - particularly Microsoft's version is full of compromises which crept in along the...
19
by: Jane Austine | last post by:
As far as I know python's threading module models after Java's. However, I can't find something equivalent to Java's interrupt and isInterrupted methods, along with InterruptedException....
3
by: David Harrison | last post by:
I am working on an application on Mac OS X that calls out to python via PyImport_ImportModule(). I find that if the imported module creates and starts a python thread, the thread seems to be...
4
by: Antal Rutz | last post by:
Hi, All! I'm new to threading. I have some design questions: Task: I collect data and store them in an RDBMS (mysql or pgsql) The question is how to do that with threading? The...
6
by: CK | last post by:
I have the following code in a windows service, when I start the windows service process1 and process2 work fine , but final process (3) doesnt get called. i stop and restart the windows service...
7
by: Anthony Nystrom | last post by:
What is the correct way to stop a thread? abort? sleep? Will it start up again... Just curious... If the thread is enabling a form, if the form is disposed is the thread as well? Thanks, ...
4
by: Bob | last post by:
- For cleanup, is it sufficient to set a Thread to Nothing after it's done? - It is OK to pass objects out of the thread? (dumb question maybe but I want to be sure) - What's the best way to...
4
by: DBC User | last post by:
I have a background process which reads a table to see if there are any pending requests. If there are any, then it will start a worker thread (only 10 allowed at a time) and executes a method. In...
4
by: Steven | last post by:
I am taking an "advanced" VB.Net course via web at a state university toward an information science degree. This is my second VB class and I am kind of disappointed in it. This week we covered...
19
by: frankiespark | last post by:
Hello all, I was perusing the internet for information on threading when I came across this group. Since there seems to be a lot of good ideas and useful info I thought I'd pose a question. ...
0
by: DolphinDB | last post by:
Tired of spending countless mintues downsampling your data? Look no further! In this article, you’ll learn how to efficiently downsample 6.48 billion high-frequency records to 61 million...
1
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 6 Mar 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:15 (7.15PM). In this month's session, we are pleased to welcome back...
0
by: Vimpel783 | last post by:
Hello! Guys, I found this code on the Internet, but I need to modify it a little. It works well, the problem is this: Data is sent from only one cell, in this case B5, but it is necessary that data...
0
by: jfyes | last post by:
As a hardware engineer, after seeing that CEIWEI recently released a new tool for Modbus RTU Over TCP/UDP filtering and monitoring, I actively went to its official website to take a look. It turned...
1
by: PapaRatzi | last post by:
Hello, I am teaching myself MS Access forms design and Visual Basic. I've created a table to capture a list of Top 30 singles and forms to capture new entries. The final step is a form (unbound)...
1
by: CloudSolutions | last post by:
Introduction: For many beginners and individual users, requiring a credit card and email registration may pose a barrier when starting to use cloud servers. However, some cloud server providers now...
1
by: Defcon1945 | last post by:
I'm trying to learn Python using Pycharm but import shutil doesn't work
1
by: Shællîpôpï 09 | last post by:
If u are using a keypad phone, how do u turn on JavaScript, to access features like WhatsApp, Facebook, Instagram....
0
by: af34tf | last post by:
Hi Guys, I have a domain whose name is BytesLimited.com, and I want to sell it. Does anyone know about platforms that allow me to list my domain in auction for free. Thank you

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.