Dear:
I need to know what should I do for the following scenario:
1. Application will continously listen to a folder waiting a files to be
copied
2. so for each file copies should be a thread to process the file then
deliver the reuslts
3. the expected number of file that maybe copied at same time is more than
20,000 files.
so is using ThreadPool in this case is right solution???
if not what should I use to manage the threads?> 8 3495
Raed Sawalha <Ra*********@di scussions.micro soft.com> wrote: I need to know what should I do for the following scenario:
1. Application will continously listen to a folder waiting a files to be copied 2. so for each file copies should be a thread to process the file then deliver the reuslts 3. the expected number of file that maybe copied at same time is more than 20,000 files. so is using ThreadPool in this case is right solution??? if not what should I use to manage the threads?>
Personally I'd suggest avoiding the system threadpool, as it's easy to
accidentally deadlock it. You can easily create a custom threadpool
though - I have one you can use in my MiscUtil library.
See http://www.pobox.com/~skeet/csharp/miscutil
--
Jon Skeet - <sk***@pobox.co m> http://www.pobox.com/~skeet
If replying to the group, please do not mail me too
I downloading Threading DLL, so can I use it without care about thread
managing inside my program.
"Jon Skeet [C# MVP]" wrote: Raed Sawalha <Ra*********@di scussions.micro soft.com> wrote: I need to know what should I do for the following scenario:
1. Application will continously listen to a folder waiting a files to be copied 2. so for each file copies should be a thread to process the file then deliver the reuslts 3. the expected number of file that maybe copied at same time is more than 20,000 files. so is using ThreadPool in this case is right solution??? if not what should I use to manage the threads?>
Personally I'd suggest avoiding the system threadpool, as it's easy to accidentally deadlock it. You can easily create a custom threadpool though - I have one you can use in my MiscUtil library.
See http://www.pobox.com/~skeet/csharp/miscutil
-- Jon Skeet - <sk***@pobox.co m> http://www.pobox.com/~skeet If replying to the group, please do not mail me too
Raed Sawalha <Ra*********@di scussions.micro soft.com> wrote: I downloading Threading DLL, so can I use it without care about thread managing inside my program.
What do you mean by "threading DLL"?
You pretty much *always* need to care about thread management in your
program.
--
Jon Skeet - <sk***@pobox.co m> http://www.pobox.com/~skeet
If replying to the group, please do not mail me too
Oh! I just meant MiscUtil.dll, but realy I ougth to use in Managed C++ so I
did like
this
#using "C:\Utilities\M iscUtil.dll"
using namespace MiscUtil::Threa ding;
but when application loaded it throws exception that file MiscUtil.dll not
found, do u know the write way to import the DLL , because I used to import
..NET DLLs like this
#using <mscorlib.dll >
using namespace System;
#using "System.dll "
#using "System.Data.dl l"
.....etc
but with MiscUtil.dll does not work ,why?
"Jon Skeet [C# MVP]" wrote: Raed Sawalha <Ra*********@di scussions.micro soft.com> wrote: I downloading Threading DLL, so can I use it without care about thread managing inside my program.
What do you mean by "threading DLL"?
You pretty much *always* need to care about thread management in your program.
-- Jon Skeet - <sk***@pobox.co m> http://www.pobox.com/~skeet If replying to the group, please do not mail me too
Raed Sawalha <Ra*********@di scussions.micro soft.com> wrote: Oh! I just meant MiscUtil.dll, but realy I ougth to use in Managed C++ so I did like this #using "C:\Utilities\M iscUtil.dll" using namespace MiscUtil::Threa ding;
but when application loaded it throws exception that file MiscUtil.dll not found, do u know the write way to import the DLL , because I used to import .NET DLLs like this #using <mscorlib.dll > using namespace System; #using "System.dll " #using "System.Data.dl l" ....etc
but with MiscUtil.dll does not work ,why?
I don't know how you do it in Managed C++, I'm afraid - in C# you just
add a reference to the DLL in the project.
--
Jon Skeet - <sk***@pobox.co m> http://www.pobox.com/~skeet
If replying to the group, please do not mail me too
moving 20,000 files at a time from one location to another. Adding threads
won't help very much (it will help a little, but not much). The reason is
that you are not thread-bound or even processor-bound... you are disk-bound.
In other words, you can't go any faster than the time it takes to move
blocks of data off of your disk (and either back onto the disk or out your
network port).
After more than a small number of threads (<12), there will be no advantage
to adding more threads... so plan to queue the requests and run through the
queue on each thread as fast as you can.
--
--- Nick Malik [Microsoft]
MCSD, CFPS, Certified Scrummaster http://blogs.msdn.com/nickmalik
Disclaimer: Opinions expressed in this forum are my own, and not
representative of my employer.
I do not answer questions on behalf of my employer. I'm just a
programmer helping programmers.
--
"Raed Sawalha" <Ra*********@di scussions.micro soft.com> wrote in message
news:8D******** *************** ***********@mic rosoft.com... Dear:
I need to know what should I do for the following scenario:
1. Application will continously listen to a folder waiting a files to be copied 2. so for each file copies should be a thread to process the file then deliver the reuslts 3. the expected number of file that maybe copied at same time is more
than 20,000 files. so is using ThreadPool in this case is right solution??? if not what should I use to manage the threads?>
Nick,
However it can give a nice sound hearing that diskhead going from left to
right all the time?
:-))
Cor
I agree Nick. I would only add that adding threads would probably not even
help a little and would only slow it down. Would need to test
implementation to be sure.
--
William Stacey, MVP http://mvp.support.microsoft.com
"Nick Malik [Microsoft]" <ni*******@hotm ail.nospam.com> wrote in message
news:xcWAd.2576 62$5K2.166205@a ttbi_s03... moving 20,000 files at a time from one location to another. Adding
threads won't help very much (it will help a little, but not much). The reason is that you are not thread-bound or even processor-bound... you are
disk-bound. In other words, you can't go any faster than the time it takes to move blocks of data off of your disk (and either back onto the disk or out your network port).
After more than a small number of threads (<12), there will be no
advantage to adding more threads... so plan to queue the requests and run through
the queue on each thread as fast as you can.
-- --- Nick Malik [Microsoft] MCSD, CFPS, Certified Scrummaster http://blogs.msdn.com/nickmalik
Disclaimer: Opinions expressed in this forum are my own, and not representative of my employer. I do not answer questions on behalf of my employer. I'm just a programmer helping programmers. -- "Raed Sawalha" <Ra*********@di scussions.micro soft.com> wrote in message news:8D******** *************** ***********@mic rosoft.com... Dear:
I need to know what should I do for the following scenario:
1. Application will continously listen to a folder waiting a files to be copied 2. so for each file copies should be a thread to process the file then deliver the reuslts 3. the expected number of file that maybe copied at same time is more than 20,000 files. so is using ThreadPool in this case is right solution??? if not what should I use to manage the threads?>
This thread has been closed and replies have been disabled. Please start a new discussion. Similar topics |
by: Anthony_Barker |
last post by:
I have been reading a book about the evolution of the Basic
programming language. The author states that Basic - particularly
Microsoft's version is full of compromises which crept in along the
language's 30+ year evolution.
What to you think python largest compromises are?
The three that come to my mind are significant whitespace, dynamic
typing, and that it is interpreted - not compiled. These three put
python under fire and cause...
|
by: Jane Austine |
last post by:
As far as I know python's threading module models after Java's.
However, I can't find something equivalent to Java's interrupt and
isInterrupted methods, along with InterruptedException.
"somethread.interrupt()" will wake somethread up when it's in
sleeping/waiting state.
Is there any way of doing this with python's thread? I suppose thread
interrupt is a very primitive functionality for stopping a blocked
thread.
|
by: David Harrison |
last post by:
I am working on an application on Mac OS X that calls out
to python via PyImport_ImportModule(). I find that if
the imported module creates and starts a python thread,
the thread seems to be killed when the import of
the module is complete. Is this expected? Does
python have to be in control to allow threads to run?
Would it be better to arrange things such that the
file is processed using PyRun_SimpleFile?
David S. Harrison
|
by: Antal Rutz |
last post by:
Hi, All!
I'm new to threading. I have some design questions:
Task: I collect data and store them in an RDBMS (mysql or pgsql)
The question is how to do that with threading?
The data-collecting piece of the code runs in a thread.
1. Open the db, and each thread writes the result immediately.
(Sub-question: which is better: cursor object passed to the thread
|
by: CK |
last post by:
I have the following code in a windows service, when I start the windows
service process1 and process2 work fine , but final process (3) doesnt get
called. i stop and restart the windows service and the final process(3) gets
called. what am I doing wrong with the threading? by the way
Directory.GetFiles(IncomingXMLPath1).Length is some global outcome from
process 1.
Thanks
1)
| |
by: Anthony Nystrom |
last post by:
What is the correct way to stop a thread? abort? sleep? Will it start up
again... Just curious... If the thread is enabling a form, if the form is
disposed is the thread as well?
Thanks,
Anthony Nystrom
|
by: Bob |
last post by:
- For cleanup, is it sufficient to set a Thread to Nothing after it's done?
- It is OK to pass objects out of the thread? (dumb question maybe but I
want to be sure)
- What's the best way to process messages coming out of a thread? I want to
queue them up, but MessageQueue doesn't look like what I need. Should I just
make my own queue class? If so I'll have to worry about enumerator
synchronization... a pointer to a 'best practice'...
|
by: DBC User |
last post by:
I have a background process which reads a table to see if there are any
pending requests. If there are any, then it will start a worker thread
(only 10 allowed at a time) and executes a method. In this method, I
iniate a PROCESS and on completion, it reduces the available worker
thread and continue.
I have couple of questions;
1. Since I am launching multiple threads on a same method, do I have to
take care of locking so that each one...
|
by: Steven |
last post by:
I am taking an "advanced" VB.Net course via web at a state university
toward an information science degree. This is my second VB class and
I am kind of disappointed in it. This week we covered threading. The
instructor is of the opinion that threading is not very useful and
glossed over the subject. At least he admitted that he was glossing
it over. I've noticed that many of the postings on this board have to
do with threading and I...
|
by: frankiespark |
last post by:
Hello all,
I was perusing the internet for information on threading when I came
across this group. Since there seems to be a lot of good ideas and
useful info I thought I'd pose a question.
Threading is a new concept for me to implement. Here is my problem.
I have a system that receives xml files and records their file
locations in a database. I can potentially receive thousands,
|
by: marktang |
last post by:
ONU (Optical Network Unit) is one of the key components for providing high-speed Internet services. Its primary function is to act as an endpoint device located at the user's premises. However, people are often confused as to whether an ONU can Work As a Router. In this blog post, we’ll explore What is ONU, What Is Router, ONU & Router’s main usage, and What is the difference between ONU and Router. Let’s take a closer look !
Part I. Meaning of...
| |
by: Oralloy |
last post by:
Hello folks,
I am unable to find appropriate documentation on the type promotion of bit-fields when using the generalised comparison operator "<=>".
The problem is that using the GNU compilers, it seems that the internal comparison operator "<=>" tries to promote arguments from unsigned to signed.
This is as boiled down as I can make it.
Here is my compilation command:
g++-12 -std=c++20 -Wnarrowing bit_field.cpp
Here is the code in...
|
by: jinu1996 |
last post by:
In today's digital age, having a compelling online presence is paramount for businesses aiming to thrive in a competitive landscape. At the heart of this digital strategy lies an intricately woven tapestry of website design and digital marketing. It's not merely about having a website; it's about crafting an immersive digital experience that captivates audiences and drives business growth.
The Art of Business Website Design
Your website is...
|
by: tracyyun |
last post by:
Dear forum friends,
With the development of smart home technology, a variety of wireless communication protocols have appeared on the market, such as Zigbee, Z-Wave, Wi-Fi, Bluetooth, etc. Each protocol has its own unique characteristics and advantages, but as a user who is planning to build a smart home system, I am a bit confused by the choice of these technologies. I'm particularly interested in Zigbee because I've heard it does some...
|
by: isladogs |
last post by:
The next Access Europe User Group meeting will be on Wednesday 1 May 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM).
In this session, we are pleased to welcome a new presenter, Adolph Dupré who will be discussing some powerful techniques for using class modules.
He will explain when you may want to use classes instead of User Defined Types (UDT). For example, to manage the data in unbound forms.
Adolph will...
|
by: conductexam |
last post by:
I have .net C# application in which I am extracting data from word file and save it in database particularly. To store word all data as it is I am converting the whole word file firstly in HTML and then checking html paragraph one by one.
At the time of converting from word file to html my equations which are in the word document file was convert into image.
Globals.ThisAddIn.Application.ActiveDocument.Select();...
|
by: TSSRALBI |
last post by:
Hello
I'm a network technician in training and I need your help.
I am currently learning how to create and manage the different types of VPNs and I have a question about LAN-to-LAN VPNs.
The last exercise I practiced was to create a LAN-to-LAN VPN between two Pfsense firewalls, by using IPSEC protocols.
I succeeded, with both firewalls in the same network. But I'm wondering if it's possible to do the same thing, with 2 Pfsense firewalls...
| |
by: adsilva |
last post by:
A Windows Forms form does not have the event Unload, like VB6. What one acts like?
|
by: muto222 |
last post by:
How can i add a mobile payment intergratation into php mysql website.
| |