473,237 Members | 1,375 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,237 software developers and data experts.

How do I keep parameters from being mixed up using Multithreading?

Please see the code below as I am trying to use multithreading for copying
files to a new location in a way that improves performance of the client
windows application.

The problem occurs when 2 or more threads are created, the ImportOneFile
method attempts to add a previously added file. If I allow 4 maximum threads
and process 4 files, the last file is attempted 4 times and none of the other
files are added to the destination. If I reduce maximum threads to 2, the
second file is attempted twice, the first one is forgotten and other files
are added to the destination.

Anyone have any idea why the values passed to ImportOneFile are sometimes
"forgotten"? I thought the lock statement would protect against this.

Thanks in Advance, Mark

Sample code:
private void ImportFiles(string sourceFolderPath)
{
int numberOfRunningThreads;
int maxNumberThreads = 4;

//navigate through all of the files in this folder
FileInfo[] currentFolderFiles;
currentFolderFiles = new DirectoryInfo(sourceFolderPath)).GetFiles();

foreach (FileInfo currentFile in currentFolderFiles)
{

//make sure that we don’t exceed the number of threads specified
while (numberOfRunningThreads > (maxNumberThreads - 1))
{
//wait
Thread.Sleep(500);
}

//create a string array to pass to ImportFile
string[] importDetails = new string[2];

//set the array to the file name and the doc library path
importDetails[0] = currentFile.FullName;
importDetails[1] = destinationPath;

//assign the method to the callback to the ImportOneFile method
WaitCallback callBack = new WaitCallback(ImportOneFile);

//interlock increment the number of threads running
Interlocked.Increment(ref _numberOfRunningThreads);

//create a thread on the queue and provide the method with the
string array
ThreadPool.QueueUserWorkItem(callBack, (object)importDetails);
} //end of for loop

//make sure all of the threads are completed processing before leaving method
int loopCount = 0;
while ((_numberOfRunningThreads > 0) && (loopCount < 6000))
{
Thread.Sleep(50);
loopCount++;
}

if (loopCount > 5999)
{
throw new Exception(“Timeout waiting for threads to complete.”);
}
}

private void ImportOneFile(object fileDetails)
{
//implement thread locking for protecting while using multithreading
lock(this)
{
//Using QueueUserWorkItem, parameters have to be sent to an object
// so here the parameters are repackaged in a string
string[] detailStrings = (string[])fileDetails;

//get an implementation of the file adder from the factory
CustomFileAdder fileAdder = new CustomFileAdder;

//add file, provide destination path, file name, contents
of file
fileAdder.AddFile(detailStrings[1],
Path.GetFileName(detailStrings[0]), BinaryFileContents(detailStrings[0]));
}
//interlock decrement since we are done
Interlocked.Decrement(ref _numberOfRunningThreads);

}

Dec 13 '05 #1
5 2557
Please see the code below as I am trying to use multithreading for copying
files to a new location in a way that improves performance of the client
windows application.
Please post relevant parts of your actual code that compiles. The code
you posted have a number of errors in it, and it's pointless to guess
what's wrong if we're not looking at your actual code.

I thought the lock statement would protect against this.


By using lock(this) (which is a bad thing in itself) around the entire
ImportOneFile method body you're effectively serializing access to
that code - only one thread at a time will run it. So the point of
having multiple threads here is sort of lost.
Mattias

--
Mattias Sjgren [C# MVP] mattias @ mvps.org
http://www.msjogren.net/dotnet/ | http://www.dotnetinterop.com
Please reply only to the newsgroup.
Dec 13 '05 #2
Thanks for responding, Mattias!

Sorry about the code. Let me know if you have any issues with this:

//you will need the following references
using System.IO;
using System.Threading;

//add the following methods to a class, calling ImportFiles
private void ImportFiles()
{

//create a string array to pass to ImportFile
string[] importDetails = new string[2];

int numberOfRunningThreads = 0;
int maxNumberThreads = 4;

string sourceFolderPath = "E:\\Data\\TestUploadDocs";
string destinationPath = "E:\\Data\\Destination";

//navigate through all of the files in this folder
FileInfo[] currentFolderFiles;
currentFolderFiles = new DirectoryInfo(sourceFolderPath)).GetFiles();
foreach (FileInfo currentFile in currentFolderFiles)
{

//make sure that we don’t exceed the number of threads specified
while (numberOfRunningThreads > (maxNumberThreads - 1))
{
//wait
Thread.Sleep(500);
}

//set the array to the file name and the doc library path
importDetails[0] = currentFile.FullName;
importDetails[1] = destinationPath;

//assign the method to the callback to the ImportOneFile method
WaitCallback callBack = new WaitCallback(ImportOneFile);

//interlock increment the number of threads running
Interlocked.Increment(ref _numberOfRunningThreads);

//create a thread on the queue and provide the method with the
string array
ThreadPool.QueueUserWorkItem(callBack, (object)importDetails);
}

//make sure all of the threads are completed processing before leaving method
int loopCount = 0;
while ((_numberOfRunningThreads > 0) && (loopCount < 6000))
{
Thread.Sleep(50);
loopCount++;
}

if (loopCount > 5999)
{
throw new Exception("Timeout waiting for threads to complete.");
}
}

private void ImportOneFile(object fileDetails)
{
//implement thread locking for protecting while using multithreading
lock(this)
{
//Using QueueUserWorkItem, parameters have to be sent to an object
// so here the parameters are repackaged in a string
string[] detailStrings = (string[])fileDetails;
Console.WriteLine("Inside ImportOneFile the file is {0} and the
destination path is {1}", detailStrings[0], detailStrings[1]);
}
//interlock decrement since we are done
Interlocked.Decrement(ref _numberOfRunningThreads);

}
"Mattias Sjögren" wrote:
Please see the code below as I am trying to use multithreading for copying
files to a new location in a way that improves performance of the client
windows application.


Please post relevant parts of your actual code that compiles. The code
you posted have a number of errors in it, and it's pointless to guess
what's wrong if we're not looking at your actual code.

I thought the lock statement would protect against this.


By using lock(this) (which is a bad thing in itself) around the entire
ImportOneFile method body you're effectively serializing access to
that code - only one thread at a time will run it. So the point of
having multiple threads here is sort of lost.
Mattias

--
Mattias Sjögren [C# MVP] mattias @ mvps.org
http://www.msjogren.net/dotnet/ | http://www.dotnetinterop.com
Please reply only to the newsgroup.

Dec 13 '05 #3
private void ImportFiles()
{

//create a string array to pass to ImportFile
string[] importDetails = new string[2];
Here (unlike your first post) you're creating the array once outside
the foreach loop. By reusing the same array all the time you're likely
to see the behavior you initially described. Moving the array inside
the foreach loop should fix it.

currentFolderFiles = new DirectoryInfo(sourceFolderPath)).GetFiles();
You still have unbalanced parentheses here.

//make sure that we dont exceed the number of threads specified
while (numberOfRunningThreads > (maxNumberThreads - 1))
{
//wait
Thread.Sleep(500);
}
This will never run since the local numberOfRunningThreads variable is
never incremented, only the _numberOfRunningThreads field.

lock(this)
{


You don't need the lock statement here.
Mattias

--
Mattias Sjgren [C# MVP] mattias @ mvps.org
http://www.msjogren.net/dotnet/ | http://www.dotnetinterop.com
Please reply only to the newsgroup.
Dec 13 '05 #4
Thanks, Mattias!

I had earlier created the string array inside the for loop and had this
issue. Now it is back inside the for loop and I took off the lock statement.
It is behaving as I expected now.

"Mattias Sjögren" wrote:
private void ImportFiles()
{

//create a string array to pass to ImportFile
string[] importDetails = new string[2];


Here (unlike your first post) you're creating the array once outside
the foreach loop. By reusing the same array all the time you're likely
to see the behavior you initially described. Moving the array inside
the foreach loop should fix it.

currentFolderFiles = new DirectoryInfo(sourceFolderPath)).GetFiles();


You still have unbalanced parentheses here.

//make sure that we don’t exceed the number of threads specified
while (numberOfRunningThreads > (maxNumberThreads - 1))
{
//wait
Thread.Sleep(500);
}


This will never run since the local numberOfRunningThreads variable is
never incremented, only the _numberOfRunningThreads field.

lock(this)
{


You don't need the lock statement here.
Mattias

--
Mattias Sjögren [C# MVP] mattias @ mvps.org
http://www.msjogren.net/dotnet/ | http://www.dotnetinterop.com
Please reply only to the newsgroup.

Dec 13 '05 #5
You need to synchronize reads of the numberOfRunningThreads variable as
well. Otherwise, changes made by other threads will not be seen and
you could end up with an infinite loop. You can use the
Interlocked.CompareExchange or Interlocked.Read methods to do that.

Better yet, use a semaphore to throttle the number of threads allowed
to enter a specified region of code. It's a cleaner solution than
incrementing/decrementing the numberOfRunningThreads variable and
attempting to block depending on its value.

Also, why limit the number of concurrently running work items in the
first place? Just queue them all up and let the ThreadPool do what it
does best. Don't worry about how many threads are executing.

Brian

mr*******@newsgroup.nospam wrote:
Thanks for responding, Mattias!

Sorry about the code. Let me know if you have any issues with this:

//you will need the following references
using System.IO;
using System.Threading;

//add the following methods to a class, calling ImportFiles
private void ImportFiles()
{

//create a string array to pass to ImportFile
string[] importDetails = new string[2];

int numberOfRunningThreads = 0;
int maxNumberThreads = 4;

string sourceFolderPath = "E:\\Data\\TestUploadDocs";
string destinationPath = "E:\\Data\\Destination";

//navigate through all of the files in this folder
FileInfo[] currentFolderFiles;
currentFolderFiles = new DirectoryInfo(sourceFolderPath)).GetFiles();
foreach (FileInfo currentFile in currentFolderFiles)
{

//make sure that we don't exceed the number of threads specified
while (numberOfRunningThreads > (maxNumberThreads - 1))
{
//wait
Thread.Sleep(500);
}

//set the array to the file name and the doc library path
importDetails[0] = currentFile.FullName;
importDetails[1] = destinationPath;

//assign the method to the callback to the ImportOneFile method
WaitCallback callBack = new WaitCallback(ImportOneFile);

//interlock increment the number of threads running
Interlocked.Increment(ref _numberOfRunningThreads);

//create a thread on the queue and provide the method with the
string array
ThreadPool.QueueUserWorkItem(callBack, (object)importDetails);
}

//make sure all of the threads are completed processing before leaving method
int loopCount = 0;
while ((_numberOfRunningThreads > 0) && (loopCount < 6000))
{
Thread.Sleep(50);
loopCount++;
}

if (loopCount > 5999)
{
throw new Exception("Timeout waiting for threads to complete.");
}
}

private void ImportOneFile(object fileDetails)
{
//implement thread locking for protecting while using multithreading
lock(this)
{
//Using QueueUserWorkItem, parameters have to be sent to an object
// so here the parameters are repackaged in a string
string[] detailStrings = (string[])fileDetails;
Console.WriteLine("Inside ImportOneFile the file is {0} and the
destination path is {1}", detailStrings[0], detailStrings[1]);
}
//interlock decrement since we are done
Interlocked.Decrement(ref _numberOfRunningThreads);

}
"Mattias Sjgren" wrote:
Please see the code below as I am trying to use multithreading for copying
files to a new location in a way that improves performance of the client
windows application.


Please post relevant parts of your actual code that compiles. The code
you posted have a number of errors in it, and it's pointless to guess
what's wrong if we're not looking at your actual code.

I thought the lock statement would protect against this.


By using lock(this) (which is a bad thing in itself) around the entire
ImportOneFile method body you're effectively serializing access to
that code - only one thread at a time will run it. So the point of
having multiple threads here is sort of lost.
Mattias

--
Mattias Sjgren [C# MVP] mattias @ mvps.org
http://www.msjogren.net/dotnet/ | http://www.dotnetinterop.com
Please reply only to the newsgroup.


Dec 13 '05 #6

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

12
by: John Devereux | last post by:
Hi, I would like to know if their is a conversion specifier, or other method, that will allow me to not use particular arguments to printf. Failing that, is it permissable to simply "not use"...
1
by: Matthias Klpper | last post by:
Hi, I have a little problem with supplying a mixed set of byVal and byRef parameters to a dynamically created PInvoke method. I have the following message signature which works perfectly when...
16
by: Robert Zurer | last post by:
Can anyone suggest the best book or part of a book on this subject. I'm looking for an in-depth treatment with examples in C# TIA Robert Zurer robert@zurer.com
6
by: VM | last post by:
I'm trying to add multithreading to my win application but I'm having trouble with the code since the method to be threaded has parameters. How can I add multithreading to a method with parameters?...
8
by: Nadav | last post by:
Hi, I am writing a performence critical application, this require me to stick to unmanaged C++ as performance is much better using unmanaged C++ ( about 33% better ), Still, I am trying to avoid...
4
by: Mike Dinnis | last post by:
Hi, I've been working through a number of turorials to try to learn more about retrieving data from a SQL database. I think i've mastered techniques where i create a sql string in the page and...
2
by: partybob99 | last post by:
I am trying to call SP_Password from some vb.net code. This should be very straight forward but no matter what I do, I keep getting errors. Here is the code strConnectString = "Data Source=" +...
4
by: Dan Krantz | last post by:
I have the following template to ensure that a given number (val) falls into a range (between vmin & vmax): template<typename T> T ForceNumericRange( const T& val, const T& vmin, const T& vmax)...
2
by: Anish G | last post by:
Hi, I am new to working on threading. I need to implement multithreading for calling four functions (each functions are having parameters) in different threads so that i can improve the...
2
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 7 Feb 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:30 (7.30PM). In this month's session, the creator of the excellent VBE...
0
by: fareedcanada | last post by:
Hello I am trying to split number on their count. suppose i have 121314151617 (12cnt) then number should be split like 12,13,14,15,16,17 and if 11314151617 (11cnt) then should be split like...
0
Git
by: egorbl4 | last post by:
Скачал я git, хотел начать настройку, а там вылезло вот это Что это? Что мне с этим делать? ...
1
by: davi5007 | last post by:
Hi, Basically, I am trying to automate a field named TraceabilityNo into a web page from an access form. I've got the serial held in the variable strSearchString. How can I get this into the...
0
by: MeoLessi9 | last post by:
I have VirtualBox installed on Windows 11 and now I would like to install Kali on a virtual machine. However, on the official website, I see two options: "Installer images" and "Virtual machines"....
0
by: DolphinDB | last post by:
The formulas of 101 quantitative trading alphas used by WorldQuant were presented in the paper 101 Formulaic Alphas. However, some formulas are complex, leading to challenges in calculation. Take...
0
by: DolphinDB | last post by:
Tired of spending countless mintues downsampling your data? Look no further! In this article, youll learn how to efficiently downsample 6.48 billion high-frequency records to 61 million...
0
by: Aftab Ahmad | last post by:
So, I have written a code for a cmd called "Send WhatsApp Message" to open and send WhatsApp messaage. The code is given below. Dim IE As Object Set IE =...
0
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 6 Mar 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:15 (7.15PM). In this month's session, we are pleased to welcome back...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.