473,326 Members | 2,095 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,326 software developers and data experts.

FileSystemWatcher limits problem

I have a windows service that uses a filesystemwatcher
to wait for files and process them to a database.
Sometimes a large group of files will be cut and paste
for the filesystemwatcher to process. I ran into
trouble when trying to process a group of files
at a time.
The code below fixed my problem for about 15 large files
( around 100k each ). However when I tried to use it
on a group of small files ( 90 files ) it crashes (
around 6k each ). I am going to try increasing the time
for thread sleep. Any other solutions ?
Here is my code for the file system watcher using a
windows service to wait for files-

private void ProcessFiles(string FullPath)
{
while (true)
{
try
{
objFiles.Parse(FullPath);
break;
}
catch (System.IO.IOException)
{
System.Threading.Thread.Sleep( 1000 );
}
}

Nov 16 '05 #1
1 4134
This isn't realllly what the FileSystemWatcher is for. It's used more in the
context of logging file changes on system, rather than actually reading out
the file contents. I've heard of previous issues on this where the files
"exist" at the designated location, but the copy has not been completed.
This then leads to access violations on shared files etc etc...

What I've done in the past is to (for synchronous solutions) write my own
class that is responsible for polling a given folder. When new files arrive,
the array of file names are then used to extract the data, and pass it back
to the object creator. This allows you to then put in exception handling for
different errors you could get on an NT file system. One example is that
there was a real time anti virus process running in the background, and if
this happened to "monitor" a file while I was trying to access and remove
the file, then an exception occurs. This knowledge means I can plan for it
my own way...

I've included a snippet of some work I did a while back...

// some data members (set with an initialisation method)
protected string pathToMonitor = "";
protected string fileSearchPattern = "*.xml";
protected Queue foundFiles = new Queue();
protected string fileToClear = "";
// ------------------------------------------------------------------------
// Receive Methods
// ------------------------------------------------------------------------

// call this to sit and wait for data, returns
OperationResult.Success
// with data containing one files contents.
public override OperationResult Receive( out string data )
{
data = "";
dataBufferToClear = "";

/*** Status.Listening is a static turned on and off by the start
up/ shut down methods of the windows service ***/
while ( Status.Listening )
{

//
// the queueing system, although a little more complex,
// ensures that the files are processed in some order
// according to when they arrived.
//

if ( foundFiles.Count == 0 )
{
// get all the files matching our criteria
string[] Files = Directory.GetFiles( pathToMonitor,
fileSearchPattern );

if ( Files != null )
{
foreach ( string fileName in Files )
{
foundFiles.Enqueue ( fileName );
}
}

/*** a method that checks Status.Listening stays true
while it waits for 5 seconds ***/
Methods.SleepItOff( 5 );

}
// if the queue is not empty, then there's new files
if ( foundFiles.Count == 0 )
{
continue;
}

string currentFilename = (string)foundFiles.Dequeue();

/*** GetFileContents simply opens the file, extracts the
data and closes the reader again ***/
data = Methods.GetFileContents(currentFilename);
//
// finally set the file for removal

fileToClear = currentFilename;

if ( data != null )
{
if ( data != "" )
{
dataBufferToClear = data;

/*** my own result enumerations ***/
return OperationResult.Success ;
}
}

Methods.SleepItOff(5);

}

return OperationResult.NoMessageFound;

}

// call this when you've successfully put the file into the database
// If an error occurs in the processing of a file, then you know
// that the file data is still there and hasn't been lost.
public override void CompleteReceive()
{
string data = Methods.GetFileContents(fileToClear);

if ( dataBufferToClear == "" || dataBufferToClear == data )
{
Methods.DeleteFile ( fileToClear );
}

}

Hope this helps.

Dan.
"Phil396" <an*******@discussions.microsoft.com> wrote in message
news:27****************************@phx.gbl...
I have a windows service that uses a filesystemwatcher
to wait for files and process them to a database.
Sometimes a large group of files will be cut and paste
for the filesystemwatcher to process. I ran into
trouble when trying to process a group of files
at a time.
The code below fixed my problem for about 15 large files
( around 100k each ). However when I tried to use it
on a group of small files ( 90 files ) it crashes (
around 6k each ). I am going to try increasing the time
for thread sleep. Any other solutions ?
Here is my code for the file system watcher using a
windows service to wait for files-

private void ProcessFiles(string FullPath)
{
while (true)
{
try
{
objFiles.Parse(FullPath);
break;
}
catch (System.IO.IOException)
{
System.Threading.Thread.Sleep( 1000 );
}
}

Nov 16 '05 #2

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

4
by: Josh Usovsky | last post by:
I'm setting up a watched folder using FileSystemWatcher. When I drop a small file into the watched folder, I can respond to a .Created event and process the file with other code. However, if I try...
2
by: Paul | last post by:
Hi, I've been developing an application in VB.NET that uses the FileSystemWatcher and a popup notification in order to tell me when files have been downloaded. The FileSystemWatcher code in...
7
by: Allen Anderson | last post by:
I'm trying to figure out a way to catch when a file has been written to a directory. I currently have it where I can catch when the file begins writing, but this isn't helpful as I need to know...
13
by: David | last post by:
I have been working on trying to write a directory watcher service. One of the requirments is that it be able to watch multiple directories, not sub directories of one parent directory, but just...
3
by: Stampede | last post by:
Hi, I write an application which waits for incomming files in a specified directory. I thought, that using the FileSystemWatcher would be the best, as it does exactly what I need. But now I have...
20
by: J-T | last post by:
We are working on an asp.net application which is a 3-tier application.I was aksed to create a component which monitors a folder and gets the file and pass them to a class library in our business...
2
by: kmcnet | last post by:
Hello Everyone and thanks for your help in advance. I have been battling a problem for nearly a month with the FileSystemWatcher component. Basically, what I am trying to do it to monitor three...
0
by: =?Utf-8?B?Sm9uIEphY29icw==?= | last post by:
..Net 2.0 in VB Watcher = New FileSystemWatcher() AddHandler Watcher.Changed, New FileSystemEventHandler(AddressOf Watcher_Changed) AddHandler Watcher.Created, New...
5
by: =?Utf-8?B?Sm9obiBT?= | last post by:
I am trying to find out if there is a way to tell if there is already a filesystemwatcher (created by a webservice) monitoring a folder. I have a webservice that creates a filesystemwatcher,...
0
by: ryjfgjl | last post by:
ExcelToDatabase: batch import excel into database automatically...
1
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 6 Mar 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:15 (7.15PM). In this month's session, we are pleased to welcome back...
0
by: Vimpel783 | last post by:
Hello! Guys, I found this code on the Internet, but I need to modify it a little. It works well, the problem is this: Data is sent from only one cell, in this case B5, but it is necessary that data...
0
by: jfyes | last post by:
As a hardware engineer, after seeing that CEIWEI recently released a new tool for Modbus RTU Over TCP/UDP filtering and monitoring, I actively went to its official website to take a look. It turned...
0
by: ArrayDB | last post by:
The error message I've encountered is; ERROR:root:Error generating model response: exception: access violation writing 0x0000000000005140, which seems to be indicative of an access violation...
1
by: PapaRatzi | last post by:
Hello, I am teaching myself MS Access forms design and Visual Basic. I've created a table to capture a list of Top 30 singles and forms to capture new entries. The final step is a form (unbound)...
1
by: CloudSolutions | last post by:
Introduction: For many beginners and individual users, requiring a credit card and email registration may pose a barrier when starting to use cloud servers. However, some cloud server providers now...
0
by: af34tf | last post by:
Hi Guys, I have a domain whose name is BytesLimited.com, and I want to sell it. Does anyone know about platforms that allow me to list my domain in auction for free. Thank you
0
by: Faith0G | last post by:
I am starting a new it consulting business and it's been a while since I setup a new website. Is wordpress still the best web based software for hosting a 5 page website? The webpages will be...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.