FileSystemWatcher Problem...

[Stampede]

Hi,
I write an application which waits for incomming files in a specified
directory. I thought, that using the FileSystemWatcher would be the best, as
it does exactly what I need.
But now I have a problem:
I wonna get shure, that really all files are processed and I need to be
shure, that no file is processed twice.
If my application is started while there are already files in the directory
(which is realistic, as the application could be restarted, but the
application which produces the files may not be stopped at the same time). So
I thouhgt about just reading all the files which are already in the directory
and then starting the FileSystemWatcher for the new files which are arriving.
But what if a file arrives while I'm processing the old ones? So I thought
about first starting the FileSystemWatcher and then processing the old files.
But know I have the problem, that between starting the a file could be
processed twice (in a worst case cenario, where a new file arrives between
starting the FileSytemWatcher and reading the old files).

Maybe I'm just thinking to much, but is there any clean way to solve the
problem? Can I use the FileSystemWatcher instance for getting the old files
somehow?

The only solution I came up with, is to forget about the filesystemwatcher
and use a timer which checks for new files. But as there may be a longer time
period with no new files it would not be the best solution, because I would
have to let the timer look for new files about every 3 to 5 seconds, as some
of the files are time critical.

Maybe someone of you has a solution for my worst case thinking.

greetings

Florian

[Lasse Vågsæther Karlsen]

Stampede wrote:

Hi,
I write an application which waits for incomming files in a specified

<snip>

Does the files have unique filenames ?

If they do, here's how I would do it:

1. Start FileSystemWatcher
2. Grab all files in directory and process them, add each to a list
3. Grab new files through FSW, check if they are already in the list

The list, if it grows too big, could be killed a short while after
starting the app as it would probably only be needed to solve the issue
about processing a file twice due to the time window from starting FSW
to grabbing the list of files.

If the files don't have duplicate names it means that they are
overwritten and that has a whole host of other problems.

--
Lasse Vågsæther Karlsen
http://www.vkarlsen.no/
mailto:la***@vkarlsen.no
PGP KeyID: 0x2A42A1C2

[TaylorMichaelL]

FSW can be difficult to get use to. Unfortunately your goal of not getting
duplicate files isn't possible. You are going to have to handle this
contingency. The FSW can raise multiple events for the same file so you are
going to have to deal with it anyway. For example if you create a new file
in notepad and then save it you'll get at least a create notification and a
change notification. If the file is complex enough to warrant multiple saves
then you'll get multiple change notifications. I work around this solution
by either keeping track of the files that I have already processed or by
simply reprocessing them anyway (unless it would cause problems). Take as an
example a file replication program that copies files from one folder to
another. If I should happen to copy a file twice it really won't matter that
much. The only downside is the wasted cycles but since I would be running on
an arbitrary thread anyway it wouldn't matter.

As for the startup order I would recommend starting FSW first and then back
process the files. That way you won't miss any files. Otherwise there is
always the chance that a file could come in between the time you get the list
of files and start the FSW.

Michael Taylor - 9/5/05

"Stampede" wrote:

Hi,
I write an application which waits for incomming files in a specified
directory. I thought, that using the FileSystemWatcher would be the best, as
it does exactly what I need.
But now I have a problem:
I wonna get shure, that really all files are processed and I need to be
shure, that no file is processed twice.
If my application is started while there are already files in the directory
(which is realistic, as the application could be restarted, but the
application which produces the files may not be stopped at the same time). So
I thouhgt about just reading all the files which are already in the directory
and then starting the FileSystemWatcher for the new files which are arriving.
But what if a file arrives while I'm processing the old ones? So I thought
about first starting the FileSystemWatcher and then processing the old files.
But know I have the problem, that between starting the a file could be
processed twice (in a worst case cenario, where a new file arrives between
starting the FileSytemWatcher and reading the old files).

Maybe I'm just thinking to much, but is there any clean way to solve the
problem? Can I use the FileSystemWatcher instance for getting the old files
somehow?

The only solution I came up with, is to forget about the filesystemwatcher
and use a timer which checks for new files. But as there may be a longer time
period with no new files it would not be the best solution, because I would
have to let the timer look for new files about every 3 to 5 seconds, as some
of the files are time critical.

Maybe someone of you has a solution for my worst case thinking.

greetings

Florian

[Ludovic SOEUR]

There is another problem with FSW...if too many files are changed at the
same time (for ex, the moving of a directory containing 10000 files), the
FSW queue will be important. When it has not enough space (have a look to
MSDN to know the numbers), it can not store new events and they will be
lost...
To be sure to catch all events, you must start FSW BEFORE getting the first
list of files. Deal with events to manage your list but try to identify when
FSW has to many events to deal with. In that case, you have to update your
entire file list.

Hope it helps,

Ludovic Soeur.

"Stampede" <St******@discussions.microsoft.com> a écrit dans le message de
news:31**********************************@microsof t.com...

Hi,
I write an application which waits for incomming files in a specified
directory. I thought, that using the FileSystemWatcher would be the best, as it does exactly what I need.
But now I have a problem:
I wonna get shure, that really all files are processed and I need to be
shure, that no file is processed twice.
If my application is started while there are already files in the directory (which is realistic, as the application could be restarted, but the
application which produces the files may not be stopped at the same time). So I thouhgt about just reading all the files which are already in the directory and then starting the FileSystemWatcher for the new files which are arriving. But what if a file arrives while I'm processing the old ones? So I thought
about first starting the FileSystemWatcher and then processing the old files. But know I have the problem, that between starting the a file could be
processed twice (in a worst case cenario, where a new file arrives between
starting the FileSytemWatcher and reading the old files).

Maybe I'm just thinking to much, but is there any clean way to solve the
problem? Can I use the FileSystemWatcher instance for getting the old files somehow?

The only solution I came up with, is to forget about the filesystemwatcher
and use a timer which checks for new files. But as there may be a longer time period with no new files it would not be the best solution, because I would have to let the timer look for new files about every 3 to 5 seconds, as some of the files are time critical.

Maybe someone of you has a solution for my worst case thinking.

greetings

Florian