IOException File used by another process

[George]

Hi,

I am re-writing part of my application using C#. This application starts
another process which execute a "legacy" program. This legacy program writes
to a log file and before it ends, it writes a specific string to the log file.

My original program (MKS Toolkit shell program) which keeps running "grep"
checking the "exit string" on the "log files". There are no file sharing
problem.

However, when I use File.Open or StreamReader to open the log files,
IOException is raised. I can see why, since the log files are open for
writes.

I am certain that Toolkit program such as "grep" or C function such as
"fopen()" will not encounter this problem.

Please note that there is no way for me to make any code changes to the
legacy program which is writing to the log file continuously.

Is there a way for me to by pass this restriction in C#?

Thanks,


--
George

[Willy Denoyette [MVP]]

"George" <wa**@nospam.nospam> wrote in message
news:A4**********************************@microsof t.com...
| Hi,
|
| I am re-writing part of my application using C#. This application starts
| another process which execute a "legacy" program. This legacy program
writes
| to a log file and before it ends, it writes a specific string to the log
file.
|
| My original program (MKS Toolkit shell program) which keeps running "grep"
| checking the "exit string" on the "log files". There are no file sharing
| problem.
|
| However, when I use File.Open or StreamReader to open the log files,
| IOException is raised. I can see why, since the log files are open for
| writes.
|

Please post your code.

| I am certain that Toolkit program such as "grep" or C function such as
| "fopen()" will not encounter this problem.
|
| Please note that there is no way for me to make any code changes to the
| legacy program which is writing to the log file continuously.
|
| Is there a way for me to by pass this restriction in C#?
|

This has nothing to do with C#, file sharing is controlled by the OS
depending on the open and sharing modes specified during an open.

Willy.

[George]

Code:

It's fairly generic,

public System.Boolean ExitCheck(System.IO.DirectoryInfo
oBatchJobDirInfo)
{
System.IO.FileInfo[] logFileArray;
System.String line;
System.Boolean hasExit = false;
System.String pattern = @"SERVER: EXIT";

try
{
logFileArray = oBatchJobDirInfo.GetFiles("*.log");

foreach (System.IO.FileInfo oFileInfo in logFileArray)
{

using (System.IO.StreamReader sr = new
System.IO.StreamReader(oFileInfo.FullName))
{
line = sr.ReadLine();
if (line == null)
{
//end of file.
break;
}

if (System.Text.RegularExpressions.Regex.IsMatch(
line, pattern,
System.Text.RegularExpressions.RegexOptions.Ignore Case))
{
//exited.
hasExit = true;
break;
}
}
}

}
catch (Exception ex)
{
throw (ex);
}

return hasExit ;
}

This has nothing to do with C#, file sharing is controlled by the OS
depending on the open and sharing modes specified during an open.
Yes, and no. Correct me if I am wrong here, I thought

OS implements
1. Basic file system functionality such as fopen() or fclose() in C. No
file locking.

2. File locking functionalities which did not apear until newer OSs are
introduced. (I blieve older version of UNIX did not have file locking
implemented.)
3. (Possibly) File open/close functionalities which does file locking
automatically.

I think program such as notepad uses fopen() when opening a file without
checking against file locking. You can try open a "locked" file with
notepad, and I believe you can still open this locked file. "Wordpad" will
not do the same.

I assume, System.IO.File.Open() in C# exposes (3), where file locking are
done automatically.

What I am asking here is, if there is an equivalent function/method in C# to
C's fopen()?

Thanks,

--
George
"Willy Denoyette [MVP]" wrote:

"George" <wa**@nospam.nospam> wrote in message
news:A4**********************************@microsof t.com...
| Hi,
|
| I am re-writing part of my application using C#. This application starts
| another process which execute a "legacy" program. This legacy program
writes
| to a log file and before it ends, it writes a specific string to the log
file.
|
| My original program (MKS Toolkit shell program) which keeps running "grep"
| checking the "exit string" on the "log files". There are no file sharing
| problem.
|
| However, when I use File.Open or StreamReader to open the log files,
| IOException is raised. I can see why, since the log files are open for
| writes.
|

Please post your code.

| I am certain that Toolkit program such as "grep" or C function such as
| "fopen()" will not encounter this problem.
|
| Please note that there is no way for me to make any code changes to the
| legacy program which is writing to the log file continuously.
|
| Is there a way for me to by pass this restriction in C#?
|

This has nothing to do with C#, file sharing is controlled by the OS
depending on the open and sharing modes specified during an open.

Willy.

[Walter Wang [MSFT]]

Hi George,

Thank you for your post.

I think the cause of this problem is that you need to open the file with
flag FILE_SHARE_WRITE since another process is already opening the for
writing. Here's the description of this flag in MSDN:

Enables subsequent open operations on an object to request write
access.
Otherwise, other processes cannot open the object if they request write
access.
If this flag is not specified, but the object has been opened for write
access, the function fails.

In .NET, we can use FileStream to specify a FileShare mode:

* None
* Read
* ReadWrite
* Write

Hope this helps. Please feel free to post here if anything is unclear.

Regards,
Walter Wang (wa****@online.microsoft.com, remove 'online.')
Microsoft Online Community Support

==================================================
When responding to posts, please "Reply to Group" via your newsreader so
that others may learn and benefit from your issue.
==================================================

This posting is provided "AS IS" with no warranties, and confers no rights.

[Willy Denoyette [MVP]]

"George" <wa**@nospam.nospam> wrote in message
news:E8**********************************@microsof t.com...
| Code:
|
| It's fairly generic,
|
| public System.Boolean ExitCheck(System.IO.DirectoryInfo
| oBatchJobDirInfo)
| {
| System.IO.FileInfo[] logFileArray;
| System.String line;
| System.Boolean hasExit = false;
| System.String pattern = @"SERVER: EXIT";
|
| try
| {
| logFileArray = oBatchJobDirInfo.GetFiles("*.log");
|
| foreach (System.IO.FileInfo oFileInfo in logFileArray)
| {
|
| using (System.IO.StreamReader sr = new
| System.IO.StreamReader(oFileInfo.FullName))
| {
| line = sr.ReadLine();
| if (line == null)
| {
| //end of file.
| break;
| }
|
| if (System.Text.RegularExpressions.Regex.IsMatch(
| line, pattern,
| System.Text.RegularExpressions.RegexOptions.Ignore Case))
| {
| //exited.
| hasExit = true;
| break;
| }
| }
| }
|
| }
| catch (Exception ex)
| {
| throw (ex);
| }
|
| return hasExit ;
| }
|
|
| > This has nothing to do with C#, file sharing is controlled by the OS
| > depending on the open and sharing modes specified during an open.
|
| Yes, and no. Correct me if I am wrong here, I thought
|

I'm sorry, but you are wrong and confused, you are talking about file
"locking" and I'm talking about file sharing, both have things in common but
are not exactly the same!

| OS implements
| 1. Basic file system functionality such as fopen() or fclose() in C. No
| file locking.

Windows FS API's are implemented through the Win32 FS API's, the language IO
API's are just sitting in top of these Win32 API's (at the library level),
that means that whatever IO library you use (C library, STL, .NET etc), you
will always end by a call through Win32 for you file accesses on Windows.

|
| 2. File locking functionalities which did not apear until newer OSs are
| introduced. (I blieve older version of UNIX did not have file locking
| implemented.)

All Unix versions I know did have file locking implemented from the very
beginning. But lets talk about sharing because this is your real issue, if
you want to share file access across processes, you will have to open the
file for shared access, else the call will fail when one tries to open a
file, which is already open, in a mode that is not allowed by the process
who initially opened the file.
|
|
| 3. (Possibly) File open/close functionalities which does file locking
| automatically.
|
| I think program such as notepad uses fopen() when opening a file without
| checking against file locking.

Again, notepad doesn't open the file using fopen. Notepad is a Win32
application which uses the Win32 API's directly.
You can try open a "locked" file with
| notepad, and I believe you can still open this locked file.

Not sure here what exactly you mean with "locked" file, but if by this you
mean - a file which is opened for exclusive access - then you are wrong,
notepad will fail to open the file.
Notepad can only open a file which is:
- not yet open, or
- opened for shared read/write access.
Just write a small C# program that opens a text file with different sharing
modes and try to open the same file using notepad, you will understand what
I mean.

"Wordpad" will
| not do the same.
|
'fopen' opens the file with the access mode you specified during the call
(see later), the sharing mode is impicitely set to not shared. That means
that an fopen call will fail when the file is already open, be it by another
process or another thread in your own process.
| I assume, System.IO.File.Open() in C# exposes (3), where file locking are
| done automatically.
|

Again, file sharing not locking.

| What I am asking here is, if there is an equivalent function/method in C#
to
| C's fopen()?
|

The equivalent is File.Open("c:\\somefile", FileMode.Open, FileAccess.Read,
FileShare.None);

Note that you can set the access mode to Read, Write and ReadWrite, just
like you would do with fopen's +r, +rw, +w.

Now, what you are actually doing here:

| using (System.IO.StreamReader sr = new
| System.IO.StreamReader(oFileInfo.FullName))

is opening a file for read access, with the sharing mode set to read
(implicitely set by StreamReader).
In this case, when another process has previously opened the file for
exclusive access, you won't be able to open the same file again. If the
other process has openend the file with shared read, you will be able to
open the file for read access only, if the other process specified shared
write, you can only open the file for write access.

Willy.

[George]

Hi Willy,

Thank you for your reply. I ended up implementing a work around. However,
I would like to find out more by keeping the discussion with you. If my
writing seems "argumentative" or "accusive", my apology. I am trying to get
my points across.

OK. Here is some of my findings:

I wrote some sample codes.

Sample codes 1:

private void btnCmdPrompt_Click(object sender, EventArgs e)
{
System.IO.FileStream oFileStream;

try
{
oFileStream = new System.IO.FileStream(@"E:\tmp\abc.txt",
System.IO.FileMode.Create, System.IO.FileAccess.Write);

oFileStream.Close();

}
catch (Exception ex)
{
throw (ex);
}

return;
}

This simply opens a file for write, without specifying System.IO.FileShare
mode. I am able to open abc.txt (before Close(), I used break point) with
Notepad.exe. However, WordPad.exe does not allow access to the abc.txt.

Sample codes 2:
If I change the FileOpen() to include FileShare.Read mode,

oFileStream = new System.IO.FileStream(@"E:\tmp\abc.txt",
System.IO.FileMode.Create, System.IO.FileAccess.Write,
System.IO.FileShare.Read);

Exactly the same behaviour.
Sample codes 3:
If I change the FileOpen() to include FileShare.None mode,

oFileStream = new System.IO.FileStream(@"E:\tmp\abc.txt",
System.IO.FileMode.Create, System.IO.FileAccess.Write,
System.IO.FileShare.None);

Neither Notepad.exe and Wordpad.exe can open abc.txt.
I image the legacy program must have used file open code similar to Sample
Code 1. In this case Notepad.exe can still open abc.txt. My subsequent
File.Open() seems to behave more like Wordpad.exe.

What I wanted to find out was how can I simulate Notepad.exe's behaviour?
Back to UNIX for a bit. The way I used to lock files in UNIX was,

fd = open("abc.txt", O_CREAT);
flock(fd, LOCK_EX);
....

I mainly used flock() to lock a file and prvent certain sharings. Hence, I
do not understand your difference between "sharing" and "locking" for files.
(Not lock() in Threading which is for multi-threading locking) Of course,
flock() and NFS does not work too well together.
Feel free to give above sample a try and hopefully you will have the same
results. Any comments on why Notepad.exe and Wordpad.exe behave differently
are appreciated.

Thanks,

--
George

[George]

Unfortunately, the application which opens the file for writing is a legacy
3rd party program. Changing the codes will be very hard. It was compiled in
Visual Studio 6.0 anyway.
Please see my latest post about Notepad.exe Wordpad.exe differences. Any
comments are appreciated.

Thanks,
--
George
"Walter Wang [MSFT]" wrote:

Hi George,

Thank you for your post.

I think the cause of this problem is that you need to open the file with
flag FILE_SHARE_WRITE since another process is already opening the for
writing. Here's the description of this flag in MSDN:

Enables subsequent open operations on an object to request write
access.
Otherwise, other processes cannot open the object if they request write
access.
If this flag is not specified, but the object has been opened for write
access, the function fails.

In .NET, we can use FileStream to specify a FileShare mode:

* None
* Read
* ReadWrite
* Write

Hope this helps. Please feel free to post here if anything is unclear.

Regards,
Walter Wang (wa****@online.microsoft.com, remove 'online.')
Microsoft Online Community Support

==================================================
When responding to posts, please "Reply to Group" via your newsreader so
that others may learn and benefit from your issue.
==================================================

This posting is provided "AS IS" with no warranties, and confers no rights.

[George]

Unfortunately, the application which opens the file for writing is a legacy
3rd party program. Changing the codes will be very hard. It was compiled in
Visual Studio 6.0 anyway.
Please see my latest post about Notepad.exe Wordpad.exe differences. Any
comments are appreciated.

Thanks,
--
George
"Walter Wang [MSFT]" wrote:

Hi George,

Thank you for your post.

I think the cause of this problem is that you need to open the file with
flag FILE_SHARE_WRITE since another process is already opening the for
writing. Here's the description of this flag in MSDN:

Enables subsequent open operations on an object to request write
access.
Otherwise, other processes cannot open the object if they request write
access.
If this flag is not specified, but the object has been opened for write
access, the function fails.

In .NET, we can use FileStream to specify a FileShare mode:

* None
* Read
* ReadWrite
* Write

Hope this helps. Please feel free to post here if anything is unclear.

Regards,
Walter Wang (wa****@online.microsoft.com, remove 'online.')
Microsoft Online Community Support

==================================================
When responding to posts, please "Reply to Group" via your newsreader so
that others may learn and benefit from your issue.
==================================================

This posting is provided "AS IS" with no warranties, and confers no rights.

[Willy Denoyette [MVP]]

"George" <wa**@nospam.nospam> wrote in message
news:70**********************************@microsof t.com...
| Hi Willy,
|
| Thank you for your reply. I ended up implementing a work around.
However,
| I would like to find out more by keeping the discussion with you. If my
| writing seems "argumentative" or "accusive", my apology. I am trying to
get
| my points across.
|
| OK. Here is some of my findings:
|
| I wrote some sample codes.
|
| Sample codes 1:
|
| private void btnCmdPrompt_Click(object sender, EventArgs e)
| {
| System.IO.FileStream oFileStream;
|
| try
| {
| oFileStream = new System.IO.FileStream(@"E:\tmp\abc.txt",
| System.IO.FileMode.Create, System.IO.FileAccess.Write);
|
| oFileStream.Close();
|
| }
| catch (Exception ex)
| {
| throw (ex);
| }
|
| return;
| }
|
| This simply opens a file for write, without specifying System.IO.FileShare
| mode.

The FileStream's FileShare mode is Read by default.

I am able to open abc.txt (before Close(), I used break point) with
| Notepad.exe. However, WordPad.exe does not allow access to the abc.txt.
|

This is because Notepad at program start only needs to read the file
contents, so, it opens the file for Read Access and "Sharing" mode Read,
reads it's contents into an internal buffer and closes the file, result - no
sharing violation.
Just try to write to the file from withing Notepad, you will get a File
sharing violation because Notepad must open the file for write access in
order to flush the buffer, which obviously fails because another thread
(your progarm) has opened the file with sharing mode Read not Write or
ReadWrite.
WordPad on the other end opens the file for Read/Write access, this
obviously results in a sharing violation, because you opened the file for
shared read access only.
| Sample codes 2:
| If I change the FileOpen() to include FileShare.Read mode,
|
| oFileStream = new System.IO.FileStream(@"E:\tmp\abc.txt",
| System.IO.FileMode.Create, System.IO.FileAccess.Write,
| System.IO.FileShare.Read);
|
| Exactly the same behaviour.
|
|

This is normal, see above.

| Sample codes 3:
| If I change the FileOpen() to include FileShare.None mode,
|
| oFileStream = new System.IO.FileStream(@"E:\tmp\abc.txt",
| System.IO.FileMode.Create, System.IO.FileAccess.Write,
| System.IO.FileShare.None);
|
| Neither Notepad.exe and Wordpad.exe can open abc.txt.
|

This is normal as you opened the file for exclusive access (that is no
sharing possible).

|
| I image the legacy program must have used file open code similar to Sample
| Code 1. In this case Notepad.exe can still open abc.txt. My subsequent
| File.Open() seems to behave more like Wordpad.exe.
|

Don't see what you mean with "My subsequent ".

| What I wanted to find out was how can I simulate Notepad.exe's behaviour?
|

If the third party program has opened the file for "shared" Read only, all
you can do is open the file for Read access, the other program did not want
to share the file for Write.
|
| Back to UNIX for a bit. The way I used to lock files in UNIX was,
|
| fd = open("abc.txt", O_CREAT);
| flock(fd, LOCK_EX);
| ...
|

The same can be done in Windows, FileLock is the Win32 API which allows you
to lock a file or a file portion, but like I said the semantics of file
"locking" and file "Sharing" are different, sure you can lock the whole file
contents which results in nearly the same behavior as a ShareMode.None, but
locking (on Windows) is more dynamic as you can change the portion or unlock
the file without the need to close and re-open the file, something you can't
do with the Sharing mode.
The C run-time library has no flock (BSD) function, instead it has _locking,
but again the semantics are different, and as I said before, the libary
function calls into Win32 anyway, so _locking uses FileLock under the
covers.
| I mainly used flock() to lock a file and prvent certain sharings. Hence,
I
| do not understand your difference between "sharing" and "locking" for
files.

The Sharing mode attribute is effectively a form of file locking, but the
semantics aren't exactly the same. Sharing has much more control over what
kind of lock you want while flock (BSD) is limitted to applying locks on a
file (using an advisory lock) and is IMO not used for general file control
(lockf (System V) using fcntl, offers the same level of control on Unix).
| (Not lock() in Threading which is for multi-threading locking) Of course,
| flock() and NFS does not work too well together.
|
That's right, flock doesn't work with NFS.

Willy.

[Walter Wang [MSFT]]

Hi George,

Thank you for your update.

I'm sorry I didn't make myself very clear in my last post. What I mean is,
use the appropriate file sharing mode in your code, not to change the code
in the 3rd party program.

It's somewhat confusing in the documentation of CreateFile (win32) or
FileStream (.NET) about the file sharing mode. When a process first create
a file for writing, if it's not setting the file sharing mode to NONE, then
another process can open the file for reading, as long as it specify
FileShare.Write or FileShare.ReadWrite as sharing mode:

FileStream fs = new FileStream(@"c:\temp\1.txt", FileMode.Open,
FileAccess.Read, FileShare.ReadWrite);

And apparently the 3rd party program is not setting file sharing mode to
NONE since you've said that grep can read it.

Hope this helps. Please feel free to post here if anything is unclear.

Regards,
Walter Wang (wa****@online.microsoft.com, remove 'online.')
Microsoft Online Community Support

==================================================
When responding to posts, please "Reply to Group" via your newsreader so
that others may learn and benefit from your issue.
==================================================

This posting is provided "AS IS" with no warranties, and confers no rights.

[George]

Hi Willy,

Appreciate your explanation in the previous e-mail.

Since I am still encountering the same problem, I have written a simple C
program to simulate the legacy program that creates and writes to the log
file.

If I were to run the following program

<program nameE:\abc.txt w

It creates file E:\abc.txt for write and wait for a signal to close the file.

Before closing the file, I am

1. Unable to open a FileStream to read this file.

2. Unable to open this file using Wordpad.exe

3. However, I am able to open this file using notepad.exe.

On the other hand, if I run

<program nameE:\abc.txt r

I am able to

1. Open the File Stream reader

2. Open the file with Wordpad.exe

3. Open the file with Notepad.exe
The only thing I can conclude is that fopen(<filename>, "w"), which does not
have a FileShare formal parameter, is equivalent to (as far as file sharing
goes)

System.IO.FileStream(@"E:\abc.txt", System.IO.FileMode.Create,
System.IO.FileAccess.Write, System.IO.FileShare.None);

Hence, When I try to open a StreamReader() to a file opened by
fopen(<filename, "w"), it will always be denied because of the file sharing
problem.
If you have time and opportunity, please do give the following code a try
and see if you are getting the same results.
My next option is to use System.Runtime.InteropServices to use
fopen(<filename, "r") to open a log file for read.....

The program is pasted below.

Thanks,
----START---

#include <stdio.h>
#include <stdlib.h>
#include <io.h>
#include <string.h>

void usage(const char *progname)
{
fprintf(stdout, "Usage:\n\t%s%s\n",
progname, " <filename<r/w>");

return;
}

int main(int argc, char* argv[])
{
int err = 0;
char buf[100];
FILE *fp = NULL;
char *filename = NULL;


if (argc != 3)
{
usage(argv[0]);
err = 1;
goto cleanup;
}
filename = argv[1];
if (strcmp(argv[2], "w") == 0)
{
fprintf(stdout, "Open file for write ... ");
}
else if (strcmp(argv[2], "r") == 0)
{
fprintf(stdout, "Open file for read ... ");
}
else
{
fprintf(stderr, "Open option %s not available!\n", argv[2]);
err = 3;
goto cleanup;
}

fp = fopen(filename, argv[2]);
if (fp == NULL)
{
fprintf(stderr, "Error: Open file failed!\n");
err = 2;
goto cleanup;
}

//To stop
fprintf(stdout, "Type a character and press Enter to close the file ...\n");
fscanf(stdin, "%s", buf);
fclose(fp);
//To stop
fprintf(stdout, "File closed. Type a character and press Enter to exit
....\n");
fscanf(stdin, "%s", buf);

cleanup:

return err;
}

---END---
--
George

[Mike .]

George,

I'm experiencing the same issue while trying to monitor a specific file
which seems to be locked by its parent process. Trying to open a stream
with read only access even if it is trying to use FileShare.Read.

Your idea it use Interop may be a solution, although it seems more like
a hack.

Mike

*** Sent via Developersdex http://www.developersdex.com ***

[George]

Hi everyone,

Walter Wang provided additional clarification to his post on July 2 2006,
and I am able to resolve my problem. The problem was that I was not able to
open a log file for read if I use System.IO.StreamReader. The log file is
opened for writing by another process, most likely using fopen() in C.

The solution is to open the file using FileShare.ReadWrite (or
FileShare.Write. I have not tried this yet, but Walter has).

For example, if a log file currently opened for write and FileShare is not
set to NONE by the "writing" process, the subsequent process can open the
file for READ as

using (System.IO.StreamReader sr = new System.IO.StreamReader(
new System.IO.FileStream(trailFilename,
System.IO.FileMode.Open,
System.IO.FileAccess.Read,
System.IO.FileShare.ReadWrite)))
{
}

No IOException will be raised.

This does puzzle me and I would like to put it out here and see if anyone
can see through either the problem/bug or my mistaken logic:

1. FileShare field in FileStream should affect the subsquent opening rather
than current stream opening. Hence, why FileShare.ReadWrite works while
FileShare.Read does not?

2. If 1st process open the file for write, this process most likely does not
want additional file open for write by other processes. Hence,
FileShare.ReadWrite by subsequent process could have contradict the intention
of the first process. Hence specifying FileShare.Read for Read access does
not make sense to me at all.

Any comments are welcome.

Thanks.

--
George

[Walter Wang [MSFT]]

Hi George,

Thank you for your update and reply to my last mail.

I understand that this issue is difficult to understand according to the
MSDN Library. I say so because I was surprised by this behavior myself at
first.

However, I think you will have a better understanding of the issue after
reading Raymond Chen's explanation below:

#How do the FILE_SHARE_* bits interact with the desired access bits?
http://blogs.msdn.com/oldnewthing/ar...11/129759.aspx

Hope this helps. Please feel free to post here if anything is unclear.
Regards,
Walter Wang (wa****@online.microsoft.com, remove 'online.')
Microsoft Online Community Support

==================================================
When responding to posts, please "Reply to Group" via your newsreader so
that others may learn and benefit from your issue.
==================================================

This posting is provided "AS IS" with no warranties, and confers no rights.