Wrap command shell in System.Diagnostics.Process

[Jon Davis]

Does anyone know why using System.Diagnostics.Process to "wrap" a console
application does not always transmit the I/O, depending on what processes
you're trying to "consume"? PowerShell, for example, does not seem to
process any I/O through the Process object.

I know that in the case of PowerShell there are better ways to "wrap" the
console by directly interfacing with the assemblies of
System.Management.Automation or some similarly named namespace, but I'm
trying to use generic command line wrappers for multiple types of processes
that use the console I/O, and PowerShell was a handy example of why this
won't work.

ProcessStartInfo psi = new ProcessStartInfo(
@"C:\Windows\system32\WindowsPowerShell\v1.0\power shell.exe");
psi.Arguments = "-NoLogo";
psi.UseShellExecute = false;
psi.CreateNoWindow = true;
psi.RedirectStandardInput = true;
psi.RedirectStandardOutput = true;
//psi.RedirectStandardError = true;
Process process = new Process();
process.StartInfo = psi;
bool started = process.Start();
if (started)
{
process.StandardInput.WriteLine("2+2");
process.StandardInput.Flush();
string ret = process.StandardOutput.ReadLine(); // <-- stalls here
System.Console.WriteLine("PowerShell says \"2+2=" + ret + "\".");
}
Another one I was trying to "wrap" was the original implementation of Dave
Raggett's HTML Tidy. The stuff below sometimes stalls on ReadToEnd(). It
seemed to always stall until I added "process.StandardInput.Close();" after
"process.StandardInput.Flush();" but it still stalls on ReadToEnd() half the
time.

...
_TidyProcessStartInfo = new ProcessStartInfo(
Directory.GetParent(System.Reflection.Assembly.Get ExecutingAssembly().Location)
+ "\\tidy.exe");
_TidyProcessStartInfo.UseShellExecute = false;
_TidyProcessStartInfo.CreateNoWindow = true;
_TidyProcessStartInfo.RedirectStandardInput = true;
_TidyProcessStartInfo.RedirectStandardOutput = true;
_TidyProcessStartInfo.RedirectStandardError = true;
.. . .
Process process = new Process();
process.StartInfo = _TidyProcessStartInfo;
process.ErrorDataReceived += new
DataReceivedEventHandler(Exe_ErrorDataReceived);
bool started = process.Start();
if (started)
{
//process.StandardInput.AutoFlush = true;
process.StandardInput.WriteLine(input);
process.StandardInput.Flush();
process.StandardInput.Close();
ret = process.StandardOutput.ReadToEnd();
}

Thanks,
Jon

[Chris Dunaway]

On Apr 5, 12:20 pm, "Jon Davis" <[e-mail]>
wrote:

Does anyone know why using System.Diagnostics.Process to "wrap" a console
application does not always transmit the I/O, depending on what processes
you're trying to "consume"? PowerShell, for example, does not seem to
process any I/O through the Process object.

I know that in the case of PowerShell there are better ways to "wrap" the
console by directly interfacing with the assemblies of
System.Management.Automation or some similarly named namespace, but I'm
trying to use generic command line wrappers for multiple types of processes
that use the console I/O, and PowerShell was a handy example of why this
won't work.

ProcessStartInfo psi = new ProcessStartInfo(
@"C:\Windows\system32\WindowsPowerShell\v1.0\power shell.exe");
psi.Arguments = "-NoLogo";
psi.UseShellExecute = false;
psi.CreateNoWindow = true;
psi.RedirectStandardInput = true;
psi.RedirectStandardOutput = true;
//psi.RedirectStandardError = true;
Process process = new Process();
process.StartInfo = psi;
bool started = process.Start();
if (started)
{
process.StandardInput.WriteLine("2+2");
process.StandardInput.Flush();
string ret = process.StandardOutput.ReadLine(); // <-- stalls here
System.Console.WriteLine("PowerShell says \"2+2=" + ret + "\".");

}

Another one I was trying to "wrap" was the original implementation of Dave
Raggett's HTML Tidy. The stuff below sometimes stalls on ReadToEnd(). It
seemed to always stall until I added "process.StandardInput.Close();" after
"process.StandardInput.Flush();" but it still stalls on ReadToEnd() half the
time.

..
_TidyProcessStartInfo = new ProcessStartInfo(
Directory.GetParent(System.Reflection.Assembly.Get ExecutingAssembly().Location)
+ "\\tidy.exe");
_TidyProcessStartInfo.UseShellExecute = false;
_TidyProcessStartInfo.CreateNoWindow = true;
_TidyProcessStartInfo.RedirectStandardInput = true;
_TidyProcessStartInfo.RedirectStandardOutput = true;
_TidyProcessStartInfo.RedirectStandardError = true;
. . .
Process process = new Process();
process.StartInfo = _TidyProcessStartInfo;
process.ErrorDataReceived += new
DataReceivedEventHandler(Exe_ErrorDataReceived);
bool started = process.Start();
if (started)
{
//process.StandardInput.AutoFlush = true;
process.StandardInput.WriteLine(input);
process.StandardInput.Flush();
process.StandardInput.Close();
ret = process.StandardOutput.ReadToEnd();

}

Thanks,
Jon

You are probably hitting a deadlock issue when redirecting out. See
the docs on the RedirectStandardOutput property for more information:

http://msdn2.microsoft.com/en-us/lib...ardoutput.aspx

Chris

[Jon Davis]

"Chris Dunaway" <du******@gmail.comwrote in message
news:11**********************@n59g2000hsh.googlegr oups.com...

You are probably hitting a deadlock issue when redirecting out. See
the docs on the RedirectStandardOutput property for more information:

http://msdn2.microsoft.com/en-us/lib...ardoutput.aspx

Chris

Clearly it is deadlocking, the problem is I don't know how to apply any
workaround to the code I provided in the OP given the samples provided in
the referenced link, which although appears detailed at first glance is
really too brief to be useful.

Perhaps someone has the time and patience to make my two samples work? :)

Jon

[Peter Duniho]

On Thu, 05 Apr 2007 13:24:44 -0700, Jon Davis
<jo*@REMOVE.ME.PLEASE.jondavis.netwrote:

Clearly it is deadlocking, the problem is I don't know how to apply any
workaround to the code I provided in the OP given the samples provided in
the referenced link, which although appears detailed at first glance is
really too brief to be useful.

Perhaps someone has the time and patience to make my two samples work? :)

Most of us probably don't have the associated programs you're running
installed. I know I don't.

That said, reading the documentation it occurs to me that you may be
running into a problem with your processing of the StandardError stream.
That is, I don't see anything in your code that would read from that
stream. According to the documentation, if your own application does not
keep up with reading data from the streams, the child process may block
once the stream's buffer is full. That would prevent your child process
from continuing, while your parent process sits there waiting to read more
from the other stream.

If you are going to redirect both StandardError and StandardOutput, it
seems to me that the only robust way to do that is to ensure that both
streams are being read from simultaneously. You can do that by providing
for asychronous reading of at least one of them (ie BeginOutputReadLine
and/or BeginErrorReadLine). That way, you can ensure that you will always
be reading from both and not preventing the child process from
continuing. Alternatively, use two different threads to read from the two
streams.

Both of these techniques are documented near the end of the "Remarks"
section for the RedirectStandardOutput property in MSDN (the link you say
is "too brief to be useful").

Pete

[Peter Duniho]

In addition to my previous post...

Obviously if you do not actually need to read the StandardError stream,
then perhaps the simplest solution is to not redirect that stream in the
first place. :)

[Jon Davis]

StandardError was commented out, to avoid that exact scenario.

Jon

"Peter Duniho" <Np*********@nnowslpianmk.comwrote in message
news:op***************@petes-computer.local...

On Thu, 05 Apr 2007 13:24:44 -0700, Jon Davis
<jo*@REMOVE.ME.PLEASE.jondavis.netwrote:

>Clearly it is deadlocking, the problem is I don't know how to apply any
workaround to the code I provided in the OP given the samples provided in
the referenced link, which although appears detailed at first glance is
really too brief to be useful.

Perhaps someone has the time and patience to make my two samples work? :)

Most of us probably don't have the associated programs you're running
installed. I know I don't.

That said, reading the documentation it occurs to me that you may be
running into a problem with your processing of the StandardError stream.
That is, I don't see anything in your code that would read from that
stream. According to the documentation, if your own application does not
keep up with reading data from the streams, the child process may block
once the stream's buffer is full. That would prevent your child process
from continuing, while your parent process sits there waiting to read more
from the other stream.

If you are going to redirect both StandardError and StandardOutput, it
seems to me that the only robust way to do that is to ensure that both
streams are being read from simultaneously. You can do that by providing
for asychronous reading of at least one of them (ie BeginOutputReadLine
and/or BeginErrorReadLine). That way, you can ensure that you will always
be reading from both and not preventing the child process from
continuing. Alternatively, use two different threads to read from the two
streams.

Both of these techniques are documented near the end of the "Remarks"
section for the RedirectStandardOutput property in MSDN (the link you say
is "too brief to be useful").

Pete

[Peter Duniho]

On Thu, 05 Apr 2007 17:10:33 -0700, Jon Davis
<jo*@REMOVE.ME.PLEASE.jondavis.netwrote:

StandardError was commented out, to avoid that exact scenario.

Not in the second example you gave.

In any case, you don't have a sample that anyone else can use to try to
reproduce your problem, so it's not possible for anyone to look directly
at what's going on. In your first example, you use ReadLine which will
block until there's a line to be read. Maybe that line never comes. In
your second example, you use ReadToEnd which cannot complete if the child
process gets blocked itself. Since RedirectStandardError isn't commented
out, it's entirely possible that's your deadlock there.

Absent a minimal-but-complete sample of code that reliably reproduces the
problem, there may not be any better advice you can get. The best method
IMHO would be to run both processes under a debugger and just look to see
what they are waiting on when things appear to hang. However, I still
can't figure out how to get VS2005's debugger to allow me to debug threads
the way previous versions did, so I'm a bit hesitant to suggest that,
since it might not be as useful advice as it initially seems. :)

Pete

[Jon Davis]

"Peter Duniho" <Np*********@nnowslpianmk.comwrote in message
news:op***************@petes-computer.local...

In any case, you don't have a sample that anyone else can use to try to
reproduce your problem, so it's not possible for anyone to look directly
at what's going on.

Lots of people have PowerShell, it's a Windows Update piece.

Just swap out PowerShell for CMD.exe:

using System;
using System.Collections.Generic;
using System.Diagnostics;
public class MyClass
{
public static void Main()
{
try {
ProcessStartInfo psi = new ProcessStartInfo(
@"C:\Windows\system32\cmd.exe");
psi.UseShellExecute = false;
psi.CreateNoWindow = true;
psi.RedirectStandardInput = true;
psi.RedirectStandardOutput = true;
//psi.RedirectStandardError = true;
Process process = new Process();
process.StartInfo = psi;
bool started = process.Start();
if (started)
{
process.StandardOutput.ReadLine(); // "Microsoft Windows"
process.StandardOutput.ReadLine(); // "Copyright Microsoft"
process.StandardOutput.ReadLine(); // [blank line following logo]
process.StandardOutput.ReadLine(); // [command entry (echo)]
process.StandardInput.WriteLine("echo Blah");
process.StandardInput.Flush();
string ret = process.StandardOutput.ReadLine(); // <-- stalls here
System.Console.WriteLine("CMD.exe says " + ret + "\".");
}
} catch (Exception e) {
}
RL();
}

}

[Peter Duniho]

On Fri, 06 Apr 2007 10:11:42 -0700, Jon Davis
<jo*@REMOVE.ME.PLEASE.jondavis.netwrote:

Lots of people have PowerShell, it's a Windows Update piece.

Just swap out PowerShell for CMD.exe:

I don't know what you mean by "Windows Update piece". I have Windows
Update, use it all the time, but don't have PowerShell installed.

In any case, thank you for the sample code. On my computer, it runs just
fine as long as I remove the extra ReadLine you've got (the one commented
"command entry (echo)").

For what it's worth, all I did was break in the debugger when the program
got blocked, and saw that it was waiting at that line which showed that
you never got to the point of writing to the input stream, meaning you
were reading too many "discardable" lines up front.

Of course, one of the things this exercise illustrates is the fragility of
the approach you're using. Even if it works fine now (as it does on my
computer...can't say whether that's true on yours), you're tied to a
precise sequence of lines of input and output. A trivial bug in your own
code causes the whole thing to just stop, and of course if there are any
changes to the external console application that can really mess things up
too (either by making your application just stop in the wrong place, or
perhaps worse by causing your application to send the wrong commands to
the process).

Granted, this is all just sample code so who knows whether this is really
how you're doing things. But if it is, beware. Many pitfalls lie ahead.

Pete

[Jon Davis]

"Peter Duniho" <Np*********@nnowslpianmk.comwrote in message
news:op***************@petes-computer.local...

On Fri, 06 Apr 2007 10:11:42 -0700, Jon Davis
<jo*@REMOVE.ME.PLEASE.jondavis.netwrote:

>Lots of people have PowerShell, it's a Windows Update piece.

Just swap out PowerShell for CMD.exe:

I don't know what you mean by "Windows Update piece". I have Windows
Update, use it all the time, but don't have PowerShell installed.

In any case, thank you for the sample code. On my computer, it runs just
fine as long as I remove the extra ReadLine you've got (the one commented
"command entry (echo)").

For what it's worth, all I did was break in the debugger when the program
got blocked, and saw that it was waiting at that line which showed that
you never got to the point of writing to the input stream, meaning you
were reading too many "discardable" lines up front.

I added them one by one as I read the ReadLine() which returned:

- two lines of the logo
- an empty line
- the echo of my "echo" command, BEFORE it gets executed.

Are you sure that what you commented out is not the command itself being
echoed rather than the execution of the ECHO command? Perhaps we have
different versions.

Of course, one of the things this exercise illustrates is the fragility of
the approach you're using. Even if it works fine now (as it does on my
computer...can't say whether that's true on yours), you're tied to a
precise sequence of lines of input and output.

Understood, but while generally such an interface would be useless as a
generic console app solution I do have some precise sequences in mind, or as
in the case of Tidy and some other EXEs I'm dealing with a constant of
either input one command and get back one line of output (ReadLine()), input
one command and get back several lines of results (ReadToEnd()), or input
multiple lines as a single input and read back the output. I am not dealing
with processes that have a dialogue of more than one input or more than one
string of output.

... beware. Many pitfalls lie ahead.

Totally. I would still like to build out my process wrappers described above
and I'm still not fully understanding what I'm missing.

Jon

[Yvan Ross]

code is in C++ but will be easily translate to C#

My test were base on part of code found at following address.

http://msdn2.microsoft.com/en-us/lib...nthandler.aspx

// my process

WorkingProcess->StartInfo->RedirectStandardOutput = true;
WorkingProcess->OutputDataReceived += gcnew DataReceivedEventHandler( ProcessOutputDataHandler ); strServiceOutput= gcnew System::Text::StringBuilder;

// my data received event handler
private: static void ProcessOutputDataHandler( Object^ sendingProcess, DataReceivedEventArgs^ Line ) {
if ( !String::IsNullOrEmpty( Line->Data ) ) {
strServiceOutput->AppendFormat ( "\n {0}",Line->Data);
}
}

// my process exit event handler instead of using waitforexit()

WorkingProcess->Exited += gcnew System::EventHandler(workingProcess_Exited);

//where the event exit look like

private: static System::Void workingProcess_Exited(System::Object ^ sender, System::EventArgs ^ e)
{
System::Diagnostics::Process^ oProcessInfo = dynamic_cast<System::Diagnostics::Process^>( sender);

System::Int16 oExitCode = oProcessInfo->ExitCode; oProcessInfo->Close(); // will flush output and error stream to variables strServiceXXXX

strMessage = String::Concat(strServiceOutput->ToString()," " ,strServiceError->ToString());

}
I was still missing some console message until I close the process before reading. Closing the process fluh it by calling the DataReceivedEventHandler event.

If you test in debugger put your breadpoint after strMessage otherwise you will have timing issue between the process and the event handler.

Hope this will help you.

EggHeadCafe.com - .NET Developer Portal of Choice
http://www.eggheadcafe.com

[Jon Davis]

Thanks for your help Yvan.

Jon

<Yvan Rosswrote in message news:20***************@arthrovision.biz...

code is in C++ but will be easily translate to C#

My test were base on part of code found at following address.

http://msdn2.microsoft.com/en-us/lib...nthandler.aspx

// my process

WorkingProcess->StartInfo->RedirectStandardOutput = true;
WorkingProcess->OutputDataReceived += gcnew DataReceivedEventHandler(
ProcessOutputDataHandler ); strServiceOutput= gcnew
System::Text::StringBuilder;

// my data received event handler
private: static void ProcessOutputDataHandler( Object^ sendingProcess,
DataReceivedEventArgs^ Line ) {
if ( !String::IsNullOrEmpty( Line->Data ) ) {
strServiceOutput->AppendFormat ( "\n {0}",Line->Data);
}
}

// my process exit event handler instead of using waitforexit()

WorkingProcess->Exited += gcnew
System::EventHandler(workingProcess_Exited);

//where the event exit look like

private: static System::Void workingProcess_Exited(System::Object ^
sender, System::EventArgs ^ e)
{
System::Diagnostics::Process^ oProcessInfo =
dynamic_cast<System::Diagnostics::Process^>( sender);

System::Int16 oExitCode = oProcessInfo->ExitCode; oProcessInfo->Close();
// will flush output and error stream to variables strServiceXXXX

strMessage = String::Concat(strServiceOutput->ToString()," "
,strServiceError->ToString());

}
I was still missing some console message until I close the process before
reading. Closing the process fluh it by calling the
DataReceivedEventHandler event.

If you test in debugger put your breadpoint after strMessage otherwise you
will have timing issue between the process and the event handler.

Hope this will help you.

EggHeadCafe.com - .NET Developer Portal of Choice
http://www.eggheadcafe.com