Read/Write from/to a process

[jas]

Hi,
I would like to start a new process and be able to read/write from/to
it. I have tried things like...

import subprocess as sp
p = sp.Popen("cmd.exe", stdout=sp.PIPE)
p.stdin.write("hostname\n")

however, it doesn't seem to work. I think the cmd.exe is catching it.

I also tried
f = open("out.txt", "w")
sys.stdout = f
os.system("cmd.exe")

...but out.txt didn't contain any output from cmd.exe

So, how can I create a process (in this case, cmd.exe) on Windows and
be able to read/write from/to it?

Thanks

[Dennis Lee Bieber]

On 24 Oct 2005 07:20:42 -0700, "jas" <co*******@gmail.com> declaimed the
following in comp.lang.python:

Hi,
I would like to start a new process and be able to read/write from/to
it. I have tried things like...

import subprocess as sp
p = sp.Popen("cmd.exe", stdout=sp.PIPE)
p.stdin.write("hostname\n")

however, it doesn't seem to work. I think the cmd.exe is catching it.

One: you didn't read any of the "returned" output...

Two: the output only seems to be available upon EOF, which means the
spawned command processor has to exit first... Though you CAN read one
character at a time, and then have to build lines and expected prompt
strings...
This seems to work:
-=-=-=-=-=-=-=-=-
import subprocess
import os

PROMPT = os.getcwd() + ">"

def getLine(proc):
ld = []
while True:
c = proc.stdout.read(1)
if c == "\r": continue #skip Windows <cr>
if c != "\n": ld.append(c) #save all but <lf>
if c is None or c == "\n" or c == ">": break
ln = "".join(ld)
return ln

p = subprocess.Popen("cmd.exe", stdout=subprocess.PIPE, stdin =
subprocess.PIPE)

print "**** START of captured output"
while True:
ln = getLine(p)
print ln
if ln == PROMPT: break
print "**** END of captured output"

p.stdin.write("ipconfig\n")
print "**** START of captured output"
while True:
ln = getLine(p)
print ln
if ln == PROMPT: break
print "**** END of captured output"

p.stdin.write("dir\n")
print "**** START of captured output"
while True:
ln = getLine(p)
print ln
if ln == PROMPT: break
print "**** END of captured output"

p.stdin.write("exit\n")
-=-=-=-=-=-=-=-=-=-
E:\UserData\Dennis Lee Bieber\My Documents>python script1.py
**** START of captured output
Microsoft Windows XP [Version 5.1.2600]
(C) Copyright 1985-2001 Microsoft Corp.

E:\UserData\Dennis Lee Bieber\My Documents>
**** END of captured output
**** START of captured output
ipconfig

Windows IP Configuration
Ethernet adapter Local Area Connection:

Connection-specific DNS Suffix . :
IP Address. . . . . . . . . . . . : 192.168.1.100
Subnet Mask . . . . . . . . . . . : 255.255.255.0
IP Address. . . . . . . . . . . . : fe80::211:11ff:fee1:f303%4
Default Gateway . . . . . . . . . : 192.168.1.1

Tunnel adapter Teredo Tunneling Pseudo-Interface:

Connection-specific DNS Suffix . :
IP Address. . . . . . . . . . . . :
3ffe:831f:4004:1956:0:fbde:bd0a:e50b
IP Address. . . . . . . . . . . . : fe80::5445:5245:444f%5
Default Gateway . . . . . . . . . : ::

Tunnel adapter Automatic Tunneling Pseudo-Interface:

Connection-specific DNS Suffix . :
IP Address. . . . . . . . . . . . : fe80::5efe:192.168.1.100%2
Default Gateway . . . . . . . . . :

E:\UserData\Dennis Lee Bieber\My Documents>
**** END of captured output
**** START of captured output
dir
Volume in drive E is Data
Volume Serial Number is 2626-D991

Directory of E:\UserData\Dennis Lee Bieber\My Documents

10/24/2005 09:23 AM <DIR>

[jas]

Thanks, that is certainly a start. As you mentioned, the "cd" could is
an issue.

Perhaps checking to see if the line ends with ">" is sufficient?

Dennis Lee Bieber wrote:

On 24 Oct 2005 07:20:42 -0700, "jas" <co*******@gmail.com> declaimed the
following in comp.lang.python:

Hi,
I would like to start a new process and be able to read/write from/to
it. I have tried things like...

import subprocess as sp
p = sp.Popen("cmd.exe", stdout=sp.PIPE)
p.stdin.write("hostname\n")

however, it doesn't seem to work. I think the cmd.exe is catching it.

One: you didn't read any of the "returned" output...

Two: the output only seems to be available upon EOF, which means the
spawned command processor has to exit first... Though you CAN read one
character at a time, and then have to build lines and expected prompt
strings...
This seems to work:
-=-=-=-=-=-=-=-=-
import subprocess
import os

PROMPT = os.getcwd() + ">"

def getLine(proc):
ld = []
while True:
c = proc.stdout.read(1)
if c == "\r": continue #skip Windows <cr>
if c != "\n": ld.append(c) #save all but <lf>
if c is None or c == "\n" or c == ">": break
ln = "".join(ld)
return ln

p = subprocess.Popen("cmd.exe", stdout=subprocess.PIPE, stdin =
subprocess.PIPE)

print "**** START of captured output"
while True:
ln = getLine(p)
print ln
if ln == PROMPT: break
print "**** END of captured output"

p.stdin.write("ipconfig\n")
print "**** START of captured output"
while True:
ln = getLine(p)
print ln
if ln == PROMPT: break
print "**** END of captured output"

p.stdin.write("dir\n")
print "**** START of captured output"
while True:
ln = getLine(p)
print ln
if ln == PROMPT: break
print "**** END of captured output"

p.stdin.write("exit\n")
-=-=-=-=-=-=-=-=-=-
E:\UserData\Dennis Lee Bieber\My Documents>python script1.py
**** START of captured output
Microsoft Windows XP [Version 5.1.2600]
(C) Copyright 1985-2001 Microsoft Corp.

E:\UserData\Dennis Lee Bieber\My Documents>
**** END of captured output
**** START of captured output
ipconfig

Windows IP Configuration
Ethernet adapter Local Area Connection:

Connection-specific DNS Suffix . :
IP Address. . . . . . . . . . . . : 192.168.1.100
Subnet Mask . . . . . . . . . . . : 255.255.255.0
IP Address. . . . . . . . . . . . : fe80::211:11ff:fee1:f303%4
Default Gateway . . . . . . . . . : 192.168.1.1

Tunnel adapter Teredo Tunneling Pseudo-Interface:

Connection-specific DNS Suffix . :
IP Address. . . . . . . . . . . . :
3ffe:831f:4004:1956:0:fbde:bd0a:e50b
IP Address. . . . . . . . . . . . : fe80::5445:5245:444f%5
Default Gateway . . . . . . . . . : ::

Tunnel adapter Automatic Tunneling Pseudo-Interface:

Connection-specific DNS Suffix . :
IP Address. . . . . . . . . . . . : fe80::5efe:192.168.1.100%2
Default Gateway . . . . . . . . . :

E:\UserData\Dennis Lee Bieber\My Documents>
**** END of captured output
**** START of captured output
dir
Volume in drive E is Data
Volume Serial Number is 2626-D991

Directory of E:\UserData\Dennis Lee Bieber\My Documents

10/24/2005 09:23 AM <DIR>
.
10/24/2005 09:23 AM <DIR>
..
07/25/2005 10:39 PM <DIR>
.metadata
10/06/2005 09:54 AM <DIR>
Ada Progs
08/13/2005 02:01 PM <DIR>
Agent Data
10/21/2005 09:29 AM 421,820 apress_offer.pdf
07/03/2005 11:36 AM 132 cp.py
07/17/2005 12:25 PM <DIR>
Cyberlink
07/06/2005 09:32 AM 102,400 db1.mdb
07/26/2005 12:20 AM 26,614 eclipse_code.xml
10/24/2005 01:08 AM <DIR>
Eudora
06/24/2005 08:50 PM 667 fake_oosums.ads
06/24/2005 08:50 PM 695 fake_oosums.ali
09/06/2005 09:01 PM <DIR>
Genealogy
07/13/2005 10:56 PM <DIR>
HomeSite
05/08/2005 01:05 PM <DIR>
Investing
10/21/2005 10:04 PM <DIR>
Java Progs
08/04/2005 10:13 PM 162 main.py
10/11/2005 10:43 PM <DIR>
My Downloads
05/01/2005 10:31 AM <DIR>
My eBooks
04/22/2005 12:09 AM <DIR>
My Music
07/10/2005 11:43 AM <DIR>
My Pictures
06/29/2005 11:55 PM <DIR>
My PSP Files
05/23/2005 09:30 AM <DIR>
My Videos
05/01/2005 12:49 PM <DIR>
Office Documents
06/27/2005 03:19 PM 7,961,778
org.eclipse.jdt.doc.user.I20050627-1435.pdf
06/27/2005 03:19 PM 6,791,109
org.eclipse.platform.doc.user.I20050627-1435.pdf
10/11/2005 10:52 PM 56 oth_tsr_rm_750.ram
07/20/2005 09:32 AM 108,457 parkerred15yc.jpg
09/03/2005 10:36 PM <DIR>
Python Progs
10/20/2005 10:38 PM <DIR>
Quicken
07/10/2005 12:09 PM 3,356,248 results.xml
06/11/2005 12:03 PM 935 Scout_Ship Database.lnk
07/03/2005 12:38 PM <DIR>
Scout_Ship My Documents
10/24/2005 09:23 AM 971 Script1.py
09/25/2005 12:40 PM 1,107 Script1_old.py
08/28/2005 11:47 AM <DIR>
SimpleMu Logs
06/24/2005 08:56 PM 1,201 student_pack.ads
06/24/2005 08:49 PM 1,144 student_pack.ads.0
06/24/2005 08:56 PM 1,342 student_pack.ali
08/02/2005 11:39 PM 4,096 t.DOC
06/20/2005 10:11 AM 104 t.rx
08/05/2005 08:41 PM 66,452 Untitled-1.tif
08/05/2005 08:41 PM <DIR>
VCheck
10/03/2005 02:58 AM <DIR>
Visual Studio
10/03/2005 02:51 AM <DIR>
Visual Studio 2005
21 File(s) 18,847,490 bytes
25 Dir(s) 267,162,845,184 bytes free

E:\UserData\Dennis Lee Bieber\My Documents>
**** END of captured output

E:\UserData\Dennis Lee Bieber\My Documents>

-=-=-=-=-=-=-=-=-

Note that my "getLine()" has to return on either a real EOL, OR on
the end of a command prompt ("stuff>"). I also had to initialize the
prompt at the start. If someone issued a "cd" command to the subprocess,
the prompt would be all out of sequence, and the code would hang.

You'll also note that the prompt /ends/ a capture sequence, and the
next command is the start of the /next/ capture sequence.

--

> ================================================== ============ <
> wl*****@ix.netcom.com | Wulfraed Dennis Lee Bieber KD6MOG <
> wu******@dm.net | Bestiaria Support Staff <
> ================================================== ============ <
> Home Page: <http://www.dm.net/~wulfraed/> <
> Overflow Page: <http://wlfraed.home.netcom.com/> <

[jas]

actually, i can't check for ">" only because if you a dir, a line can
end with a > but is not the end of the output

jas wrote:

Thanks, that is certainly a start. As you mentioned, the "cd" could is
an issue.

Perhaps checking to see if the line ends with ">" is sufficient?

Dennis Lee Bieber wrote:

On 24 Oct 2005 07:20:42 -0700, "jas" <co*******@gmail.com> declaimed the
following in comp.lang.python:

Hi,
I would like to start a new process and be able to read/write from/to
it. I have tried things like...

import subprocess as sp
p = sp.Popen("cmd.exe", stdout=sp.PIPE)
p.stdin.write("hostname\n")

however, it doesn't seem to work. I think the cmd.exe is catching it.

One: you didn't read any of the "returned" output...

Two: the output only seems to be available upon EOF, which means the
spawned command processor has to exit first... Though you CAN read one
character at a time, and then have to build lines and expected prompt
strings...
This seems to work:
-=-=-=-=-=-=-=-=-
import subprocess
import os

PROMPT = os.getcwd() + ">"

def getLine(proc):
ld = []
while True:
c = proc.stdout.read(1)
if c == "\r": continue #skip Windows <cr>
if c != "\n": ld.append(c) #save all but <lf>
if c is None or c == "\n" or c == ">": break
ln = "".join(ld)
return ln

p = subprocess.Popen("cmd.exe", stdout=subprocess.PIPE, stdin =
subprocess.PIPE)

print "**** START of captured output"
while True:
ln = getLine(p)
print ln
if ln == PROMPT: break
print "**** END of captured output"

p.stdin.write("ipconfig\n")
print "**** START of captured output"
while True:
ln = getLine(p)
print ln
if ln == PROMPT: break
print "**** END of captured output"

p.stdin.write("dir\n")
print "**** START of captured output"
while True:
ln = getLine(p)
print ln
if ln == PROMPT: break
print "**** END of captured output"

p.stdin.write("exit\n")
-=-=-=-=-=-=-=-=-=-
E:\UserData\Dennis Lee Bieber\My Documents>python script1.py
**** START of captured output
Microsoft Windows XP [Version 5.1.2600]
(C) Copyright 1985-2001 Microsoft Corp.

E:\UserData\Dennis Lee Bieber\My Documents>
**** END of captured output
**** START of captured output
ipconfig

Windows IP Configuration
Ethernet adapter Local Area Connection:

Connection-specific DNS Suffix . :
IP Address. . . . . . . . . . . . : 192.168.1.100
Subnet Mask . . . . . . . . . . . : 255.255.255.0
IP Address. . . . . . . . . . . . : fe80::211:11ff:fee1:f303%4
Default Gateway . . . . . . . . . : 192.168.1.1

Tunnel adapter Teredo Tunneling Pseudo-Interface:

Connection-specific DNS Suffix . :
IP Address. . . . . . . . . . . . :
3ffe:831f:4004:1956:0:fbde:bd0a:e50b
IP Address. . . . . . . . . . . . : fe80::5445:5245:444f%5
Default Gateway . . . . . . . . . : ::

Tunnel adapter Automatic Tunneling Pseudo-Interface:

Connection-specific DNS Suffix . :
IP Address. . . . . . . . . . . . : fe80::5efe:192.168.1.100%2
Default Gateway . . . . . . . . . :

E:\UserData\Dennis Lee Bieber\My Documents>
**** END of captured output
**** START of captured output
dir
Volume in drive E is Data
Volume Serial Number is 2626-D991

Directory of E:\UserData\Dennis Lee Bieber\My Documents

10/24/2005 09:23 AM <DIR>
.
10/24/2005 09:23 AM <DIR>
..
07/25/2005 10:39 PM <DIR>
.metadata
10/06/2005 09:54 AM <DIR>
Ada Progs
08/13/2005 02:01 PM <DIR>
Agent Data
10/21/2005 09:29 AM 421,820 apress_offer.pdf
07/03/2005 11:36 AM 132 cp.py
07/17/2005 12:25 PM <DIR>
Cyberlink
07/06/2005 09:32 AM 102,400 db1.mdb
07/26/2005 12:20 AM 26,614 eclipse_code.xml
10/24/2005 01:08 AM <DIR>
Eudora
06/24/2005 08:50 PM 667 fake_oosums.ads
06/24/2005 08:50 PM 695 fake_oosums.ali
09/06/2005 09:01 PM <DIR>
Genealogy
07/13/2005 10:56 PM <DIR>
HomeSite
05/08/2005 01:05 PM <DIR>
Investing
10/21/2005 10:04 PM <DIR>
Java Progs
08/04/2005 10:13 PM 162 main.py
10/11/2005 10:43 PM <DIR>
My Downloads
05/01/2005 10:31 AM <DIR>
My eBooks
04/22/2005 12:09 AM <DIR>
My Music
07/10/2005 11:43 AM <DIR>
My Pictures
06/29/2005 11:55 PM <DIR>
My PSP Files
05/23/2005 09:30 AM <DIR>
My Videos
05/01/2005 12:49 PM <DIR>
Office Documents
06/27/2005 03:19 PM 7,961,778
org.eclipse.jdt.doc.user.I20050627-1435.pdf
06/27/2005 03:19 PM 6,791,109
org.eclipse.platform.doc.user.I20050627-1435.pdf
10/11/2005 10:52 PM 56 oth_tsr_rm_750.ram
07/20/2005 09:32 AM 108,457 parkerred15yc.jpg
09/03/2005 10:36 PM <DIR>
Python Progs
10/20/2005 10:38 PM <DIR>
Quicken
07/10/2005 12:09 PM 3,356,248 results.xml
06/11/2005 12:03 PM 935 Scout_Ship Database.lnk
07/03/2005 12:38 PM <DIR>
Scout_Ship My Documents
10/24/2005 09:23 AM 971 Script1.py
09/25/2005 12:40 PM 1,107 Script1_old.py
08/28/2005 11:47 AM <DIR>
SimpleMu Logs
06/24/2005 08:56 PM 1,201 student_pack.ads
06/24/2005 08:49 PM 1,144 student_pack.ads.0
06/24/2005 08:56 PM 1,342 student_pack.ali
08/02/2005 11:39 PM 4,096 t.DOC
06/20/2005 10:11 AM 104 t.rx
08/05/2005 08:41 PM 66,452 Untitled-1.tif
08/05/2005 08:41 PM <DIR>
VCheck
10/03/2005 02:58 AM <DIR>
Visual Studio
10/03/2005 02:51 AM <DIR>
Visual Studio 2005
21 File(s) 18,847,490 bytes
25 Dir(s) 267,162,845,184 bytes free

E:\UserData\Dennis Lee Bieber\My Documents>
**** END of captured output

E:\UserData\Dennis Lee Bieber\My Documents>

-=-=-=-=-=-=-=-=-

Note that my "getLine()" has to return on either a real EOL, OR on
the end of a command prompt ("stuff>"). I also had to initialize the
prompt at the start. If someone issued a "cd" command to the subprocess,
the prompt would be all out of sequence, and the code would hang.

You'll also note that the prompt /ends/ a capture sequence, and the
next command is the start of the /next/ capture sequence.

--

> ================================================== ============ <
> wl*****@ix.netcom.com | Wulfraed Dennis Lee Bieber KD6MOG <
> wu******@dm.net | Bestiaria Support Staff <
> ================================================== ============ <
> Home Page: <http://www.dm.net/~wulfraed/> <
> Overflow Page: <http://wlfraed.home.netcom.com/> <

[jas]

What about having a thread which reads from subprocess.Popen()'s
stdout...instead of read/write, read/write. just always read, and
write when needed?

any comments on that idea?

jas wrote:

actually, i can't check for ">" only because if you a dir, a line can
end with a > but is not the end of the output

jas wrote:

Thanks, that is certainly a start. As you mentioned, the "cd" could is
an issue.

Perhaps checking to see if the line ends with ">" is sufficient?

Dennis Lee Bieber wrote:

On 24 Oct 2005 07:20:42 -0700, "jas" <co*******@gmail.com> declaimed the
following in comp.lang.python:

> Hi,
> I would like to start a new process and be able to read/write from/to
> it. I have tried things like...
>
> import subprocess as sp
> p = sp.Popen("cmd.exe", stdout=sp.PIPE)
> p.stdin.write("hostname\n")
>
> however, it doesn't seem to work. I think the cmd.exe is catching it.

One: you didn't read any of the "returned" output...

Two: the output only seems to be available upon EOF, which means the
spawned command processor has to exit first... Though you CAN read one
character at a time, and then have to build lines and expected prompt
strings...
This seems to work:
-=-=-=-=-=-=-=-=-
import subprocess
import os

PROMPT = os.getcwd() + ">"

def getLine(proc):
ld = []
while True:
c = proc.stdout.read(1)
if c == "\r": continue #skip Windows <cr>
if c != "\n": ld.append(c) #save all but <lf>
if c is None or c == "\n" or c == ">": break
ln = "".join(ld)
return ln

p = subprocess.Popen("cmd.exe", stdout=subprocess.PIPE, stdin =
subprocess.PIPE)

print "**** START of captured output"
while True:
ln = getLine(p)
print ln
if ln == PROMPT: break
print "**** END of captured output"

p.stdin.write("ipconfig\n")
print "**** START of captured output"
while True:
ln = getLine(p)
print ln
if ln == PROMPT: break
print "**** END of captured output"

p.stdin.write("dir\n")
print "**** START of captured output"
while True:
ln = getLine(p)
print ln
if ln == PROMPT: break
print "**** END of captured output"

p.stdin.write("exit\n")
-=-=-=-=-=-=-=-=-=-
E:\UserData\Dennis Lee Bieber\My Documents>python script1.py
**** START of captured output
Microsoft Windows XP [Version 5.1.2600]
(C) Copyright 1985-2001 Microsoft Corp.

E:\UserData\Dennis Lee Bieber\My Documents>
**** END of captured output
**** START of captured output
ipconfig

Windows IP Configuration
Ethernet adapter Local Area Connection:

Connection-specific DNS Suffix . :
IP Address. . . . . . . . . . . . : 192.168.1.100
Subnet Mask . . . . . . . . . . . : 255.255.255.0
IP Address. . . . . . . . . . . . : fe80::211:11ff:fee1:f303%4
Default Gateway . . . . . . . . . : 192.168.1.1

Tunnel adapter Teredo Tunneling Pseudo-Interface:

Connection-specific DNS Suffix . :
IP Address. . . . . . . . . . . . :
3ffe:831f:4004:1956:0:fbde:bd0a:e50b
IP Address. . . . . . . . . . . . : fe80::5445:5245:444f%5
Default Gateway . . . . . . . . . : ::

Tunnel adapter Automatic Tunneling Pseudo-Interface:

Connection-specific DNS Suffix . :
IP Address. . . . . . . . . . . . : fe80::5efe:192.168.1.100%2
Default Gateway . . . . . . . . . :

E:\UserData\Dennis Lee Bieber\My Documents>
**** END of captured output
**** START of captured output
dir
Volume in drive E is Data
Volume Serial Number is 2626-D991

Directory of E:\UserData\Dennis Lee Bieber\My Documents

10/24/2005 09:23 AM <DIR>
.
10/24/2005 09:23 AM <DIR>
..
07/25/2005 10:39 PM <DIR>
.metadata
10/06/2005 09:54 AM <DIR>
Ada Progs
08/13/2005 02:01 PM <DIR>
Agent Data
10/21/2005 09:29 AM 421,820 apress_offer.pdf
07/03/2005 11:36 AM 132 cp.py
07/17/2005 12:25 PM <DIR>
Cyberlink
07/06/2005 09:32 AM 102,400 db1.mdb
07/26/2005 12:20 AM 26,614 eclipse_code.xml
10/24/2005 01:08 AM <DIR>
Eudora
06/24/2005 08:50 PM 667 fake_oosums.ads
06/24/2005 08:50 PM 695 fake_oosums.ali
09/06/2005 09:01 PM <DIR>
Genealogy
07/13/2005 10:56 PM <DIR>
HomeSite
05/08/2005 01:05 PM <DIR>
Investing
10/21/2005 10:04 PM <DIR>
Java Progs
08/04/2005 10:13 PM 162 main.py
10/11/2005 10:43 PM <DIR>
My Downloads
05/01/2005 10:31 AM <DIR>
My eBooks
04/22/2005 12:09 AM <DIR>
My Music
07/10/2005 11:43 AM <DIR>
My Pictures
06/29/2005 11:55 PM <DIR>
My PSP Files
05/23/2005 09:30 AM <DIR>
My Videos
05/01/2005 12:49 PM <DIR>
Office Documents
06/27/2005 03:19 PM 7,961,778
org.eclipse.jdt.doc.user.I20050627-1435.pdf
06/27/2005 03:19 PM 6,791,109
org.eclipse.platform.doc.user.I20050627-1435.pdf
10/11/2005 10:52 PM 56 oth_tsr_rm_750.ram
07/20/2005 09:32 AM 108,457 parkerred15yc.jpg
09/03/2005 10:36 PM <DIR>
Python Progs
10/20/2005 10:38 PM <DIR>
Quicken
07/10/2005 12:09 PM 3,356,248 results.xml
06/11/2005 12:03 PM 935 Scout_Ship Database.lnk
07/03/2005 12:38 PM <DIR>
Scout_Ship My Documents
10/24/2005 09:23 AM 971 Script1.py
09/25/2005 12:40 PM 1,107 Script1_old.py
08/28/2005 11:47 AM <DIR>
SimpleMu Logs
06/24/2005 08:56 PM 1,201 student_pack.ads
06/24/2005 08:49 PM 1,144 student_pack.ads.0
06/24/2005 08:56 PM 1,342 student_pack.ali
08/02/2005 11:39 PM 4,096 t.DOC
06/20/2005 10:11 AM 104 t.rx
08/05/2005 08:41 PM 66,452 Untitled-1.tif
08/05/2005 08:41 PM <DIR>
VCheck
10/03/2005 02:58 AM <DIR>
Visual Studio
10/03/2005 02:51 AM <DIR>
Visual Studio 2005
21 File(s) 18,847,490 bytes
25 Dir(s) 267,162,845,184 bytes free

E:\UserData\Dennis Lee Bieber\My Documents>
**** END of captured output

E:\UserData\Dennis Lee Bieber\My Documents>

-=-=-=-=-=-=-=-=-

Note that my "getLine()" has to return on either a real EOL, OR on
the end of a command prompt ("stuff>"). I also had to initialize the
prompt at the start. If someone issued a "cd" command to the subprocess,
the prompt would be all out of sequence, and the code would hang.

You'll also note that the prompt /ends/ a capture sequence, and the
next command is the start of the /next/ capture sequence.

--
> ================================================== ============ <
> wl*****@ix.netcom.com | Wulfraed Dennis Lee Bieber KD6MOG <
> wu******@dm.net | Bestiaria Support Staff <
> ================================================== ============ <
> Home Page: <http://www.dm.net/~wulfraed/> <
> Overflow Page: <http://wlfraed.home.netcom.com/> <

[Dennis Lee Bieber]

On 24 Oct 2005 10:17:39 -0700, "jas" <co*******@gmail.com> declaimed the
following in comp.lang.python:

Thanks, that is certainly a start. As you mentioned, the "cd" could is
an issue.

Perhaps checking to see if the line ends with ">" is sufficient?
That would work IF you knew that was the end of the line. Look at my
code -- the > is being treated AS the signal for EOL because a prompt
from the system does NOT have a <cr><lf> sequence. (A more robust
version of the above would assume that if the text up to the > is NOT
the prompt, the line is not complete, read some more...)
-- ================================================== ============ <
wl*****@ix.netcom.com | Wulfraed Dennis Lee Bieber KD6MOG <
wu******@dm.net | Bestiaria Support Staff <
================================================== ============ <
Home Page: <http://www.dm.net/~wulfraed/> <
Overflow Page: <http://wlfraed.home.netcom.com/> <

[Dennis Lee Bieber]

On 24 Oct 2005 10:38:43 -0700, "jas" <co*******@gmail.com> declaimed the
following in comp.lang.python:

What about having a thread which reads from subprocess.Popen()'s
stdout...instead of read/write, read/write. just always read, and
write when needed?
Regardless of how you do it, you still have to have a means of
detecting when the subprocess has stopped for input (IE, it is at a
prompt which did not supply a <cr><lf> sequence). Unfortunately, on
Windows, the select() function only works with sockets, and not files,
or you could do a simple select with time-out... Say one second... If no
read data is available after one second assume a system prompt is
present.

You might want to see if http://pexpect.sourceforge.net/ has
anything of use... Whoops -- no... Doesn't work on base Windows...

{And PLEASE trim your replies -- there is a reason top-posting is
frowned upon, besides the fact that people coming in cold end up reading
stuff in the reverse order: top-posting is like Jeopardy, one sees the
answer and then has to figure out the question... But the other reason
is that top-posting software tends to propagate meaningless quotes
because the quote is below, and ignored by the top-poster when making
comment... You made three responses and quoted my couple of hundred
lines each time, yet the only relevant substance was the paragraph about
detecting EOL or prompt}
-- ================================================== ============ <
wl*****@ix.netcom.com | Wulfraed Dennis Lee Bieber KD6MOG <
wu******@dm.net | Bestiaria Support Staff <
================================================== ============ <
Home Page: <http://www.dm.net/~wulfraed/> <
Overflow Page: <http://wlfraed.home.netcom.com/> <

[jas]

So it seems there is no good way to handle "interactive" processes on
windows using python. By interactive I mean processes/commands that
require user interaction, such as telnet or del (to delete a file or
directory sometimes you need to confirm with a yes or no), date, etc.

os.system gives the exact behavior, but you can't redirec the output.
pexpect isn't supported on windows. Even with subprocess you can't
handle all/most cases..since you have to do things like look for he
prompt.

I modified the original suggestion so it would update the prompt, in
case the user did a "cd.." ..which works fine now. However, if a user
tries to do, "del tmp123" ...windows prompts for a "are you sure you
want to delete... Y/N?" ...so the code hangs.

I can't believe no one else has done this yet..or if they have, it
hasn't been widely discussed.

Any other suggestions?

[Steve Holden]

jas wrote:

So it seems there is no good way to handle "interactive" processes on
windows using python. By interactive I mean processes/commands that
require user interaction, such as telnet or del (to delete a file or
directory sometimes you need to confirm with a yes or no), date, etc.

os.system gives the exact behavior, but you can't redirec the output.
pexpect isn't supported on windows. Even with subprocess you can't
handle all/most cases..since you have to do things like look for he
prompt.

I modified the original suggestion so it would update the prompt, in
case the user did a "cd.." ..which works fine now. However, if a user
tries to do, "del tmp123" ...windows prompts for a "are you sure you
want to delete... Y/N?" ...so the code hangs.

I can't believe no one else has done this yet..or if they have, it
hasn't been widely discussed.

Any other suggestions?

Look at how you might do it in other languages. Then you'll realise this
isn't (just) a Python problem.

regards
Steve
--
Steve Holden [phone] [phone]
Holden Web LLC www.holdenweb.com
PyCon TX 2006 www.python.org/pycon/

[jas]

Steve Holden wrote:

Look at how you might do it in other languages. Then you'll realise this
isn't (just) a Python problem.

Yea your right. However, for example, in Java, one can use the Process
class, and then read from the stream until its the end (i.e. -1 is
returned). However, with Python when reading from
subprocess.Popen.stdout ...I don't know when to stop (except for
looking for a ">" or something). Is there a standard, like read until
"-1" or something?

As I mentioned, os.system("cmd") gives me exactly the
output/interactivity I need...but I can't redirect the output.

Thanks.

[Michael Schneider]

Jas,

I use a python called twisted to run processes as you describe.

Twisted is an event-driven framework that brings a change in the
way that you look at things.

take a look at:

http://twistedmatrix.com/projects/co...o/process.html

Good luck, hope this is useful,
Mike
jas wrote:

Hi,
I would like to start a new process and be able to read/write from/to
it. I have tried things like...

import subprocess as sp
p = sp.Popen("cmd.exe", stdout=sp.PIPE)
p.stdin.write("hostname\n")

however, it doesn't seem to work. I think the cmd.exe is catching it.

I also tried
f = open("out.txt", "w")
sys.stdout = f
os.system("cmd.exe")

..but out.txt didn't contain any output from cmd.exe

So, how can I create a process (in this case, cmd.exe) on Windows and
be able to read/write from/to it?

Thanks

--
The greatest performance improvement occurs on the transition of from
the non-working state to the working state.

[Donn Cave]

Quoth "jas" <co*******@gmail.com>:
| Steve Holden wrote:
|> Look at how you might do it in other languages. Then you'll realise this
|> isn't (just) a Python problem.
|
| Yea your right. However, for example, in Java, one can use the Process
| class, and then read from the stream until its the end (i.e. -1 is
| returned). However, with Python when reading from
| subprocess.Popen.stdout ...I don't know when to stop (except for
| looking for a ">" or something). Is there a standard, like read until
| "-1" or something?

Sure, end of file is '', a string with 0 bytes. That means the other
end of the pipe has closed, usually due to exit of the other process
that was writing to it. Not much help for you there, nor would the
Java equivalent be, I presume.

Even on UNIX, where pipes are a mainstay of ordinary applications,
this one would very likely need a pseudotty device instead, to prevent
C library block buffering, and it would still be difficult and unreliable.

Ironically the best support I've seen came from a platform that didn't
use pipes much at all, VAX/VMS (I put that in the past tense because
for all I know it may have evolved in this respect.) The pipe-like
VMS device was called a "mailbox", and the interesting feature was
that you could be notified when a read had been queued on the device.

Donn Cave, do**@drizzle.com

[Dennis Lee Bieber]

On 25 Oct 2005 05:22:20 -0700, "jas" <co*******@gmail.com> declaimed the
following in comp.lang.python:

So it seems there is no good way to handle "interactive" processes on
windows using python. By interactive I mean processes/commands that
require user interaction, such as telnet or del (to delete a file or
directory sometimes you need to confirm with a yes or no), date, etc.
telnetlib includes an "expect" method...
os.system gives the exact behavior, but you can't redirec the output.
pexpect isn't supported on windows. Even with subprocess you can't
handle all/most cases..since you have to do things like look for he
prompt.
That applies to any OS... If you have no IPC signalling mechanism
that the other process has completed an output phase and is now waiting
for input, you must scan for whatever is the "current" prompt. Even an
"expect" module is "expecting" to find something you had to specify in
advance.

Heck... How do YOU recognize that an application running in a
command window is waiting for input?

I suspect you do it by reading the output text and recognizing that
the cursor is on a line that looks like a prompt. The main difference is
that you have cognitive understanding of what IS the prompt, and can
recognize the differences between, say an [abort, retry, exit] prompt
and c:\junk> prompt. To programmatically handle this situation, you have
to code the logic to recognize the two possible prompts.

I modified the original suggestion so it would update the prompt, in
case the user did a "cd.." ..which works fine now. However, if a user
tries to do, "del tmp123" ...windows prompts for a "are you sure you
want to delete... Y/N?" ...so the code hangs.

I can't believe no one else has done this yet..or if they have, it
hasn't been widely discussed.

Any other suggestions?
Don't try to do a pass-through of everything to a command
interpreter -- instead write your own command interpreter IN Python.

Parse "cd ..." => os.chdir("...")
"del ..." => os.remove("...")

etc.

Basically, any command that can change the environment should be
handled by your program, not a pass through. But of course, if you then
try to handle an external interactive program, you'd have to know what
it uses for prompts.

The other possibility (and I have a dental appointment to get to so
can't spend another hour trying to code it) is to, yes, use a thread to
fetch characters one at a time from the subprocess output. Use a Queue
to pass those characters to another thread for assembly into "lines" (on
a <cr> or <lf> you have an obvious line -- pass it on to the main
processing thread). The Queue process will need to loop until the queue
has been empty for some defined amount of time -- this delay with no
data would be the signal that a prompt message likely has been received,
and any "incomplete" line would be returned to the main process along
with a flag saying "prompt" [pity select() can't be used, or you could
combine the Queue processing and read-1-char thread into a single
thread]
-- ================================================== ============ <
wl*****@ix.netcom.com | Wulfraed Dennis Lee Bieber KD6MOG <
wu******@dm.net | Bestiaria Support Staff <
================================================== ============ <
Home Page: <http://www.dm.net/~wulfraed/> <
Overflow Page: <http://wlfraed.home.netcom.com/> <

[Dennis Lee Bieber]

On Tue, 25 Oct 2005 15:29:12 -0000, "Donn Cave" <do**@drizzle.com>
declaimed the following in comp.lang.python:

Ironically the best support I've seen came from a platform that didn't
use pipes much at all, VAX/VMS (I put that in the past tense because
for all I know it may have evolved in this respect.) The pipe-like
VMS device was called a "mailbox", and the interesting feature was
that you could be notified when a read had been queued on the device.
Followed by the Amiga... The Amiga IPC used "message ports" (linked
lists owned by the creating process to which, if "public" [named], other
processes could send message packets). The Amiga port of REXX made use
of message ports as its native IPC...

address SOME_APPLICATION

would redirect all "non-REXX" statements to the message port named
"SOME_APPLICATION"; the default being "address COMMAND" -- a shell
interpreter. Then, any non-REXX statement would be processed by the
shell interpreter -- and the easiest way to force a non-REXX statement
was to put a quote mark around it (or the first word of it)

"delete" my_file_var

would translate my_file_var to whatever had been assigned to it earlier,
and pass a delete command to the shell.

address ED

would now route such statements to the "ed" editor

No explicit "write" needed to send stuff to the subprocess.
-- ================================================== ============ <
wl*****@ix.netcom.com | Wulfraed Dennis Lee Bieber KD6MOG <
wu******@dm.net | Bestiaria Support Staff <
================================================== ============ <
Home Page: <http://www.dm.net/~wulfraed/> <
Overflow Page: <http://wlfraed.home.netcom.com/> <

[jas]

I have setup multiple threads and a queue...which is working pretty
good. But I have one other issue...I have a new thread (since it is
different issue) here:
http://groups.google.com/group/comp....81d8982d1a0130

if you get chance, would you mind checking that out.

Thanks!

[jas]

I have setup multiple threads and a queue...which is working pretty
good. But I have one other issue...I have a new thread (since it is
different issue) here:
http://groups.google.com/group/comp....81d8982d1a0130

if you get chance, would you mind checking that out.

Thanks!

[Mike Meyer]

Dennis Lee Bieber <wl*****@ix.netcom.com> writes:

Followed by the Amiga... The Amiga IPC used "message ports" (linked
lists owned by the creating process to which, if "public" [named], other
processes could send message packets). The Amiga port of REXX made use
of message ports as its native IPC...

address SOME_APPLICATION

would redirect all "non-REXX" statements to the message port named
"SOME_APPLICATION"; the default being "address COMMAND" -- a shell
interpreter. Then, any non-REXX statement would be processed by the
shell interpreter -- and the easiest way to force a non-REXX statement
was to put a quote mark around it (or the first word of it)

"delete" my_file_var

would translate my_file_var to whatever had been assigned to it earlier,
and pass a delete command to the shell.

address ED

would now route such statements to the "ed" editor

These things weren't unique to the Amiga port of Rexx. Rexx on VM/CMS
did all these things. Tying them to the native IPC mechanism as a
brilliant move. Add in the shared library for supporting Rexx so that
adding Rexx support to an application (or to a language!) was trivial,
and Rexx the end result was that Rexx was used far more heavily on the
Amiga than on VM/CMS.

The downside was that variables had a default value of their name in
upper case, and any statement that wasn't recognized evaluated as a
string expresion and then fed to the current command processor. So
that the quote marks weren't needed - if you hadn't assigned a value
to the variable delete, then you could just do

delete my_file_var

and it would do the right thing. Of course, almost anything you typed
like that was valid, creating interesting bugs. I made it a habit to
keep "0" and "1" commands around that just logged errors, as those
would catch such things.

<mike
--
Mike Meyer <mw*@mired.org> http://www.mired.org/home/mwm/
Independent WWW/Perforce/FreeBSD/Unix consultant, email for more information.

[Bengt Richter]

On Tue, 25 Oct 2005 17:32:28 GMT, Dennis Lee Bieber <wl*****@ix.netcom.com> wrote:

On 25 Oct 2005 05:22:20 -0700, "jas" <co*******@gmail.com> declaimed the
following in comp.lang.python:

So it seems there is no good way to handle "interactive" processes on
windows using python. By interactive I mean processes/commands that
require user interaction, such as telnet or del (to delete a file or
directory sometimes you need to confirm with a yes or no), date, etc.

telnetlib includes an "expect" method...

os.system gives the exact behavior, but you can't redirec the output.
pexpect isn't supported on windows. Even with subprocess you can't
handle all/most cases..since you have to do things like look for he
prompt.

That applies to any OS... If you have no IPC signalling mechanism
that the other process has completed an output phase and is now waiting
for input, you must scan for whatever is the "current" prompt. Even an
"expect" module is "expecting" to find something you had to specify in
advance.

Heck... How do YOU recognize that an application running in a
command window is waiting for input?

I suspect you do it by reading the output text and recognizing that
the cursor is on a line that looks like a prompt. The main difference is
that you have cognitive understanding of what IS the prompt, and can
recognize the differences between, say an [abort, retry, exit] prompt
and c:\junk> prompt. To programmatically handle this situation, you have
to code the logic to recognize the two possible prompts.

If I were going to do something on windows, I think I would look at
creating a cmd.exe process and see if I could get ahold of its console
by way of the windows console-related api, or I might try to write a limited
shell of my own, but I'm not going to do it just for fun ;-)

NT and descendants have all sorts of stuff for waiting for things
and communicating etc. if you want to get down to the raw stuff, e.g.,
----
GetQueuedCompletionStatus QuickInfo Overview Group

The GetQueuedCompletionStatus function attempts to dequeue
an I/O completion packet from a specified input/output
completion port. If there is no completion packet queued,
the function waits for a pending input/output operation
associated with the completion port to complete. The
function returns when it can dequeue a completion packet, or
optionally when the function times out. If the function
returns because of an I/O operation completion, it sets
several variables that provide information about the
operation.

BOOL GetQueuedCompletionStatus(

HANDLE CompletionPort, // the I/O completion port of
interest LPDWORD lpNumberOfBytesTransferred, // to receive
number of bytes transferred during I/O LPDWORD
lpCompletionKey, // to receive file's completion key
LPOVERLAPPED * lpOverlapped, // to receive pointer to
OVERLAPPED structure DWORD dwMilliseconds // optional
timeout value );

Parameters

CompletionPort

Handle to the input/output completion port of interest. Use
the CreateIoCompletionPort function to create I/O completion
ports.

lpNumberOfBytesTransferred

Points to a variable that the function sets to the number of
bytes transferred during an I/O operation that has completed
and is causing the function to return.

lpCompletionKey

Points to a variable that the function sets to the
completion key value associated with the file handle whose
I/O operation has completed and is causing the function to
return. A completion key is a per-file key that is specified
in a call to CreateIoCompletionPort.

lpOverlapped

Points to a variable that the function sets to the address
of the OVERLAPPED structure that was specified when an
input/output operation that has completed was started. The
following functions can be used to start input/output
operations that complete using I/O completion ports. You
must pass the function an OVERLAPPED structure and a file
handle associated (by a call to CreateIoCompletionPort) with
an I/O completion port to invoke the I/O completion port
mechanism:

· ConnectNamedPipe · DeviceIoControl · LockFileEx · ReadFile
· TransactNamedPipe · WaitCommEvent · WriteFile

Even if both of the preceding conditions (passing the
function a file handle associated with a completion port and
a valid OVERLAPPED structure) are met, an application can
prevent completion port notification. To do this, specify a
valid event handle for the hEvent member of the OVERLAPPED
structure, and set the low order bit of that event handle. A
valid event handle whose low order bit is set keeps I/O
completion from being queued to the completion port.

dwMilliseconds

Specifies the number of milliseconds that the caller is
willing to wait for an input/output completion packet to
appear at the I/O completion port. If a completion packet
doesn't appear within the specified time, the function times
out, returns FALSE, and sets *lpOverlapped to NULL. Timing
out is optional. If dwMilliseconds is -1, the function will
never time out. If dwMilliseconds is zero and there is no
I/O operation to dequeue, the function will timeout
immediately.

Return Value

The GetQueuedCompletionStatus function's return value is
TRUE if the function dequeues an I/O completion packet for a
successful I/O operation from the completion port. The
function stores valid values into the variables pointed to
by lpNumberOfBytesTransferred, lpCompletionKey, and
lpOverlapped. The function's return value is FALSE, and
*lpOverlapped is set to NULL, if the function does not
dequeue an I/O completion packet from the completion port.
The function does not store valid values into the variables
pointed to by lpNumberOfBytesTransferred and
lpCompletionKey. To get extended error information, call
GetLastError.

The function's return value is FALSE, and *lpOverlapped is
not NULL, if the function dequeues an I/O completion packet
for a failed I/O operation from the completion port. The
function stores valid values into the variables pointed to
by lpNumberOfBytesTransferred, lpCompletionKey, and
lpOverlapped. To get extended error information, call
GetLastError.

Remarks

The Win32 I/O system can be instructed to send I/O
completion notification packets to input/output completion
ports, where they are queued up. The CreateIoCompletionPort
function provides a mechanism for this. When you perform an
input/output operation with a file handle that has an
associated input/output completion port, the I/O system
sends a completion notification packet to the completion
port when the I/O operation completes. The I/O completion
port places the completion packet in a first-in-first-out
queue. The GetQueuedCompletionStatus function retrieves
these queued I/O completion packets.

A server application may have several threads calling
GetQueuedCompletionStatus for the same completion port. As
input operations complete, the operating system queues
completion packets to the completion port. If threads are
actively waiting in a call to this function, queued requests
complete their call. You can call the
PostQueuedCompletionStatus function to post an I/O
completion packet to an I/O completion port. The I/O
completion packet will satisfy an outstanding call to the
GetQueuedCompletionStatus function.

See Also

ConnectNamedPipe, CreateIoCompletionPort, DeviceIoControl,
LockFileEx, OVERLAPPED, ReadFile,
PostQueuedCompletionStatus, TransactNamedPipe,
WaitCommEvent, WriteFile,
----

And then there is the console stuff,

----
Following are the functions used to access a console.

AllocConsole
CreateConsoleScreenBuffer
FillConsoleOutputAttribute
FillConsoleOutputCharacter
FlushConsoleInputBuffer
FreeConsole
GenerateConsoleCtrlEvent
GetConsoleCP
GetConsoleCursorInfo
GetConsoleMode
GetConsoleOutputCP
GetConsoleScreenBufferInfo
GetConsoleTitle
GetLargestConsoleWindowSize
GetNumberOfConsoleInputEvents
GetNumberOfConsoleMouseButtons
GetStdHandle
HandlerRoutine
PeekConsoleInput
ReadConsole
ReadConsoleInput
ReadConsoleOutput
ReadConsoleOutputAttribute
ReadConsoleOutputCharacter
ScrollConsoleScreenBuffer
SetConsoleActiveScreenBuffer
SetConsoleCP
SetConsoleCtrlHandler
SetConsoleCursorInfo
SetConsoleCursorPosition
SetConsoleMode
SetConsoleOutputCP
SetConsoleScreenBufferSize
SetConsoleTextAttribute
SetConsoleTitle
SetConsoleWindowInfo
SetStdHandle
WriteConsole
WriteConsoleInput
WriteConsoleOutput
WriteConsoleOutputAttribute
WriteConsoleOutputCharacter
----
And that is a miniscule piece of it all.
For sychronizing, there ought to be something in:
---
Following are the functions used in synchronization.

CreateEvent
CreateMutex
CreateSemaphore
DeleteCriticalSection
EnterCriticalSection
GetOverlappedResult
InitializeCriticalSection
InterlockedDecrement
InterlockedExchange
InterlockedIncrement
LeaveCriticalSection
MsgWaitForMultipleObjects
OpenEvent
OpenMutex
OpenSemaphore
PulseEvent
ReleaseMutex
ReleaseSemaphore
ResetEvent
SetEvent
WaitForMultipleObjects
WaitForMultipleObjectsEx
WaitForSingleObject
WaitForSingleObjectEx

BTW,
"""
The CreateFile function creates, opens, or truncates a file, pipe,
communications resource, disk device, or console. It returns a handle
that can be used to access the object. It can also open and return
a handle to a directory.
"""
So it seems likely someone has put together most of the pieces already,
just maybe not wrapped in a python API ;-)

Regards,
Bengt Richter

[Dennis Lee Bieber]

On 25 Oct 2005 11:26:29 -0700, "jas" <co*******@gmail.com> declaimed the
following in comp.lang.python:

I have setup multiple threads and a queue...which is working pretty
good. But I have one other issue...I have a new thread (since it is
different issue) here:
http://groups.google.com/group/comp....81d8982d1a0130

if you get chance, would you mind checking that out.
I haven't sent you my bill for the current effort yet; let's see,
about three hours of effort over two days... $150 <G> {That ignores that
a real consulting rate would have to include the overhead for both sides
of Social Security and various medical insurance coverages, not just my
base employed rate <$G$>}

What I came up with (at work, strangely -- very unproductive day)...
'Tis a tad long as I stuffed a mass of comment lines into it (not to
mention the test run capture includes a copy of the file from a "type"
command)...

-=-=-=-=-=-=-=-
import subprocess
import os
import threading
import Queue
class Shell(object):
def __init__(self, command):
# flag for shutdown
self.die = False

self.timeOut = 1.0

# queue for buffering "lines"
self.lineQueue = Queue.Queue()

# list for collecting characters read from subprocess
# allows for character reader to collect characters
# while line reader processes a prior batch
self.charList = []

# event to signal that, at least, one character is
# available. event is used to allow time-out wait()
self.charEvent = threading.Event()

# event to signal that something has been written to the
# subprocess, and that the line reader should start
# collecting stuff (this is used to avoid multiple "prompt"
# timeouts from falsely affecting the user
self.lineEvent = threading.Event()

# lock to wrap access to self.charList
self.charLock = threading.Lock()

# create subprocess object with supplied command
self.process = subprocess.Popen(command,
stdout=subprocess.PIPE,
stdin=subprocess.PIPE)

# initialize, and start the character reader
# thread -- this is the thread that actually reads the
# subprocess output stream, one character at a time
self.charThread = threading.Thread(target=self.readChar)
self.charThread.start()

# initialize, and start the line reader thread
# -- this is the thread that processes collected characters
# OR on a timeout of the event, queues up a prompt and
# resets the event flags to block until more data is expected
self.lineThread = threading.Thread(target=self.readLine)
self.lineThread.start()

# set the line event so the line reader knows it should read
up
# to the next time-out/prompt
self.lineEvent.set()

def readChar(self):
while not self.die:
# read one character at a time from subprocess; blocks
ch = self.process.stdout.read(1)

# get the lock to access the character list, append it
self.charLock.acquire()
self.charList.append(ch)
# signal the line reader that, at least one, character
# is in the list
self.charEvent.set()
# let the line reader access the character list
self.charLock.release()

def readLine(self):
lineChar = []
while not self.die:
# block until told that the subprocess has been commanded
# and thereby should be producing some sort of output
self.lineEvent.wait()
# block for up to one second waiting for the character
# reader to produce some sort of output
self.charEvent.wait(self.timeOut)
# get the lock allowing access to the character list
self.charLock.acquire()
# clear the character reader event so the next pass
# will block(time-out) if no more data
self.charEvent.clear()
# save the character list locally so the character reader
# can add more stuff to it (after clearing it)
tCharList = self.charList
self.charList = []
# release the character list lock so reader can touch it
self.charLock.release()
# if the list is not empty, we have some data
if tCharList:
# process each character looking for a new line
character
# note: no filtering is done, all characters will be
# returned to user
for ch in tCharList:
lineChar.append(ch)
# on a new line, collect all preceding and make a
# string out of them, queue the string
if ch == "\n":
self.lineQueue.put("".join(lineChar))
lineChar = []
else:
# time out, assume prompt stage, queue partial line
self.lineQueue.put("".join(lineChar))
lineChar = []
# queue None as signal
self.lineQueue.put(None)
# AND clear the line event so we don't try to read
# anymore data (and get time-out/prompts) until the
# user has sent some command
self.lineEvent.clear()

def input(self):
# return the next available data line (which may be None for
# time-out/prompt. This will BLOCK if called when the queue
# is empty
return self.lineQueue.get()

def empty(self):
# return the state of the line queue
return self.lineQueue.empty()

def send(self, data, timeout = 1.0):
# send the data to the subprocess
self.process.stdin.write(data)
# reset the timeout value -- this allows the user to specify
# a long timeout for known number crunchers so that one
# doesn't get an errorneous (early) "prompt" indication
self.timeOut = timeout
# signal the line reader that data has been sent, and it
# should start processing return data
self.lineEvent.set()

def shutdown(self):
self.die = True
self.charEvent.set()
self.charThread.join()
self.lineEvent.set()
self.lineThread.join()
myCommand = Shell("cmd.exe")

print "\n**** START of captured output"
while True:
if not myCommand.empty():
# have buffered returns
ln = myCommand.input()
if ln is None: break #prompt found?
print ln
print "\n**** END of captured output"

myCommand.send("ipconfig\n")

print "\n**** START of captured output"
while True:
if not myCommand.empty():
# have buffered returns
ln = myCommand.input()
if ln is None: break #prompt found?
print ln
print "\n**** END of captured output"

myCommand.send("dir\n", timeout=5.0)

print "\n**** START of captured output"
while True:
if not myCommand.empty():
# have buffered returns
ln = myCommand.input()
if ln is None: break #prompt found?
print ln,
print "\n**** END of captured output"

myCommand.send("type script2.py\n")

print "\n**** START of captured output"
while True:
if not myCommand.empty():
# have buffered returns
ln = myCommand.input()
if ln is None: break #prompt found?
print ln,
print "\n**** END of captured output"

myCommand.send("exit\n")

myCommand.shutdown()

-=-=-=-=-=-=-=-=-
d:\data\dbieber\My Documents>python script2.py

**** START of captured output
Microsoft Windows XP [Version 5.1.2600]

(C) Copyright 1985-2001 Microsoft Corp.

d:\data\dbieber\My Documents>

**** END of captured output

**** START of captured output
ipconfig

Windows IP Configuration

Ethernet adapter Local Area Connection:

Connection-specific DNS Suffix . : suppressed.data.com

IP Address. . . . . . . . . . . . : XXX.197.174.84

Subnet Mask . . . . . . . . . . . : 255.255.255.0

Default Gateway . . . . . . . . . : XXX.197.174.254

Ethernet adapter {C8C3DF27-306F-XXXX-AE2A-4F93DAA1E334}:

Connection-specific DNS Suffix . :

IP Address. . . . . . . . . . . . : 0.0.0.0

Subnet Mask . . . . . . . . . . . : 0.0.0.0

Default Gateway . . . . . . . . . :

d:\data\dbieber\My Documents>

**** END of captured output

**** START of captured output
dir
Volume in drive D is Data
Volume Serial Number is 0CBD-XXXX

Directory of d:\data\dbieber\My Documents

10/25/2005 03:51 PM <DIR> .
10/25/2005 03:51 PM <DIR> ..
06/02/2004 10:41 AM <DIR> .idlerc
09/25/2005 03:24 PM 13,711
114674_000E0C316C06C2ADD949120500003699.zip
07/27/2005 04:26 PM 129,798 123160main_cas-skr2-072504.wav
07/27/2005 04:25 PM 734,972 123163main_cas-skr1-112203.wav
09/16/2004 11:15 AM 24,683 2xmodtxt.gz
06/10/2005 04:12 PM 4,517,006 AA.pdf
08/25/2005 04:07 PM <DIR> Ada Progs
08/17/2005 01:49 PM 687,176 ada95_craft_oo_prog.zip
08/17/2005 01:53 PM 1,015,402 adadistilled.pdf
06/13/2005 05:01 PM 750,190 an-introduction-to-tkinter.pdf
06/09/2005 11:17 AM 16,896 art stuff.xls
06/17/2005 03:12 PM 9,490 BookList.log
02/17/2005 12:21 PM 2,174,976 BookList.mdb
10/26/2004 04:32 PM 6,836 BookList.sql
08/05/2005 07:09 PM 8,632 Booklist.zip
11/09/2004 05:18 PM 845 Boromir FTP.lnk
11/03/2004 06:02 PM 798 Boromir Telnet.lnk
07/01/2005 01:02 PM 14,313 c1_1_b.JPG
03/14/2005 12:29 PM <DIR> CDRW Image
08/17/2005 01:56 PM 18,930 chap1.html
08/17/2005 01:56 PM 136,287 chap2.html
08/17/2005 01:56 PM 81,335 chap3.html
08/17/2005 01:57 PM 27,332 chap4.html
10/05/1998 02:27 AM 907,264 COMtut.PPT
08/08/2005 04:09 PM 716,894 COMtut.zip
11/17/2004 04:01 PM 462,848 db1.mdb
07/07/2005 11:18 AM 8,024 Deep Sky3.jpg
10/06/2005 11:38 AM 416 default.gpr
05/29/2003 01:42 PM 25,600 Dell.doc
03/17/2005 12:18 PM 10,985 DH.py
09/07/2005 01:41 PM 52,736 DLB Resume SW Engr locked.doc
09/08/2005 04:00 PM 1,667,306 Draft#3.pdf.pdf
09/08/2005 11:18 AM 497,737 drizzle_update.pdf
01/06/2005 12:09 PM 3,706 DTD.py
07/26/2005 10:35 AM 26,614 eclipse_code.xml
02/24/2005 01:53 PM 3,316 Eco.py
02/24/2005 01:47 PM 6,972 Eco.pyc
01/12/2005 02:57 PM 23,040 Equifax complaint.doc
01/06/2005 12:02 PM 3,356 FCS.py
09/08/2005 03:58 PM 152,691 Focusing%20for%20Beginners.pdf
05/02/2005 04:52 PM 3,232,612 FR_2.6.5_User_Guide.pdf
06/06/2005 11:25 AM <DIR> GKL-II
10/05/2005 06:02 PM <DIR> GSTS_Config
03/15/2005 02:52 PM <DIR> GSTS_OLD
03/03/2005 02:26 PM 862,403 ink_color_charts.pdf
03/16/2005 11:48 AM <DIR> Installation
10/25/2005 03:35 PM <DIR> Installers
08/17/2005 02:22 PM 647,620 intro.html
08/17/2005 02:22 PM <DIR> intro_files
06/21/2005 03:02 PM <DIR> Java
01/07/2003 12:27 PM 4,445,260 kjv10.txt
07/13/2005 03:34 PM 1,403,262 kjv10.zip
08/24/2004 06:02 PM <DIR> Laptop Effort
09/09/2003 10:18 AM 182,407 XXXXlocatormap.pdf
09/25/2005 02:46 PM 49,152 Master Resume.doc
09/25/2005 02:46 PM 63,195 Master Resume.pdf
03/11/2005 04:21 PM 129,368 Mottishaw_Order_Form.pdf
11/22/2004 12:32 PM <DIR> My Downloads
02/11/2005 11:52 AM <DIR> My eBooks
03/15/2005 11:53 AM <DIR> My Music
06/02/2004 10:42 AM <DIR> My Pictures
08/05/2005 02:25 PM 70,201 OMONDO_features_chart.pdf
10/18/2005 05:51 PM 4,813 overload.html
10/05/2005 06:29 PM 63,278
PersonalObjectivesDennisBieber[1].rtf
07/22/2005 07:51 PM <DIR> Python Progs
08/17/2005 01:57 PM 3,300 refs.html
06/10/2005 04:16 PM 2,738,281 RM.pdf
04/27/2005 06:44 PM 56,750
sata_comparison_w_ultra_ata_tech.pdf
10/18/2004 01:02 PM 27,874 SBCPrice.html
09/19/2005 11:09 AM 2,181 scan.py
04/01/2005 04:49 PM 3,415 Script1.py
10/25/2005 03:51 PM 7,145 Script2.py
09/03/2004 03:47 PM 3,884 serial.zip
08/23/2005 03:56 PM 35,328 Sharps Disposal.doc
06/02/2004 10:42 AM <DIR> Spec-Specs
10/15/2005 07:53 PM 7,529 SSCR.log
10/21/2005 06:40 PM <DIR> TCS
07/24/2005 03:06 PM 3,213 TCSreq.csv
07/24/2005 03:06 PM 2,704 tcsreq.csv_c2r.html
07/24/2005 03:08 PM 4,544 tcsreq.csv_r2c.html
06/13/2005 05:05 PM 842,827 tkinter.pdf
10/02/2005 06:28 PM 25,504 tlm-i.svg
08/08/2005 11:08 AM 22,385 untitled1.png
07/03/2003 10:17 AM 32,541 wr-arms.png
06/14/2004 05:27 PM 14,336 Wt_HP chart.xls
12/10/2003 01:30 PM 1,089 Zipper.py
07/20/2005 11:20 AM 308,127 zvprthlp.chm
69 File(s) 30,235,641 bytes
20 Dir(s) 18,418,667,520 bytes free

d:\data\dbieber\My Documents>
**** END of captured output

**** START of captured output
type script2.py
import subprocess
import os
import threading
import Queue
class Shell(object):
def __init__(self, command):
# flag for shutdown
self.die = False

self.timeOut = 1.0

# queue for buffering "lines"
self.lineQueue = Queue.Queue()

# list for collecting characters read from subprocess
# allows for character reader to collect characters
# while line reader processes a prior batch
self.charList = []

# event to signal that, at least, one character is
# available. event is used to allow time-out wait()
self.charEvent = threading.Event()

# event to signal that something has been written to the
# subprocess, and that the line reader should start
# collecting stuff (this is used to avoid multiple "prompt"
# timeouts from falsely affecting the user
self.lineEvent = threading.Event()

# lock to wrap access to self.charList
self.charLock = threading.Lock()

# create subprocess object with supplied command
self.process = subprocess.Popen(command,
stdout=subprocess.PIPE,
stdin=subprocess.PIPE)

# initialize, and start the character reader
# thread -- this is the thread that actually reads the
# subprocess output stream, one character at a time
self.charThread = threading.Thread(target=self.readChar)
self.charThread.start()

# initialize, and start the line reader thread
# -- this is the thread that processes collected characters
# OR on a timeout of the event, queues up a prompt and
# then busy-waits until the queue is empty (meaning the
# user application has processed up to
self.lineThread = threading.Thread(target=self.readLine)
self.lineThread.start()

# set the line event so the line reader knows it should read
up
# to the next time-out/prompt
self.lineEvent.set()

def readChar(self):
while not self.die:
# read one character at a time from subprocess; blocks
ch = self.process.stdout.read(1)

# get the lock to access the character list, append it
self.charLock.acquire()
self.charList.append(ch)
# signal the line reader that, at least one, character
# is in the list
self.charEvent.set()
# let the line reader access the character list
self.charLock.release()

def readLine(self):
lineChar = []
while not self.die:
# block until told that the subprocess has been commanded
# and thereby should be producing some sort of output
self.lineEvent.wait()
# block for up to one second waiting for the character
# reader to produce some sort of output
self.charEvent.wait(self.timeOut)
# get the lock allowing access to the character list
self.charLock.acquire()
# clear the character reader event so the next pass
# will block(time-out) if no more data
self.charEvent.clear()
# save the character list locally so the character reader
# can add more stuff to it (after clearing it)
tCharList = self.charList
self.charList = []
# release the character list lock so reader can touch it
self.charLock.release()
# if the list is not empty, we have some data
if tCharList:
# process each character looking for a new line
character
# note: no filtering is done, all characters will be
# returned to user
for ch in tCharList:
lineChar.append(ch)
# on a new line, collect all preceding and make a
# string out of them, queue the string
if ch == "\n":
self.lineQueue.put("".join(lineChar))
lineChar = []
else:
# time out, assume prompt stage, queue partial line
self.lineQueue.put("".join(lineChar))
lineChar = []
# queue None as signal
self.lineQueue.put(None)
# AND clear the line event so we don't try to read
# anymore data (and get time-out/prompts) until the
# user has sent some command
self.lineEvent.clear()

def input(self):
# return the next available data line (which may be None for
# time-out/prompt. This will BLOCK if called when the queue
# is empty
return self.lineQueue.get()

def empty(self):
# return the state of the line queue
return self.lineQueue.empty()

def send(self, data, timeout = 1.0):
# send the data to the subprocess
self.process.stdin.write(data)
# reset the timeout value -- this allows the user to specify
# a long timeout for known number crunchers so that one
# doesn't get an errorneous (early) "prompt" indication
self.timeOut = timeout
# signal the line reader that data has been sent, and it
# should start processing return data
self.lineEvent.set()

def shutdown(self):
self.die = True
self.charEvent.set()
self.charThread.join()
self.lineEvent.set()
self.lineThread.join()
myCommand = Shell("cmd.exe")

print "\n**** START of captured output"
while True:
if not myCommand.empty():
# have buffered returns
ln = myCommand.input()
if ln is None: break #prompt found?
print ln
print "\n**** END of captured output"

myCommand.send("ipconfig\n")

print "\n**** START of captured output"
while True:
if not myCommand.empty():
# have buffered returns
ln = myCommand.input()
if ln is None: break #prompt found?
print ln
print "\n**** END of captured output"

myCommand.send("dir\n", timeout=5.0)

print "\n**** START of captured output"
while True:
if not myCommand.empty():
# have buffered returns
ln = myCommand.input()
if ln is None: break #prompt found?
print ln,
print "\n**** END of captured output"

myCommand.send("type script2.py\n")

print "\n**** START of captured output"
while True:
if not myCommand.empty():
# have buffered returns
ln = myCommand.input()
if ln is None: break #prompt found?
print ln,
print "\n**** END of captured output"

myCommand.send("exit\n")

myCommand.shutdown()
d:\data\dbieber\My Documents>
**** END of captured output

d:\data\dbieber\My Documents>
-- ================================================== ============ <
wl*****@ix.netcom.com | Wulfraed Dennis Lee Bieber KD6MOG <
wu******@dm.net | Bestiaria Support Staff <
================================================== ============ <
Home Page: <http://www.dm.net/~wulfraed/> <
Overflow Page: <http://wlfraed.home.netcom.com/> <