I have one script (Match1) that calls a Fortran executable as a
sub-process, and I want to write another script (Match4) that
spawns off several instances of Match1 in parallel and then waits
until they all finish running. The only way I can think of doing this
is to call it as a sub-process, rather than directly.
I'm able to get Match1 working correctly in isolation, using the
subprocess.Popen command, but calling an instance of Match1 as a
subprocess spawned from Match4 isn't working.
The command (stored as an array of strings) that I'm executing is:
['python ../src_python/Match1.py ',
'--file_ref=MISR_AM1_GRP_ELLIPSOID_GM_P228_O003571_BF _F03_0024.hdf ',
'--file_cmp=MISR_AM1_GRP_ELLIPSOID_GM_P228_O003571_DF _F03_0024.hdf ',
'--block_start=62 ', '--block_end=62 ', '--istep=16 ', "--chmetric='M2'
", "--use_textid='true '"]
and I'm calling it as:
sub1 = subprocess.Popen(command)
I get the error below. Does anybody know what this error refers
to and what I'm doing wrong? Is it even allowable to call another
script as a sub-process rather than calling it directly?
File "../src_python/Match4.py", line 24, in RunMatch4
sub1 = subprocess.Popen(command1)
File "/usr/lib64/python2.5/subprocess.py", line 593, in __init__
errread, errwrite)
File "/usr/lib64/python2.5/subprocess.py", line 1051, in _execute_child
raise child_exception
OSError: [Errno 2] No such file or directory
Thanks for any help,
Catherine
9  15440 
On Nov 19, 2008, at 2:03 PM, Catherine Moroney wrote:
The command (stored as an array of strings) that I'm executing is:
['python ../src_python/Match1.py ', '--
file_ref=MISR_AM1_GRP_ELLIPSOID_GM_P228_O003571_BF _F03_0024.hdf ',
'--file_cmp=MISR_AM1_GRP_ELLIPSOID_GM_P228_O003571_DF _F03_0024.hdf
', '--block_start=62 ', '--block_end=62 ', '--istep=16 ', "--
chmetric='M2' ", "--use_textid='true '"]
[snip]
>
I get the error below. Does anybody know what this error refers
to and what I'm doing wrong? Is it even allowable to call another
script as a sub-process rather than calling it directly?
File "../src_python/Match4.py", line 24, in RunMatch4
sub1 = subprocess.Popen(command1)
File "/usr/lib64/python2.5/subprocess.py", line 593, in __init__
errread, errwrite)
File "/usr/lib64/python2.5/subprocess.py", line 1051, in
_execute_child
raise child_exception
OSError: [Errno 2] No such file or directory
Try supplying a fully-qualified path to your script, e.g.:
['python /home/catherine/src_python/Match1.py ', '--
file_ref=MISR_AM1_GRP_ELLIPSOID_GM_P228_O003571_BF _F03_0024.hdf ', '--
file_cmp=MISR_AM1_GRP_ELLIPSOID_GM_P228_O003571_DF _F03_0024.hdf ', '--
block_start=62 ', '--block_end=62 ', '--istep=16 ', "--chmetric='M2'
", "--use_textid='true '"]
On Nov 19, 1:03*pm, Catherine Moroney
<Catherine.M.Moro...@jpl.nasa.govwrote:
I have one script (Match1) that calls a Fortran executable as a
sub-process, and I want to write another script (Match4) that
spawns off several instances of Match1 in parallel and then waits
until they all finish running. *The only way I can think of doing this
is to call it as a sub-process, rather than directly.
I'm able to get Match1 working correctly in isolation, using the
subprocess.Popen command, but calling an instance of Match1 as a
subprocess spawned from Match4 isn't working.
The command (stored as an array of strings) that I'm executing is:
['python ../src_python/Match1.py ',
'--file_ref=MISR_AM1_GRP_ELLIPSOID_GM_P228_O003571_BF _F03_0024.hdf ',
'--file_cmp=MISR_AM1_GRP_ELLIPSOID_GM_P228_O003571_DF _F03_0024.hdf ',
'--block_start=62 ', '--block_end=62 ', '--istep=16 ', "--chmetric='M2'
", "--use_textid='true '"]
and I'm calling it as:
sub1 = subprocess.Popen(command)
I get the error below. *Does anybody know what this error refers
to and what I'm doing wrong? *Is it even allowable to call another
script as a sub-process rather than calling it directly?
* File "../src_python/Match4.py", line 24, in RunMatch4
* * *sub1 = subprocess.Popen(command1)
* *File "/usr/lib64/python2.5/subprocess.py", line 593, in __init__
* * *errread, errwrite)
* *File "/usr/lib64/python2.5/subprocess.py", line 1051, in _execute_child
* * *raise child_exception
OSError: [Errno 2] No such file or directory
Thanks for any help,
Catherine
Try giving an absolute path to the python file rather than a relative
path. I don't think the subprocess module "knows" where to look
otherwise.
Mike
On Wed, Nov 19, 2008 at 2:13 PM, Philip Semanchuk <ph****@semanchuk.comwrote:
>
On Nov 19, 2008, at 2:03 PM, Catherine Moroney wrote:
>The command (stored as an array of strings) that I'm executing is:
['python ../src_python/Match1.py ',
'--file_ref=MISR_AM1_GRP_ELLIPSOID_GM_P228_O003571_BF _F03_0024.hdf ',
'--file_cmp=MISR_AM1_GRP_ELLIPSOID_GM_P228_O003571_DF _F03_0024.hdf ',
'--block_start=62 ', '--block_end=62 ', '--istep=16 ', "--chmetric='M2' ",
"--use_textid='true '"]
[snip]
>>
I get the error below. Does anybody know what this error refers
to and what I'm doing wrong? Is it even allowable to call another
script as a sub-process rather than calling it directly?
File "../src_python/Match4.py", line 24, in RunMatch4
sub1 = subprocess.Popen(command1)
File "/usr/lib64/python2.5/subprocess.py", line 593, in __init__
errread, errwrite)
File "/usr/lib64/python2.5/subprocess.py", line 1051, in _execute_child
raise child_exception
OSError: [Errno 2] No such file or directory
Try supplying a fully-qualified path to your script, e.g.:
['python /home/catherine/src_python/Match1.py ',
'--file_ref=MISR_AM1_GRP_ELLIPSOID_GM_P228_O003571_BF _F03_0024.hdf ',
'--file_cmp=MISR_AM1_GRP_ELLIPSOID_GM_P228_O003571_DF _F03_0024.hdf ',
'--block_start=62 ', '--block_end=62 ', '--istep=16 ', "--chmetric='M2' ",
"--use_textid='true '"]
I think when I came across this error, I added shell=True, e.g.
sub1 = subprocess.Popen(command, shell=True)
I just tried that, and I get the same error.
Interestingly enough, a shorter (and incorrect) version
of the command works well enough so that it gets into the
Match1 code and does the argument check there.
The following code gets into Match1:
>>command = ['python', '../src_python/Match1.py','--filex="xyz"']
sub1 = subprocess.Popen(command)
whereas this doesn't even get to call Match1:
command =
['python','/data/svn_workspace/cmm/sieglind/USC/EE569/tpaper/test/../src_python/Match1.py
', '--file_ref=MISR_AM1_GRP_ELLIPSOID_GM_P228_O003571_BF _F03_0024.hdf ',
'--file_cmp=MISR_AM1_GRP_ELLIPSOID_GM_P228_O003571_DF _F03_0024.hdf ',
'--block_start=62 ', '--block_end=62 ', '--istep=16 ', '--chmetric=M2 ',
'--use_textid=true']
sub1 = subprocess.Popen(command)
Can anybody see a reason for why the abbreviated version works, and
the full-up one doesn't?
Catherine
Philip Semanchuk wrote:
>
On Nov 19, 2008, at 2:03 PM, Catherine Moroney wrote:
>The command (stored as an array of strings) that I'm executing is:
['python ../src_python/Match1.py ',
'--file_ref=MISR_AM1_GRP_ELLIPSOID_GM_P228_O003571_BF _F03_0024.hdf ',
'--file_cmp=MISR_AM1_GRP_ELLIPSOID_GM_P228_O003571_DF _F03_0024.hdf ',
'--block_start=62 ', '--block_end=62 ', '--istep=16 ',
"--chmetric='M2' ", "--use_textid='true '"]
[snip]
>>
I get the error below. Does anybody know what this error refers
to and what I'm doing wrong? Is it even allowable to call another
script as a sub-process rather than calling it directly?
File "../src_python/Match4.py", line 24, in RunMatch4
sub1 = subprocess.Popen(command1)
File "/usr/lib64/python2.5/subprocess.py", line 593, in __init__
errread, errwrite)
File "/usr/lib64/python2.5/subprocess.py", line 1051, in _execute_child
raise child_exception
OSError: [Errno 2] No such file or directory
Try supplying a fully-qualified path to your script, e.g.:
['python /home/catherine/src_python/Match1.py ',
'--file_ref=MISR_AM1_GRP_ELLIPSOID_GM_P228_O003571_BF _F03_0024.hdf ',
'--file_cmp=MISR_AM1_GRP_ELLIPSOID_GM_P228_O003571_DF _F03_0024.hdf ',
'--block_start=62 ', '--block_end=62 ', '--istep=16 ', "--chmetric='M2'
", "--use_textid='true '"]
Dan Upton wrote:
On Wed, Nov 19, 2008 at 2:13 PM, Philip Semanchuk <ph****@semanchuk.comwrote:
>On Nov 19, 2008, at 2:03 PM, Catherine Moroney wrote:
>>The command (stored as an array of strings) that I'm executing is:
['python ../src_python/Match1.py ',
'--file_ref=MISR_AM1_GRP_ELLIPSOID_GM_P228_O003571_BF _F03_0024.hdf ',
'--file_cmp=MISR_AM1_GRP_ELLIPSOID_GM_P228_O003571_DF _F03_0024.hdf ',
'--block_start=62 ', '--block_end=62 ', '--istep=16 ', "--chmetric='M2' ",
"--use_textid='true '"]
[snip]
>>I get the error below. Does anybody know what this error refers
to and what I'm doing wrong? Is it even allowable to call another
script as a sub-process rather than calling it directly?
File "../src_python/Match4.py", line 24, in RunMatch4
sub1 = subprocess.Popen(command1)
File "/usr/lib64/python2.5/subprocess.py", line 593, in __init__
errread, errwrite)
File "/usr/lib64/python2.5/subprocess.py", line 1051, in _execute_child
raise child_exception
OSError: [Errno 2] No such file or directory
Try supplying a fully-qualified path to your script, e.g.:
['python /home/catherine/src_python/Match1.py ',
'--file_ref=MISR_AM1_GRP_ELLIPSOID_GM_P228_O003571_BF _F03_0024.hdf ',
'--file_cmp=MISR_AM1_GRP_ELLIPSOID_GM_P228_O003571_DF _F03_0024.hdf ',
'--block_start=62 ', '--block_end=62 ', '--istep=16 ', "--chmetric='M2' ",
"--use_textid='true '"]
I think when I came across this error, I added shell=True, e.g.
sub1 = subprocess.Popen(command, shell=True)
I added the shell=True and this time it got into Match1 (hurrah!),
but it then opened up an interactive python session, and didn't
complete until I manually typed 'exit' in the interactive session.
Match1 looks like:
if __name__ == "__main__":
<<< parse arguments >>>
RunMatch1(file_ref, file_cmp, iblock_start, iblock_end, \
nlinep, nsmpp, mindispx, maxdispx, mindispl, \
maxdispl, istep, chmetric, use_textid)
exit()
where the routine RunMatch1 does all the actual processing.
How do I get Match1 to run and exit normally without opening up an
interactive session, when called as a subprocess from Match4?
Catherine
On Wed, Nov 19, 2008 at 2:38 PM, Catherine Moroney
<Ca*****************@jpl.nasa.govwrote:
Dan Upton wrote:
>>
On Wed, Nov 19, 2008 at 2:13 PM, Philip Semanchuk <ph****@semanchuk.com>
wrote:
>>>
On Nov 19, 2008, at 2:03 PM, Catherine Moroney wrote:
The command (stored as an array of strings) that I'm executing is:
['python ../src_python/Match1.py ',
'--file_ref=MISR_AM1_GRP_ELLIPSOID_GM_P228_O003571_BF _F03_0024.hdf ',
'--file_cmp=MISR_AM1_GRP_ELLIPSOID_GM_P228_O003571_DF _F03_0024.hdf ',
'--block_start=62 ', '--block_end=62 ', '--istep=16 ', "--chmetric='M2'
",
"--use_textid='true '"]
[snip]
I get the error below. Does anybody know what this error refers
to and what I'm doing wrong? Is it even allowable to call another
script as a sub-process rather than calling it directly?
File "../src_python/Match4.py", line 24, in RunMatch4
sub1 = subprocess.Popen(command1)
File "/usr/lib64/python2.5/subprocess.py", line 593, in __init__
errread, errwrite)
File "/usr/lib64/python2.5/subprocess.py", line 1051, in _execute_child
raise child_exception
OSError: [Errno 2] No such file or directory
Try supplying a fully-qualified path to your script, e.g.:
['python /home/catherine/src_python/Match1.py ',
'--file_ref=MISR_AM1_GRP_ELLIPSOID_GM_P228_O003571_BF _F03_0024.hdf ',
'--file_cmp=MISR_AM1_GRP_ELLIPSOID_GM_P228_O003571_DF _F03_0024.hdf ',
'--block_start=62 ', '--block_end=62 ', '--istep=16 ', "--chmetric='M2'
",
"--use_textid='true '"]
I think when I came across this error, I added shell=True, e.g.
sub1 = subprocess.Popen(command, shell=True)
I added the shell=True and this time it got into Match1 (hurrah!),
but it then opened up an interactive python session, and didn't
complete until I manually typed 'exit' in the interactive session.
Match1 looks like:
if __name__ == "__main__":
<<< parse arguments >>>
RunMatch1(file_ref, file_cmp, iblock_start, iblock_end, \
nlinep, nsmpp, mindispx, maxdispx, mindispl, \
maxdispl, istep, chmetric, use_textid)
exit()
where the routine RunMatch1 does all the actual processing.
How do I get Match1 to run and exit normally without opening up an
interactive session, when called as a subprocess from Match4?
Alternately, rather than using a list of arguments, have you tried
just using a string? (Again, that's the way I do it and I haven't
been having any problems recently, although I'm running shell scripts
or binaries with arguments rather than trying to invoke python on a
script.)
command = "python ../src_python/Match1.py
--file_ref=MISR_AM1_GRP_ELLIPSOID_GM_P228_O003571_BF _F03_0024.hdf
--file_cmp=MISR_AM1_GRP_ELLIPSOID_GM_P228_O003571_DF _F03_0024.hdf
--block_start=62 --block_end=62 --istep=16 --chmetric='M2'
--use_textid=true"
proc = subprocess.Popen(command, shell=True)
Catherine Moroney wrote:
I have one script (Match1) that calls a Fortran executable as a
sub-process, and I want to write another script (Match4) that
spawns off several instances of Match1 in parallel and then waits
until they all finish running. The only way I can think of doing this
is to call it as a sub-process, rather than directly.
I'm able to get Match1 working correctly in isolation, using the
subprocess.Popen command, but calling an instance of Match1 as a
subprocess spawned from Match4 isn't working.
The command (stored as an array of strings) that I'm executing is:
['python ../src_python/Match1.py ',
'--file_ref=MISR_AM1_GRP_ELLIPSOID_GM_P228_O003571_BF _F03_0024.hdf ',
'--file_cmp=MISR_AM1_GRP_ELLIPSOID_GM_P228_O003571_DF _F03_0024.hdf ',
'--block_start=62 ', '--block_end=62 ', '--istep=16 ', "--chmetric='M2'
", "--use_textid='true '"]
If you want to avoid going by the shell, and you *should* for security
reasons, you need to have each of your arguments separately in the list
without the shell quoting and extra spaces, i.e.
['python', '../src_python/Match1.py',
'--file_ref=xxxx.hdf', '--file_cmp=yyyy.hdf',
'--block_start=xx', '--block_end=62', '--istep=16', '--chmetric=M2',
'--use_texid=true']
Jeremy
--
Jeremy Sanders http://www.jeremysanders.net/
Dan Upton wrote:
I think when I came across this error, I added shell=True, e.g.
sub1 = subprocess.Popen(command, shell=True)
That's really papering over the bug. You need to have the parameters
separately, including the name of the program, separately in the list. You
need to remove any shell quoting you may use on the unix/dos command line.
Jeremy
--
Jeremy Sanders http://www.jeremysanders.net/
Dan Upton wrote:
On Wed, Nov 19, 2008 at 2:38 PM, Catherine Moroney
<Ca*****************@jpl.nasa.govwrote:
>Dan Upton wrote:
>>On Wed, Nov 19, 2008 at 2:13 PM, Philip Semanchuk <ph****@semanchuk.com>
wrote:
On Nov 19, 2008, at 2:03 PM, Catherine Moroney wrote:
The command (stored as an array of strings) that I'm executing is:
>
['python ../src_python/Match1.py ',
'--file_ref=MISR_AM1_GRP_ELLIPSOID_GM_P228_O003571_BF _F03_0024.hdf ',
'--file_cmp=MISR_AM1_GRP_ELLIPSOID_GM_P228_O003571_DF _F03_0024.hdf ',
'--block_start=62 ', '--block_end=62 ', '--istep=16 ', "--chmetric='M2'
",
"--use_textid='true '"]
>
[snip]
I get the error below. Does anybody know what this error refers
to and what I'm doing wrong? Is it even allowable to call another
script as a sub-process rather than calling it directly?
>
File "../src_python/Match4.py", line 24, in RunMatch4
sub1 = subprocess.Popen(command1)
File "/usr/lib64/python2.5/subprocess.py", line 593, in __init__
errread, errwrite)
File "/usr/lib64/python2.5/subprocess.py", line 1051, in _execute_child
raise child_exception
OSError: [Errno 2] No such file or directory
Try supplying a fully-qualified path to your script, e.g.:
['python /home/catherine/src_python/Match1.py ',
'--file_ref=MISR_AM1_GRP_ELLIPSOID_GM_P228_O003571_BF _F03_0024.hdf ',
'--file_cmp=MISR_AM1_GRP_ELLIPSOID_GM_P228_O003571_DF _F03_0024.hdf ',
'--block_start=62 ', '--block_end=62 ', '--istep=16 ', "--chmetric='M2'
",
"--use_textid='true '"]
I think when I came across this error, I added shell=True, e.g.
sub1 = subprocess.Popen(command, shell=True)
I added the shell=True and this time it got into Match1 (hurrah!),
but it then opened up an interactive python session, and didn't
complete until I manually typed 'exit' in the interactive session.
Match1 looks like:
if __name__ == "__main__":
<<< parse arguments >>>
RunMatch1(file_ref, file_cmp, iblock_start, iblock_end, \
nlinep, nsmpp, mindispx, maxdispx, mindispl, \
maxdispl, istep, chmetric, use_textid)
exit()
where the routine RunMatch1 does all the actual processing.
How do I get Match1 to run and exit normally without opening up an
interactive session, when called as a subprocess from Match4?
Alternately, rather than using a list of arguments, have you tried
just using a string? (Again, that's the way I do it and I haven't
been having any problems recently, although I'm running shell scripts
or binaries with arguments rather than trying to invoke python on a
script.)
command = "python ../src_python/Match1.py
--file_ref=MISR_AM1_GRP_ELLIPSOID_GM_P228_O003571_BF _F03_0024.hdf
--file_cmp=MISR_AM1_GRP_ELLIPSOID_GM_P228_O003571_DF _F03_0024.hdf
--block_start=62 --block_end=62 --istep=16 --chmetric='M2'
--use_textid=true"
proc = subprocess.Popen(command, shell=True)
Thanks - that did the trick. I just passed in one long string
and everything actually works. Wow! I had no idea if this was
even do-able.
This is so cool, and saves me a lot of code duplication. I can
spawn off half a dozen jobs at once and then just wait for them
to finish. It's great that python can function both as a
scripting language and also a full-blown programming language
at the same time.
Catherine This thread has been closed and replies have been disabled. Please start a new discussion. Similar topics
by: Kevin T. Ryan |
last post by:
Hi Group -
I have written a "semi-program" in MS Excel related to running a football
pool. I've updated it over the past two years or so, to the point where it
is getting pretty advanced. ...
|
by: Michael Livingston |
last post by:
Hi all,
I'm currently working on a project to develop an online simulation model of
the juvenile justice system in Queensland. We've developed the simulation
model using the python module simPy...
|
by: Jan Danielsson |
last post by:
Hello all,
How do I make a python script actually a _python_ in unix:ish
environments?
I know about adding:
#!/bin/sh
..as the first row in a shell script, but when I installed python on...
|
by: soumitra.mishra |
last post by:
Hi All,
I want to call (execute) some python scripts from my C# program. I know
that this can be done but have not idea how. The other option that I
can think of is developing python web...
|
by: marco |
last post by:
hi folks,
i can not run any python scripts with dos lineendings under cygwin's python.
if i run such a scripts i get stupid syntax error messages from python.
what can i do to run these...
|
by: Ishpeck |
last post by:
I'm using Python to automate testing software for my company. I
wanted the computers in my testing lab to automatically fetch the
latest version of the python scripts from a CVS repository and...
|
by: Mark |
last post by:
Hi, I'm new to python and looking for a better idiom to use for the
manner I have been organising my python scripts. I've googled all over
the place about this but found absolutely nothing.
I'm...
|
by: joe jacob |
last post by:
I configured apache to execute python scripts using mod_python
handler. I followed below mentioned steps to configure apache.
1. In http.conf I added
<Directory...
|
by: Tim Roberts |
last post by:
David Shi <davidgshi@yahoo.co.ukwrote:
Just in case the advice from Terry was too subtle, I'd like to spell it
out. Python scripts do not need to be compiled. The interpreter runs the
..py...
|
by: vgkhle |
last post by:
hi all
How can diff python scripts be called from a main script which is again a .py? os.system("python external_script.py") works fine, but what if I want to handle the un handled...
|
by: antdb |
last post by:
Ⅰ. Advantage of AntDB: hyper-convergence + streaming processing engine
In the overall architecture, a new "hyper-convergence" concept was proposed, which integrated multiple engines and...
|
by: AndyPSV |
last post by:
HOW CAN I CREATE AN AI with an .executable file that would suck all files in the folder and on my computerHOW CAN I CREATE AN AI with an .executable file that would suck all files in the folder and...
|
by: Arjunsri |
last post by:
I have a Redshift database that I need to use as an import data source. I have configured the DSN connection using the server, port, database, and credentials and received a successful connection...
|
by: WisdomUfot |
last post by:
It's an interesting question you've got about how Gmail hides the HTTP referrer when a link in an email is clicked. While I don't have the specific technical details, Gmail likely implements measures...
|
by: Matthew3360 |
last post by:
Hi,
I have been trying to connect to a local host using php curl. But I am finding it hard to do this. I am doing the curl get request from my web server and have made sure to enable curl. I get a...
|
by: Oralloy |
last post by:
Hello Folks,
I am trying to hook up a CPU which I designed using SystemC to I/O pins on an FPGA.
My problem (spelled failure) is with the synthesis of my design into a bitstream, not the C++...
|
by: Carina712 |
last post by:
Setting background colors for Excel documents can help to improve the visual appeal of the document and make it easier to read and understand. Background colors can be used to highlight important...
|
by: Rahul1995seven |
last post by:
Introduction:
In the realm of programming languages, Python has emerged as a powerhouse. With its simplicity, versatility, and robustness, Python has gained popularity among beginners and experts...
|
by: Johno34 |
last post by:
I have this click event on my form. It speaks to a Datasheet Subform
Private Sub Command260_Click()
Dim r As DAO.Recordset
Set r = Form_frmABCD.Form.RecordsetClone
r.MoveFirst
Do
If...
| |
