I have one script (Match1) that calls a Fortran executable as a
sub-process, and I want to write another script (Match4) that
spawns off several instances of Match1 in parallel and then waits
until they all finish running. The only way I can think of doing this
is to call it as a sub-process, rather than directly.
I'm able to get Match1 working correctly in isolation, using the
subprocess.Popen command, but calling an instance of Match1 as a
subprocess spawned from Match4 isn't working.
The command (stored as an array of strings) that I'm executing is:
['python ../src_python/Match1.py ',
'--file_ref=MISR_AM1_GRP_ELLIPSOID_GM_P228_O003571_BF _F03_0024.hdf ',
'--file_cmp=MISR_AM1_GRP_ELLIPSOID_GM_P228_O003571_DF _F03_0024.hdf ',
'--block_start=62 ', '--block_end=62 ', '--istep=16 ', "--chmetric='M2'
", "--use_textid='true '"]
and I'm calling it as:
sub1 = subprocess.Popen(command)
I get the error below. Does anybody know what this error refers
to and what I'm doing wrong? Is it even allowable to call another
script as a sub-process rather than calling it directly?
File "../src_python/Match4.py", line 24, in RunMatch4
sub1 = subprocess.Popen(command1)
File "/usr/lib64/python2.5/subprocess.py", line 593, in __init__
errread, errwrite)
File "/usr/lib64/python2.5/subprocess.py", line 1051, in _execute_child
raise child_exception
OSError: [Errno 2] No such file or directory
Thanks for any help,
Catherine
9  15508 
On Nov 19, 2008, at 2:03 PM, Catherine Moroney wrote:
The command (stored as an array of strings) that I'm executing is:
['python ../src_python/Match1.py ', '--
file_ref=MISR_AM1_GRP_ELLIPSOID_GM_P228_O003571_BF _F03_0024.hdf ',
'--file_cmp=MISR_AM1_GRP_ELLIPSOID_GM_P228_O003571_DF _F03_0024.hdf
', '--block_start=62 ', '--block_end=62 ', '--istep=16 ', "--
chmetric='M2' ", "--use_textid='true '"]
[snip]
>
I get the error below. Does anybody know what this error refers
to and what I'm doing wrong? Is it even allowable to call another
script as a sub-process rather than calling it directly?
File "../src_python/Match4.py", line 24, in RunMatch4
sub1 = subprocess.Popen(command1)
File "/usr/lib64/python2.5/subprocess.py", line 593, in __init__
errread, errwrite)
File "/usr/lib64/python2.5/subprocess.py", line 1051, in
_execute_child
raise child_exception
OSError: [Errno 2] No such file or directory
Try supplying a fully-qualified path to your script, e.g.:
['python /home/catherine/src_python/Match1.py ', '--
file_ref=MISR_AM1_GRP_ELLIPSOID_GM_P228_O003571_BF _F03_0024.hdf ', '--
file_cmp=MISR_AM1_GRP_ELLIPSOID_GM_P228_O003571_DF _F03_0024.hdf ', '--
block_start=62 ', '--block_end=62 ', '--istep=16 ', "--chmetric='M2'
", "--use_textid='true '"]
On Nov 19, 1:03*pm, Catherine Moroney
<Catherine.M.Moro...@jpl.nasa.govwrote:
I have one script (Match1) that calls a Fortran executable as a
sub-process, and I want to write another script (Match4) that
spawns off several instances of Match1 in parallel and then waits
until they all finish running. *The only way I can think of doing this
is to call it as a sub-process, rather than directly.
I'm able to get Match1 working correctly in isolation, using the
subprocess.Popen command, but calling an instance of Match1 as a
subprocess spawned from Match4 isn't working.
The command (stored as an array of strings) that I'm executing is:
['python ../src_python/Match1.py ',
'--file_ref=MISR_AM1_GRP_ELLIPSOID_GM_P228_O003571_BF _F03_0024.hdf ',
'--file_cmp=MISR_AM1_GRP_ELLIPSOID_GM_P228_O003571_DF _F03_0024.hdf ',
'--block_start=62 ', '--block_end=62 ', '--istep=16 ', "--chmetric='M2'
", "--use_textid='true '"]
and I'm calling it as:
sub1 = subprocess.Popen(command)
I get the error below. *Does anybody know what this error refers
to and what I'm doing wrong? *Is it even allowable to call another
script as a sub-process rather than calling it directly?
* File "../src_python/Match4.py", line 24, in RunMatch4
* * *sub1 = subprocess.Popen(command1)
* *File "/usr/lib64/python2.5/subprocess.py", line 593, in __init__
* * *errread, errwrite)
* *File "/usr/lib64/python2.5/subprocess.py", line 1051, in _execute_child
* * *raise child_exception
OSError: [Errno 2] No such file or directory
Thanks for any help,
Catherine
Try giving an absolute path to the python file rather than a relative
path. I don't think the subprocess module "knows" where to look
otherwise.
Mike
On Wed, Nov 19, 2008 at 2:13 PM, Philip Semanchuk <ph****@semanchuk.comwrote:
>
On Nov 19, 2008, at 2:03 PM, Catherine Moroney wrote:
>The command (stored as an array of strings) that I'm executing is:
['python ../src_python/Match1.py ',
'--file_ref=MISR_AM1_GRP_ELLIPSOID_GM_P228_O003571_BF _F03_0024.hdf ',
'--file_cmp=MISR_AM1_GRP_ELLIPSOID_GM_P228_O003571_DF _F03_0024.hdf ',
'--block_start=62 ', '--block_end=62 ', '--istep=16 ', "--chmetric='M2' ",
"--use_textid='true '"]
[snip]
>>
I get the error below. Does anybody know what this error refers
to and what I'm doing wrong? Is it even allowable to call another
script as a sub-process rather than calling it directly?
File "../src_python/Match4.py", line 24, in RunMatch4
sub1 = subprocess.Popen(command1)
File "/usr/lib64/python2.5/subprocess.py", line 593, in __init__
errread, errwrite)
File "/usr/lib64/python2.5/subprocess.py", line 1051, in _execute_child
raise child_exception
OSError: [Errno 2] No such file or directory
Try supplying a fully-qualified path to your script, e.g.:
['python /home/catherine/src_python/Match1.py ',
'--file_ref=MISR_AM1_GRP_ELLIPSOID_GM_P228_O003571_BF _F03_0024.hdf ',
'--file_cmp=MISR_AM1_GRP_ELLIPSOID_GM_P228_O003571_DF _F03_0024.hdf ',
'--block_start=62 ', '--block_end=62 ', '--istep=16 ', "--chmetric='M2' ",
"--use_textid='true '"]
I think when I came across this error, I added shell=True, e.g.
sub1 = subprocess.Popen(command, shell=True)
I just tried that, and I get the same error.
Interestingly enough, a shorter (and incorrect) version
of the command works well enough so that it gets into the
Match1 code and does the argument check there.
The following code gets into Match1:
>>command = ['python', '../src_python/Match1.py','--filex="xyz"']
sub1 = subprocess.Popen(command)
whereas this doesn't even get to call Match1:
command =
['python','/data/svn_workspace/cmm/sieglind/USC/EE569/tpaper/test/../src_python/Match1.py
', '--file_ref=MISR_AM1_GRP_ELLIPSOID_GM_P228_O003571_BF _F03_0024.hdf ',
'--file_cmp=MISR_AM1_GRP_ELLIPSOID_GM_P228_O003571_DF _F03_0024.hdf ',
'--block_start=62 ', '--block_end=62 ', '--istep=16 ', '--chmetric=M2 ',
'--use_textid=true']
sub1 = subprocess.Popen(command)
Can anybody see a reason for why the abbreviated version works, and
the full-up one doesn't?
Catherine
Philip Semanchuk wrote:
>
On Nov 19, 2008, at 2:03 PM, Catherine Moroney wrote:
>The command (stored as an array of strings) that I'm executing is:
['python ../src_python/Match1.py ',
'--file_ref=MISR_AM1_GRP_ELLIPSOID_GM_P228_O003571_BF _F03_0024.hdf ',
'--file_cmp=MISR_AM1_GRP_ELLIPSOID_GM_P228_O003571_DF _F03_0024.hdf ',
'--block_start=62 ', '--block_end=62 ', '--istep=16 ',
"--chmetric='M2' ", "--use_textid='true '"]
[snip]
>>
I get the error below. Does anybody know what this error refers
to and what I'm doing wrong? Is it even allowable to call another
script as a sub-process rather than calling it directly?
File "../src_python/Match4.py", line 24, in RunMatch4
sub1 = subprocess.Popen(command1)
File "/usr/lib64/python2.5/subprocess.py", line 593, in __init__
errread, errwrite)
File "/usr/lib64/python2.5/subprocess.py", line 1051, in _execute_child
raise child_exception
OSError: [Errno 2] No such file or directory
Try supplying a fully-qualified path to your script, e.g.:
['python /home/catherine/src_python/Match1.py ',
'--file_ref=MISR_AM1_GRP_ELLIPSOID_GM_P228_O003571_BF _F03_0024.hdf ',
'--file_cmp=MISR_AM1_GRP_ELLIPSOID_GM_P228_O003571_DF _F03_0024.hdf ',
'--block_start=62 ', '--block_end=62 ', '--istep=16 ', "--chmetric='M2'
", "--use_textid='true '"]
Dan Upton wrote:
On Wed, Nov 19, 2008 at 2:13 PM, Philip Semanchuk <ph****@semanchuk.comwrote:
>On Nov 19, 2008, at 2:03 PM, Catherine Moroney wrote:
>>The command (stored as an array of strings) that I'm executing is:
['python ../src_python/Match1.py ',
'--file_ref=MISR_AM1_GRP_ELLIPSOID_GM_P228_O003571_BF _F03_0024.hdf ',
'--file_cmp=MISR_AM1_GRP_ELLIPSOID_GM_P228_O003571_DF _F03_0024.hdf ',
'--block_start=62 ', '--block_end=62 ', '--istep=16 ', "--chmetric='M2' ",
"--use_textid='true '"]
[snip]
>>I get the error below. Does anybody know what this error refers
to and what I'm doing wrong? Is it even allowable to call another
script as a sub-process rather than calling it directly?
File "../src_python/Match4.py", line 24, in RunMatch4
sub1 = subprocess.Popen(command1)
File "/usr/lib64/python2.5/subprocess.py", line 593, in __init__
errread, errwrite)
File "/usr/lib64/python2.5/subprocess.py", line 1051, in _execute_child
raise child_exception
OSError: [Errno 2] No such file or directory
Try supplying a fully-qualified path to your script, e.g.:
['python /home/catherine/src_python/Match1.py ',
'--file_ref=MISR_AM1_GRP_ELLIPSOID_GM_P228_O003571_BF _F03_0024.hdf ',
'--file_cmp=MISR_AM1_GRP_ELLIPSOID_GM_P228_O003571_DF _F03_0024.hdf ',
'--block_start=62 ', '--block_end=62 ', '--istep=16 ', "--chmetric='M2' ",
"--use_textid='true '"]
I think when I came across this error, I added shell=True, e.g.
sub1 = subprocess.Popen(command, shell=True)
I added the shell=True and this time it got into Match1 (hurrah!),
but it then opened up an interactive python session, and didn't
complete until I manually typed 'exit' in the interactive session.
Match1 looks like:
if __name__ == "__main__":
<<< parse arguments >>>
RunMatch1(file_ref, file_cmp, iblock_start, iblock_end, \
nlinep, nsmpp, mindispx, maxdispx, mindispl, \
maxdispl, istep, chmetric, use_textid)
exit()
where the routine RunMatch1 does all the actual processing.
How do I get Match1 to run and exit normally without opening up an
interactive session, when called as a subprocess from Match4?
Catherine
On Wed, Nov 19, 2008 at 2:38 PM, Catherine Moroney
<Ca*****************@jpl.nasa.govwrote:
Dan Upton wrote:
>>
On Wed, Nov 19, 2008 at 2:13 PM, Philip Semanchuk <ph****@semanchuk.com>
wrote:
>>>
On Nov 19, 2008, at 2:03 PM, Catherine Moroney wrote:
The command (stored as an array of strings) that I'm executing is:
['python ../src_python/Match1.py ',
'--file_ref=MISR_AM1_GRP_ELLIPSOID_GM_P228_O003571_BF _F03_0024.hdf ',
'--file_cmp=MISR_AM1_GRP_ELLIPSOID_GM_P228_O003571_DF _F03_0024.hdf ',
'--block_start=62 ', '--block_end=62 ', '--istep=16 ', "--chmetric='M2'
",
"--use_textid='true '"]
[snip]
I get the error below. Does anybody know what this error refers
to and what I'm doing wrong? Is it even allowable to call another
script as a sub-process rather than calling it directly?
File "../src_python/Match4.py", line 24, in RunMatch4
sub1 = subprocess.Popen(command1)
File "/usr/lib64/python2.5/subprocess.py", line 593, in __init__
errread, errwrite)
File "/usr/lib64/python2.5/subprocess.py", line 1051, in _execute_child
raise child_exception
OSError: [Errno 2] No such file or directory
Try supplying a fully-qualified path to your script, e.g.:
['python /home/catherine/src_python/Match1.py ',
'--file_ref=MISR_AM1_GRP_ELLIPSOID_GM_P228_O003571_BF _F03_0024.hdf ',
'--file_cmp=MISR_AM1_GRP_ELLIPSOID_GM_P228_O003571_DF _F03_0024.hdf ',
'--block_start=62 ', '--block_end=62 ', '--istep=16 ', "--chmetric='M2'
",
"--use_textid='true '"]
I think when I came across this error, I added shell=True, e.g.
sub1 = subprocess.Popen(command, shell=True)
I added the shell=True and this time it got into Match1 (hurrah!),
but it then opened up an interactive python session, and didn't
complete until I manually typed 'exit' in the interactive session.
Match1 looks like:
if __name__ == "__main__":
<<< parse arguments >>>
RunMatch1(file_ref, file_cmp, iblock_start, iblock_end, \
nlinep, nsmpp, mindispx, maxdispx, mindispl, \
maxdispl, istep, chmetric, use_textid)
exit()
where the routine RunMatch1 does all the actual processing.
How do I get Match1 to run and exit normally without opening up an
interactive session, when called as a subprocess from Match4?
Alternately, rather than using a list of arguments, have you tried
just using a string? (Again, that's the way I do it and I haven't
been having any problems recently, although I'm running shell scripts
or binaries with arguments rather than trying to invoke python on a
script.)
command = "python ../src_python/Match1.py
--file_ref=MISR_AM1_GRP_ELLIPSOID_GM_P228_O003571_BF _F03_0024.hdf
--file_cmp=MISR_AM1_GRP_ELLIPSOID_GM_P228_O003571_DF _F03_0024.hdf
--block_start=62 --block_end=62 --istep=16 --chmetric='M2'
--use_textid=true"
proc = subprocess.Popen(command, shell=True)
Catherine Moroney wrote:
I have one script (Match1) that calls a Fortran executable as a
sub-process, and I want to write another script (Match4) that
spawns off several instances of Match1 in parallel and then waits
until they all finish running. The only way I can think of doing this
is to call it as a sub-process, rather than directly.
I'm able to get Match1 working correctly in isolation, using the
subprocess.Popen command, but calling an instance of Match1 as a
subprocess spawned from Match4 isn't working.
The command (stored as an array of strings) that I'm executing is:
['python ../src_python/Match1.py ',
'--file_ref=MISR_AM1_GRP_ELLIPSOID_GM_P228_O003571_BF _F03_0024.hdf ',
'--file_cmp=MISR_AM1_GRP_ELLIPSOID_GM_P228_O003571_DF _F03_0024.hdf ',
'--block_start=62 ', '--block_end=62 ', '--istep=16 ', "--chmetric='M2'
", "--use_textid='true '"]
If you want to avoid going by the shell, and you *should* for security
reasons, you need to have each of your arguments separately in the list
without the shell quoting and extra spaces, i.e.
['python', '../src_python/Match1.py',
'--file_ref=xxxx.hdf', '--file_cmp=yyyy.hdf',
'--block_start=xx', '--block_end=62', '--istep=16', '--chmetric=M2',
'--use_texid=true']
Jeremy
--
Jeremy Sanders http://www.jeremysanders.net/
Dan Upton wrote:
I think when I came across this error, I added shell=True, e.g.
sub1 = subprocess.Popen(command, shell=True)
That's really papering over the bug. You need to have the parameters
separately, including the name of the program, separately in the list. You
need to remove any shell quoting you may use on the unix/dos command line.
Jeremy
--
Jeremy Sanders http://www.jeremysanders.net/
Dan Upton wrote:
On Wed, Nov 19, 2008 at 2:38 PM, Catherine Moroney
<Ca*****************@jpl.nasa.govwrote:
>Dan Upton wrote:
>>On Wed, Nov 19, 2008 at 2:13 PM, Philip Semanchuk <ph****@semanchuk.com>
wrote:
On Nov 19, 2008, at 2:03 PM, Catherine Moroney wrote:
The command (stored as an array of strings) that I'm executing is:
>
['python ../src_python/Match1.py ',
'--file_ref=MISR_AM1_GRP_ELLIPSOID_GM_P228_O003571_BF _F03_0024.hdf ',
'--file_cmp=MISR_AM1_GRP_ELLIPSOID_GM_P228_O003571_DF _F03_0024.hdf ',
'--block_start=62 ', '--block_end=62 ', '--istep=16 ', "--chmetric='M2'
",
"--use_textid='true '"]
>
[snip]
I get the error below. Does anybody know what this error refers
to and what I'm doing wrong? Is it even allowable to call another
script as a sub-process rather than calling it directly?
>
File "../src_python/Match4.py", line 24, in RunMatch4
sub1 = subprocess.Popen(command1)
File "/usr/lib64/python2.5/subprocess.py", line 593, in __init__
errread, errwrite)
File "/usr/lib64/python2.5/subprocess.py", line 1051, in _execute_child
raise child_exception
OSError: [Errno 2] No such file or directory
Try supplying a fully-qualified path to your script, e.g.:
['python /home/catherine/src_python/Match1.py ',
'--file_ref=MISR_AM1_GRP_ELLIPSOID_GM_P228_O003571_BF _F03_0024.hdf ',
'--file_cmp=MISR_AM1_GRP_ELLIPSOID_GM_P228_O003571_DF _F03_0024.hdf ',
'--block_start=62 ', '--block_end=62 ', '--istep=16 ', "--chmetric='M2'
",
"--use_textid='true '"]
I think when I came across this error, I added shell=True, e.g.
sub1 = subprocess.Popen(command, shell=True)
I added the shell=True and this time it got into Match1 (hurrah!),
but it then opened up an interactive python session, and didn't
complete until I manually typed 'exit' in the interactive session.
Match1 looks like:
if __name__ == "__main__":
<<< parse arguments >>>
RunMatch1(file_ref, file_cmp, iblock_start, iblock_end, \
nlinep, nsmpp, mindispx, maxdispx, mindispl, \
maxdispl, istep, chmetric, use_textid)
exit()
where the routine RunMatch1 does all the actual processing.
How do I get Match1 to run and exit normally without opening up an
interactive session, when called as a subprocess from Match4?
Alternately, rather than using a list of arguments, have you tried
just using a string? (Again, that's the way I do it and I haven't
been having any problems recently, although I'm running shell scripts
or binaries with arguments rather than trying to invoke python on a
script.)
command = "python ../src_python/Match1.py
--file_ref=MISR_AM1_GRP_ELLIPSOID_GM_P228_O003571_BF _F03_0024.hdf
--file_cmp=MISR_AM1_GRP_ELLIPSOID_GM_P228_O003571_DF _F03_0024.hdf
--block_start=62 --block_end=62 --istep=16 --chmetric='M2'
--use_textid=true"
proc = subprocess.Popen(command, shell=True)
Thanks - that did the trick. I just passed in one long string
and everything actually works. Wow! I had no idea if this was
even do-able.
This is so cool, and saves me a lot of code duplication. I can
spawn off half a dozen jobs at once and then just wait for them
to finish. It's great that python can function both as a
scripting language and also a full-blown programming language
at the same time.
Catherine This thread has been closed and replies have been disabled. Please start a new discussion. Similar topics
by: Kevin T. Ryan |
last post by:
Hi Group -
I have written a "semi-program" in MS Excel related to running a football
pool. I've updated it over the past two years or so, to the point where it
is getting pretty advanced. ...
|
by: Michael Livingston |
last post by:
Hi all,
I'm currently working on a project to develop an online simulation model of
the juvenile justice system in Queensland. We've developed the simulation
model using the python module simPy...
|
by: Jan Danielsson |
last post by:
Hello all,
How do I make a python script actually a _python_ in unix:ish
environments?
I know about adding:
#!/bin/sh
..as the first row in a shell script, but when I installed python on...
|
by: soumitra.mishra |
last post by:
Hi All,
I want to call (execute) some python scripts from my C# program. I know
that this can be done but have not idea how. The other option that I
can think of is developing python web...
|
by: marco |
last post by:
hi folks,
i can not run any python scripts with dos lineendings under cygwin's python.
if i run such a scripts i get stupid syntax error messages from python.
what can i do to run these...
|
by: Ishpeck |
last post by:
I'm using Python to automate testing software for my company. I
wanted the computers in my testing lab to automatically fetch the
latest version of the python scripts from a CVS repository and...
|
by: Mark |
last post by:
Hi, I'm new to python and looking for a better idiom to use for the
manner I have been organising my python scripts. I've googled all over
the place about this but found absolutely nothing.
I'm...
|
by: joe jacob |
last post by:
I configured apache to execute python scripts using mod_python
handler. I followed below mentioned steps to configure apache.
1. In http.conf I added
<Directory...
|
by: Tim Roberts |
last post by:
David Shi <davidgshi@yahoo.co.ukwrote:
Just in case the advice from Terry was too subtle, I'd like to spell it
out. Python scripts do not need to be compiled. The interpreter runs the
..py...
|
by: vgkhle |
last post by:
hi all
How can diff python scripts be called from a main script which is again a .py? os.system("python external_script.py") works fine, but what if I want to handle the un handled...
|
by: taylorcarr |
last post by:
A Canon printer is a smart device known for being advanced, efficient, and reliable. It is designed for home, office, and hybrid workspace use and can also be used for a variety of purposes. However,...
|
by: Charles Arthur |
last post by:
How do i turn on java script on a villaon, callus and itel keypad mobile phone
|
by: aa123db |
last post by:
Variable and constants
Use var or let for variables and const fror constants.
Var foo ='bar';
Let foo ='bar';const baz ='bar';
Functions
function $name$ ($parameters$) {
}
...
|
by: emmanuelkatto |
last post by:
Hi All, I am Emmanuel katto from Uganda. I want to ask what challenges you've faced while migrating a website to cloud.
Please let me know.
Thanks!
Emmanuel
|
by: BarryA |
last post by:
What are the essential steps and strategies outlined in the Data Structures and Algorithms (DSA) roadmap for aspiring data scientists? How can individuals effectively utilize this roadmap to progress...
|
by: nemocccc |
last post by:
hello, everyone, I want to develop a software for my android phone for daily needs, any suggestions?
|
by: Sonnysonu |
last post by:
This is the data of csv file
1 2 3
1 2 3
1 2 3
1 2 3
2 3
2 3
3
the lengths should be different i have to store the data by column-wise with in the specific length.
suppose the i have to...
|
by: marktang |
last post by:
ONU (Optical Network Unit) is one of the key components for providing high-speed Internet services. Its primary function is to act as an endpoint device located at the user's premises. However,...
|
by: jinu1996 |
last post by:
In today's digital age, having a compelling online presence is paramount for businesses aiming to thrive in a competitive landscape. At the heart of this digital strategy lies an intricately woven...
| |
