By using this site, you agree to our updated Privacy Policy and our Terms of Use. Manage your Cookies Settings.
443,818 Members | 1,223 Online
Bytes IT Community
+ Ask a Question
Need help? Post your question and get tips & solutions from a community of 443,818 IT Pros & Developers. It's quick & easy.

Execute commands from file

P: n/a
Hello,

Thanks for your time.

We have very big files with python commands (more or less, 500000
commands each file).

It is possible to execute them command by command, like if the
commands was typed one after the other in a interactive session?

( Better using command flags than with an small script like "while 1:
input()" )

Thanks a lot.

May 16 '07 #1
Share this Question
Share on Google+
15 Replies


P: n/a
In <11*********************@u30g2000hsc.googlegroups. com>, tmp123 wrote:
We have very big files with python commands (more or less, 500000
commands each file).

It is possible to execute them command by command, like if the
commands was typed one after the other in a interactive session?
Take a look at the `code` module in the standard library:

In [31]: code?
Type: module
Base Class: <type 'module'>
String Form: <module 'code' from '/usr/lib/python2.4/code.pyc'>
Namespace: Interactive
File: /usr/lib/python2.4/code.py
Docstring:
Utilities needed to emulate Python's interactive interpreter.

Ciao,
Marc 'BlackJack' Rintsch
May 16 '07 #2

P: n/a
tmp123 wrote:
Hello,

Thanks for your time.

We have very big files with python commands (more or less, 500000
commands each file).
Those are BIG programs. Presumably other programs are writing them?
It is possible to execute them command by command, like if the
commands was typed one after the other in a interactive session?
You need to look for "pdb", the interactive Python debugger. This is
capable of single-step operations, and supports breakpoints.
( Better using command flags than with an small script like "while 1:
input()" )

Thanks a lot.
You are pretty much going to have to run pdb then trigger your code by
calling a pdb method with a function in your code as an argument, if I
am remembering correctly how it works.

regards
Steve
--
Steve Holden +1 571 484 6266 +1 800 494 3119
Holden Web LLC/Ltd http://www.holdenweb.com
Skype: holdenweb http://del.icio.us/steve.holden
------------------ Asciimercial ---------------------
Get on the web: Blog, lens and tag your way to fame!!
holdenweb.blogspot.com squidoo.com/pythonology
tagged items: del.icio.us/steve.holden/python
All these services currently offer free registration!
-------------- Thank You for Reading ----------------

May 16 '07 #3

P: n/a
"tmp123" schrieb >
We have very big files with python commands
(more or less, 500000 commands each file).

It is possible to execute them command by command,
inp = open(cmd_file)
for line in inp:
exec line

might help. You don't get quite the same feeling as
"like if the commands was typed one after the other
in a interactive session", but perhaps this helps.

Warning: the code above is without any error checks.
You might also run into security problems, the example
above assumes you trust your input.

HTH. YMMV.
Martin


May 16 '07 #4

P: n/a
Martin Blume wrote:
"tmp123" schrieb >
>We have very big files with python commands
(more or less, 500000 commands each file).

It is possible to execute them command by command,

inp = open(cmd_file)
for line in inp:
exec line

might help. You don't get quite the same feeling as
"like if the commands was typed one after the other
in a interactive session", but perhaps this helps.

Warning: the code above is without any error checks.
You might also run into security problems, the example
above assumes you trust your input.

HTH. YMMV.
Martin
The problem with this approach is that each line executes without any
connection to the environment created by previous lies.

Try it on a file that reads something like

xxx = 42
print xxx

and you will see NameError raised because the assignment hasn't affected
the environment for the print statement.

regards
Steve
--
Steve Holden +1 571 484 6266 +1 800 494 3119
Holden Web LLC/Ltd http://www.holdenweb.com
Skype: holdenweb http://del.icio.us/steve.holden
------------------ Asciimercial ---------------------
Get on the web: Blog, lens and tag your way to fame!!
holdenweb.blogspot.com squidoo.com/pythonology
tagged items: del.icio.us/steve.holden/python
All these services currently offer free registration!
-------------- Thank You for Reading ----------------

May 16 '07 #5

P: n/a
On May 16, 1:05 pm, Steve Holden <s...@holdenweb.comwrote:
Martin Blume wrote:
"tmp123" schrieb >
We have very big files with python commands
(more or less, 500000 commands each file).
It is possible to execute them command by command,
inp = open(cmd_file)
for line in inp:
exec line
might help. You don't get quite the same feeling as
"like if the commands was typed one after the other
in a interactive session", but perhaps this helps.
Warning: the code above is without any error checks.
You might also run into security problems, the example
above assumes you trust your input.
HTH. YMMV.
Martin

The problem with this approach is that each line executes without any
connection to the environment created by previous lies.

Try it on a file that reads something like

xxx = 42
print xxx

and you will see NameError raised because the assignment hasn't affected
the environment for the print statement.

regards
Steve
--
Steve Holden +1 571 484 6266 +1 800 494 3119
Holden Web LLC/Ltd http://www.holdenweb.com
Skype: holdenweb http://del.icio.us/steve.holden
------------------ Asciimercial ---------------------
Get on the web: Blog, lens and tag your way to fame!!
holdenweb.blogspot.com squidoo.com/pythonology
tagged items: del.icio.us/steve.holden/python
All these services currently offer free registration!
-------------- Thank You for Reading ----------------
cat file:

x = 100
print x

cat file.py:
#!/usr/bin/python2.4

import os.path
import sys

file, ext = os.path.splitext(sys.argv[0])
f = open(file,'rb')
for i in f:
exec i
>./file.py
100

Don't see the problem though.

May 17 '07 #6

P: n/a
On Thu, 17 May 2007 00:30:23, i3dmaster <i3*******@gmail.comwrote
>f = open(file,'rb')
for i in f:
exec i
Why are you opening the file in binary mode?

--
Doug Woodrow

May 17 '07 #7

P: n/a
i3dmaster wrote:
On May 16, 1:05 pm, Steve Holden <s...@holdenweb.comwrote:
>Martin Blume wrote:
>>"tmp123" schrieb >
We have very big files with python commands
(more or less, 500000 commands each file).
It is possible to execute them command by command,
inp = open(cmd_file)
for line in inp:
exec line
might help. You don't get quite the same feeling as
"like if the commands was typed one after the other
in a interactive session", but perhaps this helps.
Warning: the code above is without any error checks.
You might also run into security problems, the example
above assumes you trust your input.
HTH. YMMV.
Martin
The problem with this approach is that each line executes without any
connection to the environment created by previous lies.

Try it on a file that reads something like

xxx = 42
print xxx

and you will see NameError raised because the assignment hasn't affected
the environment for the print statement.

regards
Steve
--
Steve Holden +1 571 484 6266 +1 800 494 3119
Holden Web LLC/Ltd http://www.holdenweb.com
Skype: holdenweb http://del.icio.us/steve.holden
------------------ Asciimercial ---------------------
Get on the web: Blog, lens and tag your way to fame!!
holdenweb.blogspot.com squidoo.com/pythonology
tagged items: del.icio.us/steve.holden/python
All these services currently offer free registration!
-------------- Thank You for Reading ----------------

cat file:

x = 100
print x

cat file.py:
#!/usr/bin/python2.4

import os.path
import sys

file, ext = os.path.splitext(sys.argv[0])
f = open(file,'rb')
for i in f:
exec i
>./file.py
100

Don't see the problem though.
No, because there isn't one. Now try adding a function definition and
see how well it works.

regards
Steve
--
Steve Holden +1 571 484 6266 +1 800 494 3119
Holden Web LLC/Ltd http://www.holdenweb.com
Skype: holdenweb http://del.icio.us/steve.holden
------------------ Asciimercial ---------------------
Get on the web: Blog, lens and tag your way to fame!!
holdenweb.blogspot.com squidoo.com/pythonology
tagged items: del.icio.us/steve.holden/python
All these services currently offer free registration!
-------------- Thank You for Reading ----------------

May 17 '07 #8

P: n/a
Steve Holden a écrit :
i3dmaster wrote:
>On May 16, 1:05 pm, Steve Holden <s...@holdenweb.comwrote:
>>Martin Blume wrote:
"tmp123" schrieb >
We have very big files with python commands
(more or less, 500000 commands each file).
It is possible to execute them command by command,
inp = open(cmd_file)
for line in inp:
exec line
The problem with this approach is that each line executes without any
connection to the environment created by previous lies.

Try it on a file that reads something like

xxx = 42
print xxx
cat file:

x = 100
print x

cat file.py:
#!/usr/bin/python2.4

import os.path
import sys

file, ext = os.path.splitext(sys.argv[0])
f = open(file,'rb')
for i in f:
exec i
>>./file.py
100

Don't see the problem though.
No, because there isn't one. Now try adding a function definition and
see how well it works.

regards
Steve
This is just a problem with indentation and blocks of code, the
followong will do :

commands = open("commands")
namespace, block = {}, ""
for line in commands :
line=line[:-1]
if not line : continue
if line[0].isspace() :
block += '\n' + line
continue
else :
if block.strip() :
exec block in namespace
block = line

exec block in namespace
print dict((k, v) for k, v in namespace.items() if k != "__builtins__")
with commands containing :

"""

x = 5

def toto(arg) :
print arg

def inner() :
print arg*arg

inner()
toto(x)
"""

output :
5
25
{'x': 5, 'toto': <function toto at 0x01D30C70>}

(sorry Steve for the private mail)

--
_____________

Maric Michaud
_____________

Aristote - www.aristote.info
3 place des tapis
69004 Lyon
Tel: +33 4 26 88 00 97
Mobile: +33 6 32 77 00 21

May 17 '07 #9

P: n/a
"Steve Holden" schrieb
>
Try it on a file that reads something like

xxx = 42
print xxx

and you will see NameError raised because the assignment
hasn't affected the environment for the print statement.
[...]
No, because there isn't one. Now try adding a function
definition and see how well it works.
C:\temp>more question.py
xxx=42
print xxx
def sowhat():
print xxx

print xxx
C:\temp>c:\programme\python\python
Python 2.4 (#60, Nov 30 2004, 11:49:19)
[MSC v.1310 32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license"
for more information.
>>exec open("question.py").read()
42
42
>>sowhat()
42
>>xxx
42
Seems to work great to me.

OTOH, this doesn't:
>>inp=open("question.py")
for l in inp:
.... exec l
....
42
Traceback (most recent call last):
File "<stdin>", line 2, in ?
File "<string>", line 1
def sowhat():
^
SyntaxError: unexpected EOF while parsing
So it seems to depend on the way the file is read.
Regards
Martin

May 17 '07 #10

P: n/a
On May 17, 3:02 am, Douglas Woodrow <newsgro...@nospam.demon.co.uk>
wrote:
On Thu, 17 May 2007 00:30:23, i3dmaster <i3dmas...@gmail.comwrote
f = open(file,'rb')
for i in f:
exec i

Why are you opening the file in binary mode?

--
Doug Woodrow
'b' is generally useful on systems that don't treat binary and text
files differently. It will improve portability.

May 17 '07 #11

P: n/a
Martin Blume wrote:
"Steve Holden" schrieb
>>>Try it on a file that reads something like

xxx = 42
print xxx

and you will see NameError raised because the assignment
hasn't affected the environment for the print statement.

[...]
No, because there isn't one. Now try adding a function
definition and see how well it works.
C:\temp>more question.py
xxx=42
print xxx
def sowhat():
print xxx

print xxx
C:\temp>c:\programme\python\python
Python 2.4 (#60, Nov 30 2004, 11:49:19)
[MSC v.1310 32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license"
for more information.
>>>exec open("question.py").read()
42
42
>>>sowhat()
42
>>>xxx
42
Seems to work great to me.

OTOH, this doesn't:
>>>inp=open("question.py")
for l in inp:
... exec l
...
42
Traceback (most recent call last):
File "<stdin>", line 2, in ?
File "<string>", line 1
def sowhat():
^
SyntaxError: unexpected EOF while parsing
So it seems to depend on the way the file is read.
It depends on the way the lines of the file are executed, not how they
are read. And you may remember the original poster was proposing this:

inp = open(cmd_file)
for line in inp:
exec line
As for your first example, why not just use execfile() ?

regards
Steve
--
Steve Holden +1 571 484 6266 +1 800 494 3119
Holden Web LLC/Ltd http://www.holdenweb.com
Skype: holdenweb http://del.icio.us/steve.holden
------------------ Asciimercial ---------------------
Get on the web: Blog, lens and tag your way to fame!!
holdenweb.blogspot.com squidoo.com/pythonology
tagged items: del.icio.us/steve.holden/python
All these services currently offer free registration!
-------------- Thank You for Reading ----------------

May 17 '07 #12

P: n/a
On Fri, 18 May 2007 04:45:30, Dennis Lee Bieber <wl*****@ix.netcom.com>
wrote
>On 17 May 2007 13:12:10 -0700, i3dmaster <i3*******@gmail.comdeclaimed
the following in comp.lang.python:
>'b' is generally useful on systems that don't treat binary and text
files differently. It will improve portability.

"b" is needed for binary files on systems that /do/ treat binary
differently from text. And it does add to portability only in that it
has no effect on those that treat all files the same.

However, as I recall the thread, the intent is to process text lines
from a file -- and using "b" is going to affect how the line endings are
being treated.
Yes that was my understanding too, Dennis, and the reason I queried it
in the first place. I had to remove the "b" option in order to get the
sample code to work under Windows, because the standard line termination
under Windows is carriage return + linefeed (\r\n).

Of course if I manually edit the command file so that it only has a
linefeed character at the end of each line, the binary mode works.

So I think i3dmaster's method is only portable as long as the command
file is created with unix-style line termination.

--
Doug Woodrow

May 18 '07 #13

P: n/a
"Steve Holden" schrieb

[ difference between exec open(fname).read()
and for line in open(fname): exec line ]

So it seems to depend on the way the file is read.
It depends on the way the lines of the file are executed,
not how they are read.
Could you elaborate a little bit more on the difference?
I assumed that because read() reads the whole file, the
body of my function sowhat() is present, so that it can
be parsed while the invocation of exec is still running.
If it is read and exec'd line by line, the definition of
the function is still left open at the moment exec() ends,
causing the "EOF" error. Hence my statement, "it depends
on the way the file is read".

And you may remember the original poster was
proposing this:

inp = open(cmd_file)
for line in inp:
exec line

As for your first example, why not just use execfile() ?
I assume that
execfile(fname)
is equivalent to
exec open(fname).read() ?
Regards
Martin
May 19 '07 #14

P: n/a
Martin Blume wrote:
"Steve Holden" schrieb
>>[ difference between exec open(fname).read()
and for line in open(fname): exec line ]

So it seems to depend on the way the file is read.
It depends on the way the lines of the file are executed,
not how they are read.
Could you elaborate a little bit more on the difference?
I assumed that because read() reads the whole file, the
body of my function sowhat() is present, so that it can
be parsed while the invocation of exec is still running.
If it is read and exec'd line by line, the definition of
the function is still left open at the moment exec() ends,
causing the "EOF" error. Hence my statement, "it depends
on the way the file is read".
I simply meant that the whole source has to be presented to the exec
statement and not chunked into lines.

Clearly I could read all the source in with

lines = open(cmd_file).readlines()

but if you then proceed to try and execute the source line by line as in

for l in lines:
exec l

you will hit problems because of the disjoint nature of the execution
which will breal up indented suites and so on.

I was probably just a little over-zealous in pursuing correct English
usage, in which case please accept my apology.
>
>And you may remember the original poster was
proposing this:

inp = open(cmd_file)
for line in inp:
exec line

As for your first example, why not just use execfile() ?
I assume that
execfile(fname)
is equivalent to
exec open(fname).read() ?
Pretty much.

regards
Steve
--
Steve Holden +1 571 484 6266 +1 800 494 3119
Holden Web LLC/Ltd http://www.holdenweb.com
Skype: holdenweb http://del.icio.us/steve.holden
------------------ Asciimercial ---------------------
Get on the web: Blog, lens and tag your way to fame!!
holdenweb.blogspot.com squidoo.com/pythonology
tagged items: del.icio.us/steve.holden/python
All these services currently offer free registration!
-------------- Thank You for Reading ----------------

May 19 '07 #15

P: n/a
"Steve Holden" schrieb
>
I simply meant that the whole source has to be presented
to the exec statement and not chunked into lines.
That's what I meant: With exec open(f).read() it is not
broken into several exec invocations.
>
I was probably just a little over-zealous in pursuing
correct English usage, in which case please accept
my apology.
The apology is on my part, I didn't explain my thinking
clearly enough.
Thanks for your explanations. Makes my newbie understanding
of Python much more robust.

Regards
Martin
May 19 '07 #16

This discussion thread is closed

Replies have been disabled for this discussion.