Option parser question - reading options from file as well as commandline

[Andrew Robert]

Hi Everyone.
I tried the following to get input into optionparser from either a file
or command line.
The code below detects the passed file argument and prints the file
contents but the individual swithces do not get passed to option parser.

Doing a test print of options.qmanager shows it unassigned.

Any ideas?

#
# Parse command line options and automatically build help/usage
# display
#
if len(sys.argv) == 2:
infile= open(sys.argv[1],"rb").read()
print infile
parser=OptionParser( infile )
else:
parser = OptionParser()

parser.add_option("-m","--qmanager", dest="qmanager",
help="\t\tQueue Manager to inquire against"),
parser.add_option("-q","--queue", dest="queue",
help="\t\tQueue the message will be sent to"),
parser.add_option("-t","--to", dest="mto",
help="\t\taddress any mail messages will be sent to")
(options, args) = parser.parse_args()

print options.qmanager

[Max Erickson]

Andrew Robert <an************@gmail.com> wrote in
news:12*************@corp.supernews.com:

Hi Everyone.
I tried the following to get input into optionparser from either
a file or command line.
The code below detects the passed file argument and prints the
file contents but the individual swithces do not get passed to
option parser.

Doing a test print of options.qmanager shows it unassigned.

Any ideas?

Check parser.usage, it is likely to look a lot like your infile.

I'm not sure, but I think you need to pass your alternative arguments
to parser.parse_args.

max

[Andrew Robert]

Max Erickson wrote:

Andrew Robert <an************@gmail.com> wrote in
news:12*************@corp.supernews.com:

<snip>

<\snip> Check parser.usage, it is likely to look a lot like your infile.

I'm not sure, but I think you need to pass your alternative arguments
to parser.parse_args.

max

Hi Max,

I tried passing in the read line like so:

infile= open(sys.argv[1],"rb").read()
parser=OptionParser()
print infile
(options, args) = parser.parse_args(infile)

I end up getting the following traceback.

Traceback (most recent call last):
File "C:\Documents and Settings\Andrew Robert\My
Documents\receiver.py", line 327, in ?
(options, args) = parser.parse_args(infile)
File "C:\Python24\lib\optparse.py", line 1275, in parse_args
stop = self._process_args(largs, rargs, values)
File "C:\Python24\lib\optparse.py", line 1322, in _process_args
del rargs[0]
TypeError: object doesn't support item deletion

Any ideas?

[Max Erickson]

Andrew Robert <an************@gmail.com> wrote in
news:12*************@corp.supernews.com:

Any ideas?

I don't know much about optparse, but since I was bored:

help(o.parse_args) Help on method parse_args in module optparse:

parse_args(self, args=None, values=None) method of
optparse.OptionParser instance
parse_args(args : [string] = sys.argv[1:],
values : Values = None)
-> (values : Values, args : [string])

Parse the command-line options found in 'args' (default:
sys.argv[1:]). Any errors result in a call to 'error()', which
by default prints the usage message to stderr and calls
sys.exit() with an error message. On success returns a pair
(values, args) where 'values' is an Values instance (with all
your option values) and 'args' is the list of arguments left
over after parsing options.
o.parse_args('seven') Traceback (most recent call last):
File "<pyshell#15>", line 1, in ?
o.parse_args('seven')
File "C:\bin\Python24\lib\optparse.py", line 1275, in parse_args
stop = self._process_args(largs, rargs, values)
File "C:\bin\Python24\lib\optparse.py", line 1322, in _process_args
del rargs[0]
TypeError: object doesn't support item deletion

That's the result of poking an optionParser instance in Idle.
parse_args is expecting something that looks like sys.argv[1:], which
is a list. You are passing it a string.

max

[Andrew Robert]

Max Erickson wrote:

I don't know much about optparse, but since I was bored:

help(o.parse_args) Help on method parse_args in module optparse:

parse_args(self, args=None, values=None) method of
optparse.OptionParser instance
parse_args(args : [string] = sys.argv[1:],
values : Values = None)
-> (values : Values, args : [string])

Parse the command-line options found in 'args' (default:
sys.argv[1:]). Any errors result in a call to 'error()', which
by default prints the usage message to stderr and calls
sys.exit() with an error message. On success returns a pair
(values, args) where 'values' is an Values instance (with all
your option values) and 'args' is the list of arguments left
over after parsing options.
o.parse_args('seven')

Traceback (most recent call last):
File "<pyshell#15>", line 1, in ?
o.parse_args('seven')
File "C:\bin\Python24\lib\optparse.py", line 1275, in parse_args
stop = self._process_args(largs, rargs, values)
File "C:\bin\Python24\lib\optparse.py", line 1322, in _process_args
del rargs[0]
TypeError: object doesn't support item deletion

That's the result of poking an optionParser instance in Idle.
parse_args is expecting something that looks like sys.argv[1:], which
is a list. You are passing it a string.

max

Yup.. the code now works as:

parser = OptionParser()

if len(sys.argv) == 2:
lines = open(sys.argv[1],"rb").readlines()
for line in lines:
line=line.strip()
if not line:
continue
short, long, dest, help, default = line.split(";")
help = "\t\t" + help # Prepend tabs to help message
parser.add_option(short, long, dest=dest, help=help, default=default)
else:
parser.add_option("-m","--qmanager", dest="qmanager",
help="\t\tQueue Manager to inquire against"),
parser.add_option("-q","--queue", dest="queue",
help="\t\tQueue the message will be sent to"),
parser.add_option("-d","--dest", dest="dest",
help="\t\tDestination File Name"),
parser.add_option("-f", "--file",
action="store", type="string", dest="filename",
help="File to be transmitted", metavar="FILE")

(options, args) = parser.parse_args()
thanks all for the insight

[Tim N. van der Leeuw]

Andrew Robert wrote:

Hi Everyone.
I tried the following to get input into optionparser from either a file
or command line.
The code below detects the passed file argument and prints the file
contents but the individual swithces do not get passed to option parser.

After reading your post I decided to play around with optparse a bit,
to get acquainted with it.

Experimenting with IDLE I found that the Values object returned by
parse_args has a method 'readfile', and this 'readfile' method allows
you to add options to the Values object.

The syntax should be in the form:

<option>=<value>

<option> should not include the hyphens -- if your option is added as
'-x', then you should write:

x=3

not:

-x=3

If you have options with both long names and short names, then you
should use the long name -- if your option is added as '-f', '--file',
then you should write:

file=foo.txt

not:

f=foo.txt
Also, you need the assignment-operator in your file.

I didn't find any documentation on this, and I didn't try this with any
actions; only with options added to the parser like
op.add_option('-x', dest='x')

Using this 2-step approach allows you to use optparse itself to get to
the command-line option with your command-line settings...
Out of pure personal interest, what queuing-system are you writing to
from Python? What libraries do you have for that? And is it commercial
software, or freely downloadable?
(I'd love to be able to write messages to IBM MQ Series from Python)

Cheers,

--Tim

[Andrew Robert]

Tim N. van der Leeuw wrote:

Andrew Robert wrote:

Hi Everyone.
I tried the following to get input into optionparser from either a file
or command line.
The code below detects the passed file argument and prints the file
contents but the individual swithces do not get passed to option parser.

After reading your post I decided to play around with optparse a bit,
to get acquainted with it.

Experimenting with IDLE I found that the Values object returned by
parse_args has a method 'readfile', and this 'readfile' method allows
you to add options to the Values object.

The syntax should be in the form:

<option>=<value>

<option> should not include the hyphens -- if your option is added as
'-x', then you should write:

x=3

not:

-x=3

If you have options with both long names and short names, then you
should use the long name -- if your option is added as '-f', '--file',
then you should write:

file=foo.txt

not:

f=foo.txt
Also, you need the assignment-operator in your file.

I didn't find any documentation on this, and I didn't try this with any
actions; only with options added to the parser like
op.add_option('-x', dest='x')

Using this 2-step approach allows you to use optparse itself to get to
the command-line option with your command-line settings...
Out of pure personal interest, what queuing-system are you writing to
from Python? What libraries do you have for that? And is it commercial
software, or freely downloadable?
(I'd love to be able to write messages to IBM MQ Series from Python)

Cheers,

--Tim

Hi Tim,

I am using the pymqi module which is freely available at
http://pymqi.sourceforge.net/ .

Documentation on the module can be found at
http://pymqi.sourceforge.net/pymqidoc.html .

I have a few python examples on my web site located at
http://home.townisp.com/~arobert/

There are also a lot of good examples at
http://www.koders.com/info.aspx?c=Pr...5ZH7GC9AX54PAC
..

If you come up with anything, I would be glad to see what you have.
Back to the original issue:

I'm not sure exactly what you mean about the readfile option and format.

Could you send me a code snippet so I can get a better feel for it?
Thanks,
Andy

[Tim N. van der Leeuw]

Andrew Robert wrote:

Tim N. van der Leeuw wrote:

Andrew Robert wrote: [...]

Hi Tim,

I am using the pymqi module which is freely available at
http://pymqi.sourceforge.net/ .

Documentation on the module can be found at
http://pymqi.sourceforge.net/pymqidoc.html .

I have a few python examples on my web site located at
http://home.townisp.com/~arobert/

There are also a lot of good examples at
http://www.koders.com/info.aspx?c=Pr...5ZH7GC9AX54PAC
.

If you come up with anything, I would be glad to see what you have.


Thanks a lot for these examples! I have some Java tools that send MQ
messages (reading, in fact, a ton of command-line arguments from a
file) and I need better tools. If I could use some Python for rewriting
this, it might speed me up a lot.

Back to the original issue:

I'm not sure exactly what you mean about the readfile option and format.

Could you send me a code snippet so I can get a better feel for it?
Thanks,
Andy

Here's the file I used:

===cut here===
x=4
w=6
what=7
zoo=9
===cut here===

Here's some snippets of code:

from optparse import OptionParser
op = OptionParser()
op.add_option('-x', dest='x')
op.add_option('--what', '-w', dest='what')
v=op.parse_args()[0]
v.read_file('options-test.txt')
v

<Values at 0x12a9c88: {'x': 4, 'what': 7}>

As you can see, I'm parsing an empty command-line but I could parse a
full command-line as well before loading options from file.
After parsing the command-line, I get an instance of a 'Values' object,
and on this object I call a method 'read_file' with a filename. (I
could also call 'read_module', and it will add the __doc__ string).

I didn't test what happens with a more advanced useage of OptionParser
options.
Cheers,

--Tim

[Tim N. van der Leeuw]

Andrew Robert wrote:

Hi Everyone.
I tried the following to get input into optionparser from either a file
or command line.

Hi Andrew,

I played around a bit more, not happy that the read_file method which I
discovered earlier on the Values - object takes a property-file like
input and doesn't call any callbacks (for instance).

So I wrote a little module to provide an option-parser callback for
reading arguments from file...

Since it seems that Google Groups doesn't allow me to attach the file,
I'll paste it into here, hoping that nobody objects very strongly:

===============================
"""Callback for optparse.OptionParser that interprets a command-line
argument as filename, reads the file, and adds the contents of the file
to the options to be parsed by the OptionParser.

Usage:

from args_from_file import add_argsfile_option
from optparse import OptionValueError
import sys
op = OptionParser()
add_argsfile_option(op, '-f', '--file')
op.parse_args(sys.argv)

(This sample assumes that on the command-line, there is an option '-f'
or '--file'followed by the filename with options you wish to add to
the command-line).

The implementation reads the entire contents of the file into memory.

(c) Copyright 2006 Tim N. van der Leeuw (ti*************@nl.unisys.com)
"""
import re
from optparse import OptionValueError

#SPLIT_ARGS = r'("[^"]*"|\'[^\']*\'|[^\s]+)'
SPLIT_ARGS = r'((?:")([^"]*)(?:")|(?:\')([^\']*)(?:\')|[^\s]+)'
g_split_args_re = None

def extract_args_from_string(arg_str, append_to=None):
global g_split_args_re
if not g_split_args_re:
g_split_args_re = re.compile(SPLIT_ARGS)
argv = g_split_args_re.findall(arg_str)
argv = ([v for v in x if v <> ''][-1] for x in argv)
if not append_to is None:
return append_to.extend(argv)
else:
return list(argv)

def load_file(filename, skip_comments=True):
f = None
try:
f = open(filename, 'rb', 16384)
if skip_comments:
lines = "".join((line for line in f if line.strip()[0] <>
'#'))
else:
lines = "".join(f)
return lines
finally:
if f <> None:
f.close()

def optparse_cb_load_args_from_file(option, opt, value, parser):
try:
args = load_file(value)
extract_args_from_string(args, parser.rargs)
except Exception, e:
raise OptionValueError(
'Error parsing argument %s, with reading options from
option-file "%s". Originating error: %s' %
(opt, value, e))
return

def add_argsfile_option(parser, *option_names):
return parser.add_option(*option_names,
**{'type':'string',
'action':'callback',
'callback':optparse_cb_load_args_from_file})
===============================

Feel free to use as you like. Might make a nice addition to the
standard action types of optparse in the standard library? If there's
interest in that, I might try to write a patch for optparse.

Cheers,

--Tim