String formatters with variable argument length

[Fredrik Tolf]

I've been trying to get the string formatting operator (%) to work with
more arguments than the format string requires, but I can find no way to
do that. For example:

>>"%i" % 10

'10'

>>"i" % 10

Traceback (most recent call last):
File "<stdin>", line 1, in ?
TypeError: not all arguments converted during string formatting

The thing is, I want to get format strings from the user, and I don't
want to require the user to consume all the arguments. docs.python.org
doesn't seem to have any clues on how to achieve this, and I can't think
of what to google for.

Could it be as I fear, that it is impossible?

Fredrik Tolf

[sp1d3rx]

How do you know which ones to use? Your best bet is to write a handler
for the TypeError and give a meaningful error message.

Fredrik Tolf wrote:

I've been trying to get the string formatting operator (%) to work with
more arguments than the format string requires, but I can find no way to
do that. For example:

>"%i" % 10

'10'

>"i" % 10

Traceback (most recent call last):
File "<stdin>", line 1, in ?
TypeError: not all arguments converted during string formatting

The thing is, I want to get format strings from the user, and I don't
want to require the user to consume all the arguments. docs.python.org
doesn't seem to have any clues on how to achieve this, and I can't think
of what to google for.

Could it be as I fear, that it is impossible?

Fredrik Tolf

[John Machin]

Fredrik Tolf wrote:

I've been trying to get the string formatting operator (%) to work with
more arguments than the format string requires, but I can find no way to
do that. For example:

>"%i" % 10

'10'

>"i" % 10

Traceback (most recent call last):
File "<stdin>", line 1, in ?
TypeError: not all arguments converted during string formatting

The thing is, I want to get format strings from the user, and I don't
want to require the user to consume all the arguments. docs.python.org
doesn't seem to have any clues on how to achieve this, and I can't think
of what to google for.

Could it be as I fear, that it is impossible?

Three approaches spring to mind. In descending order of my preference:

(a) don't do that

(b) parse the format string, counting the number of args required. If
the user has supplied more, throw them away.

(c) wrap your execution of format_string % args in a try/except
bracket. If you get a TypeError with that message [not guaranteed to
remain constant in the future], throw away the last arg and go around
again.

As a matter of curiosity, why don't you want the user to consume all
the arguments? Don't they get even a teensy-weensy warning message? Are
you writing a Perl interpreter in Python?

Cheers,
John

[John Machin]

John Machin wrote:

Fredrik Tolf wrote:

I've been trying to get the string formatting operator (%) to work with
more arguments than the format string requires, but I can find no way to
do that. For example:

>>"%i" % 10

'10'

>>"i" % 10

Traceback (most recent call last):
File "<stdin>", line 1, in ?
TypeError: not all arguments converted during string formatting

The thing is, I want to get format strings from the user, and I don't
want to require the user to consume all the arguments. docs.python.org
doesn't seem to have any clues on how to achieve this, and I can't think
of what to google for.

Could it be as I fear, that it is impossible?

Three approaches spring to mind. In descending order of my preference:

(a) don't do that

(b) parse the format string, counting the number of args required. If
the user has supplied more, throw them away.

(c) wrap your execution of format_string % args in a try/except
bracket. If you get a TypeError with that message [not guaranteed to
remain constant in the future], throw away the last arg and go around
again.

As a matter of curiosity, why don't you want the user to consume all
the arguments? Don't they get even a teensy-weensy warning message? Are
you writing a Perl interpreter in Python?

Another approach: instead of the "%s %.2f %.5f%%" style, give the users
a list of names of each possible arg so that they can do e.g.

"The rate for %(name)s is %(rate).5f%%"
or
"Amount borrowed: $%(amount).2f; borrower: %(name)s"

HTH,
John

[Fredrik Tolf]

On Thu, 2006-11-30 at 16:26 -0800, John Machin wrote:

Fredrik Tolf wrote:

[...]

The thing is, I want to get format strings from the user, and I don't
want to require the user to consume all the arguments. docs.python.org
doesn't seem to have any clues on how to achieve this, and I can't think
of what to google for.

Three approaches spring to mind. In descending order of my preference:

(a) don't do that

It would be a possibility, since all current uses actually do have the
right number of parameters. I would just like to keep the option
available.

(b) parse the format string, counting the number of args required. If
the user has supplied more, throw them away.

I was thinking of that, but it just seems far too ugly.

(c) wrap your execution of format_string % args in a try/except
bracket. If you get a TypeError with that message [not guaranteed to
remain constant in the future], throw away the last arg and go around
again.

That might be a good possibility. Thanks for the idea! I do consider it
quite a bit ugly, but that often happens when languages try to police
programmers... :P

As a matter of curiosity, why don't you want the user to consume all
the arguments? Don't they get even a teensy-weensy warning message? Are
you writing a Perl interpreter in Python?

Well basically, I'm rewriting a autodownloader for a file-sharing
network in Python (previously written as a bash script, using the printf
command), and I have a number of files scattered over my hard drive
specifying search expressions, into which a potentially optional episode
number can be inserted using sprintf-like arguments (using
fsexpr="`printf "$sexpr" "$curep"`" in bash). I would like to keep it as
a printf parameter, in order to be able to write e.g. %02i, and I would
like to keep it optional, for downloading non-episoded stuff.

I couldn't help noticing that the named variant of the % operator (using
a dict, that is) doesn't require all its arguments to be consumed. Using
that would require me to rewrite *all* the existing files, though.

Anyway, thanks!

Fredrik Tolf

[Matimus]

Could it be as I fear, that it is impossible?

p'shaw, you underestimate the power of the Python...

I think this is a poor solution (I would go with the solution in John
Machin's second post) but here it is anyway since it does essentially
what you asked for:

def print_data(fmtstr):
data = (10,11,12,13)
for i in xrange(len(data)):
try:
print fmtstr %data[:i+1]
break
except:
pass
 

[John Machin]

Fredrik Tolf wrote:

On Thu, 2006-11-30 at 16:26 -0800, John Machin wrote:

Fredrik Tolf wrote:

[...]

The thing is, I want to get format strings from the user, and I don't
want to require the user to consume all the arguments. docs.python.org
doesn't seem to have any clues on how to achieve this, and I can't think
of what to google for.

Three approaches spring to mind. In descending order of my preference:

(a) don't do that

It would be a possibility, since all current uses actually do have the
right number of parameters. I would just like to keep the option
available.

(b) parse the format string, counting the number of args required. If
the user has supplied more, throw them away.

I was thinking of that, but it just seems far too ugly.

what's ugly about this:
[untested]:

def count_format_args(s):
pending = False
count = 0
for c in s:
if c == "%":
# doubled % chars aren't counted
pending = not pending
elif pending:
count += 1
pending = False
return count

output = format % arglist[:count_format_args(format)]

>

(c) wrap your execution of format_string % args in a try/except
bracket. If you get a TypeError with that message [not guaranteed to
remain constant in the future], throw away the last arg and go around
again.

That might be a good possibility. Thanks for the idea! I do consider it
quite a bit ugly, but that often happens when languages try to police
programmers... :P

As a matter of curiosity, why don't you want the user to consume all
the arguments? Don't they get even a teensy-weensy warning message? Are
you writing a Perl interpreter in Python?

Well basically, I'm rewriting a autodownloader for a file-sharing
network in Python (previously written as a bash script, using the printf
command), and I have a number of files scattered over my hard drive
specifying search expressions, into which a potentially optional episode
number can be inserted using sprintf-like arguments (using
fsexpr="`printf "$sexpr" "$curep"`" in bash). I would like to keep it as
a printf parameter, in order to be able to write e.g. %02i, and I would
like to keep it optional, for downloading non-episoded stuff.

I couldn't help noticing that the named variant of the % operator (using
a dict, that is) doesn't require all its arguments to be consumed. Using
that would require me to rewrite *all* the existing files, though.

So offer the named variant as an option for new users or new uses by
old users.

Cheers,
John

[Peter Otten]

John Machin wrote:

Fredrik Tolf wrote:

The thing is, I want to get format strings from the user, and I don't
want to require the user to consume all the arguments.

what's ugly about this:
[untested]:

def count_format_args(s):
pending = False
count = 0
for c in s:
if c == "%":
# doubled % chars aren't counted
pending = not pending
elif pending:
count += 1
pending = False
return count

output = format % arglist[:count_format_args(format)]

Keep in mind, though, that it doesn't take '*' into account:

>>count_format_args("%*.*f")

1

>>"%*.*f" % (3,2,1)

'1.00'

And just because I don't think I've seen it before:

>>count_format_args("%42%")

1

>>"%42%" % ()

' %'

Peter

[John Machin]

Peter Otten wrote:

John Machin wrote:

Fredrik Tolf wrote:

The thing is, I want to get format strings from the user, and I don't
want to require the user to consume all the arguments.

what's ugly about this:
[untested]:

def count_format_args(s):
pending = False
count = 0
for c in s:
if c == "%":
# doubled % chars aren't counted
pending = not pending
elif pending:
count += 1
pending = False
return count

output = format % arglist[:count_format_args(format)]

Keep in mind, though, that it doesn't take '*' into account:

>count_format_args("%*.*f")

1

>"%*.*f" % (3,2,1)

'1.00'

A good point. Adding checking for "*" would make it rather ugly, as "*"
is special only inside a conversion specifier.

>
And just because I don't think I've seen it before:

>count_format_args("%42%")

1

>"%42%" % ()

' %'

Hmmm ... I hadn't seen that before either. I would have said if shown
that input that I could have put in an error check that pending was not
true at the end of the loop, but was relying instead on an exception
from the formatting operator.

Even better: >>"%-42%" % ()
'% '

:-)
Before gentle readers consider nominating the Python core dev team to
thedailyWTF.com, they might wish to:
(1) read the Python documentation
(http://docs.python.org/lib/typesseq-strings.html)
[it is not a simple escape mechanism; the second % is a "conversion
type"]
(2) compare those docs carefully with K&R v2 section B1.2 Formatted
Output (pp 243-245)
(3) note that not everything that emanated from Murray Hill NJ obeyed
the Law of Least Astonishment
:-)

Cheers,
John