string formatting with mapping & '*'... is this a bug?

[Pierre Fortin]

Hi!

"Python Essential Reference" - 2nd Ed, on P. 47 states that a string
format can include "*" for a field width (no restrictions noted); yet...

"%*d" % (6,2) # works as expected ' 2'

Now, with a mapping....
mapping = {'one':1,'two':2}
These work as expected...
"%(two)d" % mapping '2' "%(two)6d" % mapping ' 2'

Attempting to specify a field width via '*' to a mapping fails....
"%(two)*d" % (mapping,6) Traceback (most recent call last):
File "<stdin>", line 1, in ?
TypeError: format requires a mapping "%(two)-*d" % (6,mapping)

Traceback (most recent call last):
File "<stdin>", line 1, in ?
TypeError: format requires a mapping

Am I expecting too much...?

Pierre

[OKB (not okblacke)]

Pierre Fortin wrote:

Attempting to specify a field width via '*' to a mapping fails....

"%(two)*d" % (mapping,6) Traceback (most recent call last):
File "<stdin>", line 1, in ?
TypeError: format requires a mapping "%(two)-*d" % (6,mapping) Traceback (most recent call last):
File "<stdin>", line 1, in ?
TypeError: format requires a mapping

I don't think this has anything to do with the * thing. You can't
mix the sequence and mapping forms of %. For instance, this also fails

m = {'one': 1, 'two': 2}
"%(two)d %d" % (m, 6)

Traceback (most recent call last):
File "<pyshell#8>", line 1, in -toplevel-
"%(two)d %d" % (m, 6)
TypeError: format requires a mapping

The error message is telling you the truth: you're passing the %
operator a tuple, but your format string has a "%(two)" in it, which
means it requires not a tuple but a mapping.

--
--OKB (not okblacke)
Brendan Barnwell
"Do not follow where the path may lead. Go, instead, where there is
no path, and leave a trail."
--author unknown

[Pierre Fortin]

On 9 Sep 2004 18:45:37 GMT OKB wrote:

Pierre Fortin wrote:

Attempting to specify a field width via '*' to a mapping fails....

> "%(two)*d" % (mapping,6)

Traceback (most recent call last):
File "<stdin>", line 1, in ?
TypeError: format requires a mapping

> "%(two)-*d" % (6,mapping)

Traceback (most recent call last):
File "<stdin>", line 1, in ?
TypeError: format requires a mapping

I don't think this has anything to do with the * thing. You can't

mix the sequence and mapping forms of %. For instance, this also fails

m = {'one': 1, 'two': 2}
"%(two)d %d" % (m, 6)
Traceback (most recent call last):
File "<pyshell#8>", line 1, in -toplevel-
"%(two)d %d" % (m, 6)
TypeError: format requires a mapping

The error message is telling you the truth: you're passing the %
operator a tuple, but your format string has a "%(two)" in it, which
means it requires not a tuple but a mapping.

Darn... I was hoping that Python would let me format strings with
mappings in a more readable/maintainable way than thusly:

mapping = {'one':1,'two':2,'size':6}
eval("'%%(two)%(size)dd' %% mapping" % mapping) ' 2'
-or- size=6
eval("'%%(two)%dd' %% mapping" % size)

' 2'

Changing my copy of the book on P.48 to read:
"""
In addition, the asterisk (*) character may be used in place of any number
in any** width field. If present, the width will be read from the next
item in the tuple.

** EXCEPT with mappings described in (1) above
"""
Pierre

[Diez B. Roggisch]

> Darn... I was hoping that Python would let me format strings with

mappings in a more readable/maintainable way than thusly:

Maybe you can simply combine the sequence and the map - if you don't have
numeric keys, that might work. Like this:

vals = ['first' , 'second']
map = {'foo' : 'bar'}

map2 = dict(map.items() + [(str(i), v) for i, v in enumerate(vals)])

print "%(0)s %(foo)s %(1)s" % map2

Not perfect, but IMHO better than your eval-based solution.

--
Regards,

Diez B. Roggisch

[Alex Martelli]

Diez B. Roggisch <de*********@web.de> wrote:
...

map2 = dict(map.items() + [(str(i), v) for i, v in enumerate(vals)])
An equivalent ctor call that's slightly better, IMHO:

map2 = dict([ (str(i),v) for i,v in enumerate(vals) ], **map)

Don't ignore the power of dict's keyword arguments...
Not perfect, but IMHO better than your eval-based solution.

I agree, mine is just a tiny improvement on your post, IMHO.
Alex

[Pierre Fortin]

On Fri, 10 Sep 2004 00:20:24 +0200 Alex wrote:

Diez B. Roggisch <de*********@web.de> wrote:
...

map2 = dict(map.items() + [(str(i), v) for i, v in enumerate(vals)])

An equivalent ctor call that's slightly better, IMHO:

map2 = dict([ (str(i),v) for i,v in enumerate(vals) ], **map)

Don't ignore the power of dict's keyword arguments...

Not perfect, but IMHO better than your eval-based solution.

I agree, mine is just a tiny improvement on your post, IMHO.
Alex

Thanks for that; but it's not what I wanted... (see original post)

I was hoping to use the likes of: "%(key)*.*f" % map
however, unlike with the non-(key) formats, there appears to be no way to
specify a "*.*" size when a map is used...

Pierre

[Pierre Fortin]

On Fri, 10 Sep 2004 13:31:09 +0200 Alex wrote:

You don't need this with Python 2.3 and later -- just call dict with
keyword args, it just works!-) Cool! :^)

# line continuations must be left-justified to avoid extra spaces

nope, you can format as you wish, thanks to Python's handy idea
(mutuated from C) of merging logically adjacent literals:

I'd bumped into er... "issues" a few years ago with "print (...)" so I
never re-explored it... thanks for updating me... :>

Anyway, your solution is neat but it seems a bit repetitive to me.
Agreed... but I've not yet explored classes beyond some really trivial
stuff..

class formatter:
def __init__(self, format_map, default_width=6, default_prec=2):
self.__dict__.update(locals())
def __getitem__(self, varname):
return '%%(%s)%d.%d" % (varname,
self.format_map.get('w'+varname, self.default_width),
self.format_map.get('p'+varname, self.default_prec))

now you can modify your code to do
fmt = formatter(fmt)
OK so far... this works, until....
[[you can actually simplify your fmt dict a lot thanks to the
defaults]], and then code the printing like:

AH-HA!!!! anything _not_ specified is defaulted... took me a minute to
realize that I needed to remove the EXplicits in favor of the IMplicits...
not just remove their values... DUH!! Nice!!

MANY THANKS!!

Pierre

PS: Here's the latest incarnation of the test script... for me, the
"voodoo" part is
self.__dict__.update(locals())
Not sure I could have found that, since without it, the error:
Traceback (most recent call last):
File "./foo", line 40, in ?
print ( "%(Date)ss %(Open)sf %(High)sf %(Low)sf "
File "./foo", line 7, in __getitem__
return "%%(%s)%d.%d" % (varname,
AttributeError: formatter instance has no attribute 'format_map'
seems cryptic until one realizes it's an initilization problem...
Right?

-------------------------
#!/usr/bin/env python

class formatter:
def __init__(self, format_map, default_width=6, default_prec=2):
self.__dict__.update(locals())
def __getitem__(self, varname):
return "%%(%s)%d.%d" % (varname,
self.format_map.get('w'+varname, self.default_width),
self.format_map.get('p'+varname, self.default_prec))

fmt = dict( wDate=10, pDate=10,
wVolume=10, pVolume=0,
wChange=5, pChange=5, )

fmt = formatter(fmt)

# data will be read from several thousand files
sampledata = [
"9-Sep-04,19.49,20.03,19.35,19.93,60077400,19.93",
"8-Sep-04,18.96,19.53,18.92,18.97,52020600,18.96",
"7-Sep-04,18.98,19.18,18.84,18.85,45498100,18.84",
]

change=["down","up","n/c"]

for D in sampledata:
Date, Open, High, Low, Close, Volume, AdjClose = D.split(',')

map = dict(Date=Date,
Open=float(Open),
High=float(High),
Low=float(Low),
Close=float(Close),
Volume=int(Volume),
AdjClose=float(AdjClose),
Change=change[int(float(AdjClose) >= float(Open)) +
int(float(AdjClose) == float(Open))]
)

print ( "%(Date)ss %(Open)sf %(High)sf %(Low)sf "
"%(Close)sf %(Volume)sd %(AdjClose)sf %(Change)ss"
% fmt % map )
----------------------

MUCH simpler/cleaner!! Though I gotta understand how the 2 mappings got
used. Is it as simple as print() iterating over N mappings, taking the
next character(s) for the format..? Elegant if so... though more than 2
mappings could be mind-bending... :^)

Note that I changed "%%(Change)s" to likewise get rid of that oddity...

This has turned into quite a "less is more" education for me... :^)

Thanks again!

[Alex Martelli]

Pierre Fortin <pf*****@pfortin.com> wrote:
...

[[you can actually simplify your fmt dict a lot thanks to the
defaults]], and then code the printing like:
AH-HA!!!! anything _not_ specified is defaulted... took me a minute to
realize that I needed to remove the EXplicits in favor of the IMplicits...
not just remove their values... DUH!! Nice!!

MANY THANKS!!

You're welcome!

Pierre

PS: Here's the latest incarnation of the test script... for me, the
"voodoo" part is
self.__dict__.update(locals())
Ah, sorry, my favourite idiom to avoid the boilerplate of

def __init__(self, fee, fie, foo, fum):
self.fee = fee
self.fie = fie
self.foo = foo
self.fum = fum

Boilerplate is bad, and I'm keen on "Once, and ONLY once!" as a
programming practice, so having to mention each of those names three
times in totally repetitive ways makes me shiver. My favourite idiom
does, per se, leave a silly self.self around (which means a silly
reference loop) so I should remember to 'del self.self' after it...

Not sure I could have found that, since without it, the error:
Traceback (most recent call last):
File "./foo", line 40, in ?
print ( "%(Date)ss %(Open)sf %(High)sf %(Low)sf "
File "./foo", line 7, in __getitem__
return "%%(%s)%d.%d" % (varname,
AttributeError: formatter instance has no attribute 'format_map'
seems cryptic until one realizes it's an initilization problem...
Right?
Right, you have to self.format_map=format_map and so on, or shortcut
that through the above-mentioned idiom.
Date, Open, High, Low, Close, Volume, AdjClose = D.split(',')

map = dict(Date=Date,
Open=float(Open),
High=float(High),
Low=float(Low),
Close=float(Close),
Volume=int(Volume),
AdjClose=float(AdjClose),
Change=change[int(float(AdjClose) >= float(Open)) +
int(float(AdjClose) == float(Open))]
)
I could suggest some deboilerplatization here, too, but maybe that would
be pushing things too far...

print ( "%(Date)ss %(Open)sf %(High)sf %(Low)sf "
"%(Close)sf %(Volume)sd %(AdjClose)sf %(Change)ss"
% fmt % map )
----------------------

MUCH simpler/cleaner!! Though I gotta understand how the 2 mappings got
used. Is it as simple as print() iterating over N mappings, taking the
next character(s) for the format..? Elegant if so... though more than 2
mappings could be mind-bending... :^)
No, print itself has nothing to do with it; % is left associative, so

print something % fmt % map

is equivalent to, say:

temp1 = something % fmt
temp2 = temp1 % map
print temp2

Come to think of that, you SHOULD hoist the "real format" out of the
loop, what I'm calling temp1 (silly name) in this snippet -- the fmt
does not change between legs of the loop, so you don't really want to
recompute the identical temp1 thousands of times (premature optimization
is the root of all evil in programming, sure, but here, this simple
'hoist a constant out of the loop' is just avoiding waste... waste not,
want not... -- and in some sense splitting the format computation out of
the formatting makes things _simpler_!).

Note that I changed "%%(Change)s" to likewise get rid of that oddity...
Yep, makes things more regular. Though the computation of Change itself
is strange, but, whatever... it IS a Python oddity that True+False
equals 1;-).

This has turned into quite a "less is more" education for me... :^)

Thanks again!

And again, you're welcome!
Alex

[Pierre Fortin]

On Fri, 10 Sep 2004 16:34:31 +0200 Alex wrote:

Pierre Fortin <pf*****@pfortin.com> wrote:

PS: Here's the latest incarnation of the test script... for me, the
"voodoo" part is
self.__dict__.update(locals())

Ah, sorry, my favourite idiom to avoid the boilerplate of

def __init__(self, fee, fie, foo, fum):
self.fee = fee
self.fie = fie
self.foo = foo
self.fum = fum

Boilerplate is bad, and I'm keen on "Once, and ONLY once!" as a
programming practice, so having to mention each of those names three
times in totally repetitive ways makes me shiver. My favourite idiom

I fully agree with "once and ONLY once"... but you've pointed out that
"not-at-all is better than once"... :^)
does, per se, leave a silly self.self around (which means a silly
reference loop) so I should remember to 'del self.self' after it...
Since "self" is a convention (could use "this"), t'would be nice if Python
could avoid foo.foo ref. loops in a future release...

map = dict(Date=Date,
Open=float(Open),
High=float(High),
Low=float(Low),
Close=float(Close),
Volume=int(Volume),
AdjClose=float(AdjClose),
Change=change[int(float(AdjClose) >= float(Open)) +
int(float(AdjClose) == float(Open))]
)

I could suggest some deboilerplatization here, too, but maybe that would
be pushing things too far...

Hey! You're on a roll and I'm open for more abu... er education if this
isn't an imposition... :^)
No, print itself has nothing to do with it; % is left associative, so
Of course.... lack of sleep triggered that faulty logic...
Come to think of that, you SHOULD hoist the "real format" out of the
loop,
That's a given in my book... :>
it IS a Python oddity that True+False
equals 1;-).

That's OK with me... it was True+True equals *1* (OR v. +) that forced me
to wrap them with int()'s

Take care,
Pierre

[Jeff Epler]

On Fri, Sep 10, 2004 at 08:27:44AM +0200, Alex Martelli wrote:

Pierre Fortin <pf*****@pfortin.com> wrote:
...

I was hoping to use the likes of: "%(key)*.*f" % map
however, unlike with the non-(key) formats, there appears to be no way to
specify a "*.*" size when a map is used...

It is not clear to me where you would expect to get the values for those
stars from -- 'map' being a dictionary it has no concept of "ordering",
so there is no concept of "next value" after map['key'].

Perhaps Python string formatting should be enhanced similarly to the $
extension (from SuS) in C.
By default, the arguments are used in the order given, where
each `*' and each conversion specifier asks for the next argument
(and it is an error if insufficiently many arguments are
given). One can also specify explicitly which argument is
taken, at each place where an argument is required, by writing
`%m$' instead of `%' and `*m$' instead of `*', where the
decimal integer m denotes the position in the argument list of
the desired argument, indexed starting from 1. Thus,
printf("%*d", width, num);
and
printf("%2$*1$d", width, num);
are equivalent.
In the case of Python, this would become
print "%(width)*(num)d" % {'width': width, 'num': num}

Jeff

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.1 (GNU/Linux)

iD8DBQFBQctPJd01MZaTXX0RAiQfAJ48AyWgk/77DdQQKiD2Pcrtw500MQCdGrkX
/Z8tnGwoT0pcrV773W1Gx0Q=
=wLh0
-----END PGP SIGNATURE-----

[Pierre Fortin]

On Fri, 10 Sep 2004 10:42:08 -0500 Jeff wrote:

In the case of Python, this would become
print "%(width)*(num)d" % {'width': width, 'num': num}

If this did change, I'd prefer:
print "%(num)*(width)d" % {'width': width, 'num': num}
^^^ ^^^^^
in keeping with "in this order" (PyEssenRef 2nd ed. - P.47)

Pierre

[Jeff Epler]

> On Fri, 10 Sep 2004 10:42:08 -0500 Jeff wrote:

In the case of Python, this would become
print "%(width)*(num)d" % {'width': width, 'num': num}

On Fri, Sep 10, 2004 at 12:01:40PM -0400, Pierre Fortin wrote: If this did change, I'd prefer:
print "%(num)*(width)d" % {'width': width, 'num': num}
^^^ ^^^^^
in keeping with "in this order" (PyEssenRef 2nd ed. - P.47)

I think I got my Python example wrong. Either way, I think it should
match the SuS printf extension behavior, but with dict keys instead of
numeric indices.

Jeff

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.1 (GNU/Linux)

iD8DBQFBQdkzJd01MZaTXX0RAvlqAJ9J/KsIPlmQMHSPZs4o2JBT4qRSVwCgrSf+
U9KOTk0IMBc+fZ1CFDL0G04=
=2QkI
-----END PGP SIGNATURE-----

[Alex Martelli]

Pierre Fortin <pf*****@pfortin.com> wrote:
...

self.__dict__.update(locals())

Ah, sorry, my favourite idiom to avoid the boilerplate of

def __init__(self, fee, fie, foo, fum):
self.fee = fee
self.fie = fie
self.foo = foo
self.fum = fum

Boilerplate is bad, and I'm keen on "Once, and ONLY once!" as a
programming practice, so having to mention each of those names three
times in totally repetitive ways makes me shiver. My favourite idiom

I fully agree with "once and ONLY once"... but you've pointed out that
"not-at-all is better than once"... :^)

Well, there is 'once' here -- the names of the attributes are listed
once, in the argument list of 'def __init__'

does, per se, leave a silly self.self around (which means a silly
reference loop) so I should remember to 'del self.self' after it...

Since "self" is a convention (could use "this"), t'would be nice if Python
could avoid foo.foo ref. loops in a future release...

I don't see how to do that cleanly, simply and within general rules.
"Special cases are not special enough to break the rules"...

map = dict(Date=Date,
Open=float(Open),
High=float(High),
Low=float(Low),
Close=float(Close),
Volume=int(Volume),
AdjClose=float(AdjClose),
Change=change[int(float(AdjClose) >= float(Open)) +
int(float(AdjClose) == float(Open))]
)

I could suggest some deboilerplatization here, too, but maybe that would
be pushing things too far...

Hey! You're on a roll and I'm open for more abu... er education if this
isn't an imposition... :^)

Sure, see later.

it IS a Python oddity that True+False
equals 1;-).

That's OK with me... it was True+True equals *1* (OR v. +) that forced me
to wrap them with int()'s

Nope:

True+True

2

so I'm not sure what you mean...?

Anyway, back to map. You have a bunch of local variables you want to
also set into a dict, except that most of them will need to be passed
through some one-argument function first. Change can fall into that
pattern if you just compute a local variable of that name just before
you compute map, so assume you've done that.

Now, once, outside of the loop, you prepare...:

howto_makemap = dict(Date=str, Volume=int,
Change=change.__getitem__)

then, when it's time to prepare map,

map = {}
for k, v in locals():
try: map[k] = howto_makemap.get(k, float)(v)
except (ValueError, TypeError): pass

voila -- that's all you need. You may end up with a few extraneous
entries in map (corresponding to local variables that just happen to be
castable into a float but have nothing to do with 'map') but that ain't
gonna break anything.
Whether I'd do this in production code -- I dunno. It does reduce
boilerplate, but unfortunately it can only do this via a CLEVER trick.
Now CLEVER is a _criticism_, for code that's meant to be read and
maintained... something that happens over and over, like the
self.__dict__update idea, can be forgiven some cleverness, but a one-off
need is *probably* best met with simplicity even if that does mean some
boilerplate. After all, if it's clever, 8 months from now when you need
to touch your code again you'll have a hard time figuring out what you
were doing here. Good code is linear and obvious, either because of
intrinsic simplicity or familiarity through idiomatic (frequent) use...

So, in production code, I think I'd eschew this cleverness and rather
take the boilerplate hit. Simplicity never comes easy to those of us
who are born with a tortuous mind, but it's still worth striving for!
Alex

[Pierre Fortin]

On Fri, 10 Sep 2004 23:39:26 +0200 Alex wrote:

Pierre Fortin <pf*****@pfortin.com> wrote: "Special cases are not special enough to break the rules"...
Touche'

it IS a Python oddity that True+False
equals 1;-).

That's OK with me... it was True+True equals *1* (OR v. +) that
forced me to wrap them with int()'s

Nope:

True+True

2

so I'm not sure what you mean...?

My bad... burned by operator priority...
Change=change[float(AdjClose) >= float(Open) +
float(AdjClose) == float(Open) ]
should read:
Change=change[(float(AdjClose) >= float(Open)) +
(float(AdjClose) == float(Open)) ]

So, in production code, I think I'd eschew this cleverness and rather
take the boilerplate hit. Simplicity never comes easy to those of us
who are born with a tortuous mind, but it's still worth striving for!

Amen.... I've been too clever in the past so I'll agree and leave the
boilerplate...

Thanks!
Pierre
-30-

[Andrew Dalke]

Alex

Ah, sorry, my favourite idiom to avoid the boilerplate of

def __init__(self, fee, fie, foo, fum):
self.fee = fee
self.fie = fie
self.foo = foo
self.fum = fum

A "clever" (a-hem :) solution can use decorators.
It's hard to make the call signature correct though.
Here's a hack solution using exec, which is the wrong
way to do it. For example, it gets the nested scope
wrong for the context of the original function. But
I'm not Raymond Hettinger and I can't fiddle Python's
byte code well enough for a more correct solution.

# Ugly hack! Do not use this! Only for grins!

def store_args(f):
co = f.func_code
# I think this is right way to get the parameter list
argnames = co.co_varnames[:co.co_argcount]
if argnames[:1] != ("self",):
raise TypeError("first parameter must be 'self'")
argnames = argnames[1:]

argdef = list(argnames)
if f.func_defaults:
start = len(argnames) - len(f.func_defaults)
for i in range(len(f.func_default)):
argdef[start+i] = "%s = _defaults[%s]" % (argdef[start+i], i)

lines = ["def %s(self, %s):" % (f.func_name, ", ".join(argdef))]
for argname in argnames:
lines.append(" self.%s = %s" % (argname, argname))

lines.append(" return __func(self, %s)\n" %
(", ".join(argnames), ))

code = "\n".join(lines)

g = f.func_globals.copy()
g["__func"] = f
d = {"_defaults": f.func_defaults}

exec code in g, d
return d[f.func_name]
class Spam:
@store_args
def __init__(self, x, y, z):
pass

a = Spam(3,4,5)
print a.x, a.y, a.z 3 4 5 b = Spam(z=8,x=6,y=7)
print b.x, b.y, b.z 6 7 8

Andrew
da***@dalkescientific.com

[Pierre Fortin]

On Fri, 10 Sep 2004 11:40:48 -0400 I responded to Alex:

On Fri, 10 Sep 2004 16:34:31 +0200 Alex wrote:

Come to think of that, you SHOULD hoist the "real format" out of the
loop,

That's a given in my book... :>

SHEESH!!! Just now dawned on me what you really meant...

# hoist *this* part of the formatting code...!
format = ( "%(Date)ss %(Open)sf %(High)sf %(Low)sf "
"%(Close)sf %(Volume)sd %(AdjClose)sf %(Change)ss"
% fmt )

for ...:
...
print format % map

At least I'm getting it... :>

Pierre
-30-

[Bengt Richter]

On Fri, 10 Sep 2004 23:39:26 +0200, al*****@yahoo.com (Alex Martelli) wrote:

Pierre Fortin <pf*****@pfortin.com> wrote:
...

> > self.__dict__.update(locals())
>
> Ah, sorry, my favourite idiom to avoid the boilerplate of
>
> def __init__(self, fee, fie, foo, fum):
> self.fee = fee
> self.fie = fie
> self.foo = foo
> self.fum = fum
>
> Boilerplate is bad, and I'm keen on "Once, and ONLY once!" as a
> programming practice, so having to mention each of those names three
> times in totally repetitive ways makes me shiver. My favourite idiom

I fully agree with "once and ONLY once"... but you've pointed out that
"not-at-all is better than once"... :^)

Well, there is 'once' here -- the names of the attributes are listed
once, in the argument list of 'def __init__'

> does, per se, leave a silly self.self around (which means a silly
> reference loop) so I should remember to 'del self.self' after it...

Since "self" is a convention (could use "this"), t'would be nice if Python
could avoid foo.foo ref. loops in a future release...

I don't see how to do that cleanly, simply and within general rules.
"Special cases are not special enough to break the rules"...

You could use a helper function to strip out the name, which you know
anyway if you know to write this.__dict__ or self.__dict__, e.g.,

def updd(firstname, locd): ... sansfirst = locd.copy()
... del sansfirst[firstname]
... locd[firstname].__dict__.update(sansfirst)
... class C(object): ... def __init__(a1,a2,a3): updd('a1',locals())
... c=C(2,3)
vars(c) {'a3': 3, 'a2': 2}

The locd.copy etc could be replaced by not using update. Actually, you could
probably inherit from a class with a metaclass that would let you write
just def __init__(a1, a2, a3): pass and have all the methods' __init__s
appropriately modified.

BTW, do you know offhand why somedict.update(obj) seems to use obj.keys() and
obj.__getitem__ if obj has object as base, but not if it has dict as base?
Ran into that trying to wrap locals() to filter out self or whatever:
class D(dict): ... def keys(self): return [1,2,3]
... def __getitem__(self, key): return 2*key
... dd = D({'a':123})
dd.keys() [1, 2, 3] dd[5] 10 dd {'a': 123} d={}
d.update(dd)
d {'a': 123}

Ok, now this: class E(object): ... def keys(self): return [1,2,3]
... def __getitem__(self, key): return 2*key
... def __init__(self, *a): print a
... ee = E({'a':123}) ({'a': 123},) ee.keys() [1, 2, 3] ee[3] 6 ee <__main__.E object at 0x009C72F0> e={}
e.update(ee)
e {1: 2, 2: 4, 3: 6}

The methods are there and do override the keys and __getitem__ of dict: dd.keys(), ee.keys() ([1, 2, 3], [1, 2, 3]) dd[5], ee[5] (10, 10)
But that d.update(dd) doesn't match what you'd think from
---- help({}.update) Help on built-in function update:

update(...)
D.update(E) -> None. Update D from E: for k in E.keys(): D[k] = E[k]
----

I see D.mro() [<class '__main__.D'>, <type 'dict'>, <type 'object'>] E.mro()

[<class '__main__.E'>, <type 'object'>]

Is there another method that update looks for first that serves update better
than keys() and __getitem__ so it ignores the latter? Presumably a method that
dict would have but not object?

ISTM I recall a similar problem with some builtin functions not calling overriding
subclass methods as one might expect?

Regards,
Bengt Richter

[Alex Martelli]

Pierre Fortin <pf*****@pfortin.com> wrote:

On Fri, 10 Sep 2004 11:40:48 -0400 I responded to Alex:

On Fri, 10 Sep 2004 16:34:31 +0200 Alex wrote:

Come to think of that, you SHOULD hoist the "real format" out of the
loop,

That's a given in my book... :>

SHEESH!!! Just now dawned on me what you really meant...

# hoist *this* part of the formatting code...!
format = ( "%(Date)ss %(Open)sf %(High)sf %(Low)sf "
"%(Close)sf %(Volume)sd %(AdjClose)sf %(Change)ss"
% fmt )

for ...:
...
print format % map

At least I'm getting it... :>

Yep, that's exactly what I meant.
Alex

[Alex Martelli]

Andrew Dalke <ad****@mindspring.com> wrote:
...

A "clever" (a-hem :) solution can use decorators.
True.
It's hard to make the call signature correct though.

Maybe module inspect could help -- it's generally the best way to
introspect function signatures in Python. But, perhaps accepting some
sloppiness at the margins, we might code something like (untested!!!):

import itertools

def store_args(f):
co = f.func_code
args = co.co_varnames[1:co.co_argcount]
defs = dict(itertools.izip(reversed(args), f.func_defaults or ()))
def inner(self, *as, **kw):
kkw = dict(defs, **kw)
kkw.update(itertools.izip(args, as))
for kv in kkw.iteritems(): setattr(self, *kv)
return f(self, *as, **kw)
inner.__name__ = f.__name__
return inner

class foo:
@store_args
def __init__(self, a, b, c): pass
def __str__(self): return str(self.__dict__)

f = foo(1, 2, 3); print f
You could add sanity checks such as, f is not allowed a *foo argument
(it makes no sense in this context for f to get arguments of which it
does not know the names). But apart from such issues at the margins,
the signature check happens at inner's call to f -- if any arguments are
missing or not recognized, that call will raise. So, inner's job is
just to consider names and values with the right priority: defaults
first (least priority), then explicitly passed ones (either order since
they can't clash -- if they do the call to f will raise). Here, I build
the auxiliary dict kkw just that way.

I think I still prefer good old self.__dict__.update(locals()),
personally: it's way simpler. Of course, this one has some pluses, such
as calls to setattr rather than __dict__ updates, so it will work in
presence of __slots__, properties, or other weird descriptors, while my
favourite, simpler, more explicit solution doesn't.
Alex

[Andrew Dalke]

Me:

It's hard to make the call signature correct though.

Alex: Maybe module inspect could help -- it's generally the best way to
introspect function signatures in Python. But, perhaps accepting some
sloppiness at the margins, we might code something like (untested!!!):

I had considered that approach. My concern is that
with @decorators current introspection techniques, like
help(), pydoc, and tooltips will become less useful.

For example, here's what help() does on your 'foo' class.

class foo
| Methods defined here:
|
| __init__(self, *as, **kw)
|
| __str__(self)

The inability to propogate the call signature seems to
be an ugly limitation to decorators, and I don't see
an elegant solution to it. I've thought of some hacks,
like calling a function with the original function (f)
and the wrapped function (g), to have it return a new
function (h) with the right signature so that calling
h() pivots and passes things to g() so it can do it's
work then call f().

Matters gets even more complicated with @decorator chains.

BTW, I thought there might be a problem already with
classmethod and staticmethod but it looks like the inspect
module knows how to peer inside to get what it needs.
The same trick won't work for user-defined @decorator
wrappers.

Andrew
da***@dalkescientific.com

[Peter Otten]

I've written a class that automatically generates a format string ready for
dual application of '%'. Instead of

d = dict(val=1.234, width=10, prec=2)
"%%(val)%(width)d.%(prec)df" % d % d ' 1.23'

you can now write
Format("%(val)*(width).*(prec)f") % d ' 1.23'

which is slightly more readable. Not tested beyond what you see.

Peter

import re

class Format:
"""
Extends the format string to allow dict substitution for width and
precision.
Format("%(value)*(width)s") % dict(value=1.234, width=10) ' 1.234'
Format("%(value)*(width).*(prec)f") % dict(value=1.234, width=-10,

prec=2)
'1.23 '

"""
_cache = {}
# Generously allow all ascii characters as format specifiers :-)
rOuter = re.compile(r"(%(\(.*?\)|[^A-Za-z%])*[A-Za-z%])")
rInner = re.compile(r"\*\(.*?\)")

def __init__(self, format):
self.format = self.prepare(format)

def subInner(self, match):
# called for every width/prec specifier, e. g. "*(width)"
s = match.group(0)
return "%" + s[1:] + "s"

def subOuter(self, match):
# called for every complete format, e. g. "%(value)*(width)s"
s = match.group(0)
if s == "%%":
return "%%%%"
return "%" + self.rInner.sub(self.subInner, s)

def prepare(self, format):
""" Modify the format for a two-pass 'format % dict % dict'
appliction. The first pass replaces width/prec specifiers
with integer literals
"""
cached = self._cache.get(format)
if cached is not None:
return cached
result = self._cache[format] = self.rOuter.sub(self.subOuter,
format)
return result

def __mod__(self, dict):
return self.format % dict % dict

if __name__ == "__main__":
f = Format("%(value)*(width).*(prec)f (literal) "
"%(string)s [%(integer)3d] %% [%(integer)-*(width)d]")
print f % dict(value=1.2345, width=5, prec=2, string="so what",
integer=11)
# Abusing your code as a test case...
fmt = { 'wDate':10, 'wOpen':6, 'wHigh':6, 'wLow':6, # width
'wClose':6, 'wVolume':10, 'wAdjClose':6,
'pDate':10, 'pOpen':2, 'pHigh':2, 'pLow':2, # precision
'pClose':2, 'pVolume':0, 'pAdjClose':2 }

# data will be read from several thousand files
sampledata = [
"9-Sep-04,19.49,20.03,19.35,19.93,60077400,19.93",
"8-Sep-04,18.96,19.53,18.92,18.97,52020600,18.96",
"7-Sep-04,18.98,19.18,18.84,18.85,45498100,18.84",
]

change=["down","up","n/c"]

for D in sampledata:
Date, Open, High, Low, Close, Volume, AdjClose = D.split(',')

map = dict(Date=Date,
Open=float(Open),
High=float(High),
Low=float(Low),
Close=float(Close),
Volume=int(Volume),
AdjClose=float(AdjClose),
#
Change=change[int(float(AdjClose) >= float(Open)) +
int(float(AdjClose) == float(Open))]
)
map.update(fmt)
new = Format(
"%(Date)*(wDate).*(pDate)s "
"%(Open)*(wOpen).*(pOpen)f "
"%(High)*(wHigh).*(pHigh)f "
"%(Low)*(wLow).*(pLow)f "
"%(Close)*(wClose).*(pClose)f "
"%(Volume)*(wVolume).*(pVolume)d "
"%(AdjClose)*(wAdjClose).*(pAdjClose)f "
"%(Change)s") % map

old = (
"%%(Date)%(wDate)d.%(pDate)ds "
"%%(Open)%(wOpen)d.%(pOpen)df "
"%%(High)%(wHigh)d.%(pHigh)df "
"%%(Low)%(wLow)d.%(pLow)df "
"%%(Close)%(wClose)d.%(pClose)df "
"%%(Volume)%(wVolume)d.%(pVolume)dd "
"%%(AdjClose)%(wAdjClose)d.%(pAdjClose)df "
"%%(Change)s") % fmt % map

assert old == new