473,322 Members | 1,501 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,322 software developers and data experts.

Logging messages with dictionary args in Python 2.4b1 leads to exception

Hi!

I'm trying to do this in Py2.4b1:

-------------------------------
import logging
values = {'test':'bla'}
logging.log(logging.FATAL, 'Test is %(test)s', values)
-------------------------------

This leads to en exception:

-----------------------------
Traceback (most recent call last):
File "/usr/lib/python2.4/logging/__init__.py", line 692, in emit
msg = self.format(record)
File "/usr/lib/python2.4/logging/__init__.py", line 578, in format
return fmt.format(record)
File "/usr/lib/python2.4/logging/__init__.py", line 368, in format
record.message = record.getMessage()
File "/usr/lib/python2.4/logging/__init__.py", line 239, in getMessage
msg = msg % self.args
TypeError: format requires a mapping
-----------------------------

Now, values /is/ a mapping. I looked into logging/__init__.py, but I can
only see that the '*args' parameter (as which 'values' is fed in) should
have been unpacked like this in logging.log():

apply(root.log, (level, msg)+args, kwargs)

Which should lead to the second argument being '(level, msg, args)' in my
understanding and which should allow using dicts when doing this later on
in LogRecord:

msg = msg % self.args

Now, I'm somewhat confused that this doesn't work and I'm also worried
that this is only done in the module methods and not inside the class that
actually handles the logging (the root object in this case). This should
encourage a diverting behaviour of the module methods and the Logger
object methods with the same name. This looks like a bug to me. Adding the
tuple should /only/ be done inside the class, not in the module functions.

Wouldn't it be better to handle dictionaries directly inside LogRecord?
Something like this would do the trick:

-----------------------
if len(self.args) == 1:
msg = msg % self.args[0] # may be a single value or a dictionary
else:
msg = msg % self.args # looks like a value tuple
-----------------------

Could anyone please comment on this?

Stefan
Jul 18 '05 #1
15 2930
Stefan Behnel <be*******@dvs1.informatik.tu-darmstadt.de> wrote in message news:<cl**********@lnx107.hrz.tu-darmstadt.de>...
Now, I'm somewhat confused that this doesn't work and I'm also worried
that this is only done in the module methods and not inside the class that
actually handles the logging (the root object in this case). This should
encourage a diverting behaviour of the module methods and the Logger
object methods with the same name. This looks like a bug to me. Adding the
tuple should /only/ be done inside the class, not in the module functions.


It's not a bug; however, the use case is not unreasonable, so I'll try
and get the functionality (sent by Stefan to me as a patch) into 2.4
final.

Regards,
Vinay Sajip
Jul 18 '05 #2
Stefan Behnel <be*******@dvs1.informatik.tu-darmstadt.de> wrote in message news:<cl**********@lnx107.hrz.tu-darmstadt.de>...
Hi!

I'm trying to do this in Py2.4b1:

-------------------------------
import logging
values = {'test':'bla'}
logging.log(logging.FATAL, 'Test is %(test)s', values)
-------------------------------

This leads to en exception:

-----------------------------
Traceback (most recent call last):
File "/usr/lib/python2.4/logging/__init__.py", line 692, in emit
msg = self.format(record)
File "/usr/lib/python2.4/logging/__init__.py", line 578, in format
return fmt.format(record)
File "/usr/lib/python2.4/logging/__init__.py", line 368, in format
record.message = record.getMessage()
File "/usr/lib/python2.4/logging/__init__.py", line 239, in getMessage
msg = msg % self.args
TypeError: format requires a mapping
-----------------------------
Not sure about 2.4 but but in 2.3:-
logging.fatal( 'Test is %(test)s', values)

with the same error, however :-
logging.fatal( 'Test is %(test)s' % values)

is fine. An why shouldn't you use it in this fashion?
Now, values /is/ a mapping. I looked into logging/__init__.py, but I can
only see that the '*args' parameter (as which 'values' is fed in) should
have been unpacked like this in logging.log():

apply(root.log, (level, msg)+args, kwargs)

Which should lead to the second argument being '(level, msg, args)' in my
understanding and which should allow using dicts when doing this later on
in LogRecord:

msg = msg % self.args

*args is not a mapping, its a tuple. Positional arguments in the
format string will be found.
Now, I'm somewhat confused that this doesn't work and I'm also worried
that this is only done in the module methods and not inside the class that
actually handles the logging (the root object in this case). This should
encourage a diverting behaviour of the module methods and the Logger
object methods with the same name. This looks like a bug to me. Adding the
tuple should /only/ be done inside the class, not in the module functions.

Wouldn't it be better to handle dictionaries directly inside LogRecord?
Something like this would do the trick:

-----------------------
if len(self.args) == 1:
msg = msg % self.args[0] # may be a single value or a dictionary
else:
msg = msg % self.args # looks like a value tuple
-----------------------

Could anyone please comment on this?

Stefan


I haven't read the source so I punt on commenting.

Regards, Paul Clinch
Jul 18 '05 #3
Paul Clinch schrieb:
logging.fatal( 'Test is %(test)s' % values)
is fine. An why shouldn't you use it in this fashion?


I thought a logging API was usually trying to do costly things only when
it knows it has to. You /could/ do that, but since it doesn't come for
free - why not first do the check if you will actually log this message?

Ok, I know there's an overhead anyway: function calls, object creation,
etc. But copying strings and creating string representations of objects
can be /very/ expensive and should only be done if the log message is
definitely written.

I'd simply say that constructing a log message from arguments is such a
common thing that it is worth providing efficient API support for it. And
I think it's truely counter-pythonic to prevent the usage of dictionaries
at this point.

Stefan
Jul 18 '05 #4
Stefan Behnel wrote:
Paul Clinch schrieb:
logging.fatal( 'Test is %(test)s' % values)
is fine. An why shouldn't you use it in this fashion?

I thought a logging API was usually trying to do costly things only when
it knows it has to. You /could/ do that, but since it doesn't come for
free - why not first do the check if you will actually log this message?

Ok, I know there's an overhead anyway: function calls, object creation,
etc. But copying strings and creating string representations of objects
can be /very/ expensive and should only be done if the log message is
definitely written.

Well that's as may be, but is the logging function going to be taking up
much of your program's run time? I'd suspect not. Premature
optimisation, etc., etc.
I'd simply say that constructing a log message from arguments is such a
common thing that it is worth providing efficient API support for it.
And I think it's truely counter-pythonic to prevent the usage of
dictionaries at this point.

Your original message observed that LogRecord performs

msg = msg % self.args

and you said "this should work"; that was where your mistake lay. The
call was constructed using

apply(root.log, (level, msg)+args, kwargs)

and I presume that args was actually obtained from a formal parameter
*args (allowing the functiopn call to aggregate all non-keyword
arguments). Thus for your call to work the usage in LogRecord would have
had to be

msg = msg % self.args[0]

The point of the interface currently used is that it allows you to pass
a variable number of arguments for substitution into the message when
it's actually built. The *args construct neatly aggregates them into a
tuple. So when the substituion was performed inside LogRecord you were
actually providing a tuple whose single element was a dictionary, an
attempt doomed to failure.

A possible enhancement which might meet your arguments would allow the
user to provide EITHER a tuple of arguments OR a set of keyword
arguments. LogRecord could then say:

if args:
msg = msg % args
else:
msg = msg % kwargs

and you would then be able to make a call like

logging.log(logging.FATAL, 'Test is %(test)s', **values)

But with the code you presented, it's no use bleating that "values *is*
a mapping". While this is certainly true, values was unfortunately just
the only element in the args tuple, and it's the args tuple that's
currently used for substituion. I presume the patch you sent Vijay will
do something similar?

regards
Steve
--
http://www.holdenweb.com
http://pydish.holdenweb.com
Holden Web LLC +1 800 494 3119
Jul 18 '05 #5
Steve Holden schrieb:
A possible enhancement which might meet your arguments would allow the
user to provide EITHER a tuple of arguments OR a set of keyword
arguments. LogRecord could then say:

if args:
msg = msg % args
else:
msg = msg % kwargs

and you would then be able to make a call like

logging.log(logging.FATAL, 'Test is %(test)s', **values)

But with the code you presented, it's no use bleating that "values *is*
a mapping". While this is certainly true, values was unfortunately just
the only element in the args tuple, and it's the args tuple that's
currently used for substituion. I presume the patch you sent Vijay will
do something similar?

What it does is: check if there is only one element in args and then
unpack it.

This leads to three different cases how
msg = msg % args
is evaluated:

1) the tuple is longer than one, i.e. args actually is a tuple
=> current beahaviour

2) the tuple contained one element, but not a dict, i.e. args is a value
=> replace the single template in msg with that value
=> current behaviour

3) the tuple contained one element, which was a dict, i.e. args is a dict
=> replace the dictionary templates in msg with the values in args

I find this very much the expected behaviour, which when used by actual
code looks like this:

log("message %s %d", 1, 2)
or
log("message %(a) %(b)", {'a':1, 'b':2})
I find it absolutely wrong if the second raises an exception. That would
differ from any other common use case of dictionaries in Python.

Even this works:
log("message %s", {'a':1, 'b':2})

So the only (!) change is the additional support for dictionary arguments.
No drawbacks.

I think, this is totally the right thing to do.

Stefan
Jul 18 '05 #6
> I thought a logging API was usually trying to do costly things only when
it knows it has to. You /could/ do that, but since it doesn't come for
free - why not first do the check if you will actually log this message?
Right.
I'd simply say that constructing a log message from arguments is such a
common thing that it is worth providing efficient API support for it. And
I think it's truely counter-pythonic to prevent the usage of dictionaries
at this point.


Patch checked into CVS. Now:
logging.warn('%(string)s and the %(number)d dwarves', {'string':

'Snow White', 'number':7})

gives

WARNING:root:Snow White and the 7 dwarves

Regards,
Vinay
Jul 18 '05 #7
Stefan Behnel <be*******@dvs1.informatik.tu-darmstadt.de> wrote:
...
Even this works:
log("message %s", {'a':1, 'b':2})

So the only (!) change is the additional support for dictionary arguments.
No drawbacks.


Actually, I think that:

log('msg %s', (1, 2, 3))

works w/o your patch but your patch breaks it, so it's not strictly true
that there are no drawbacks, I believe. Possibly the drawback's minor,
though.
Alex
Jul 18 '05 #8
Alex Martelli schrieb:
log('msg %s', (1, 2, 3))

works w/o your patch but your patch breaks it, so it's not strictly true
that there are no drawbacks, I believe. Possibly the drawback's minor,
though.


I just checked that, you're right.

"msg %s" % (1,2,3)

throws an exception, while

"msg %s" % ((1,2,3),)

does not.

It's maybe not that a common use case, I guess, but it's definitely a
deviation from the original interface as in Python 2.3, so it breaks
compatibility.

Maybe we still need a check for the actual type of args?

The only problematic case is a tuple as only argument, everything else
should work. So there are three possible solutions for backward compatibility:

1) document the change :o)
2) check for a dictionary as first argument
3) check for a tuple as first argument

Since the tuple is the only problematic thing you could hand in, I
personally feel a preference for making that the "exception" (i.e. I vote
for 3).

Any comments?

Stefan
Jul 18 '05 #9
> The only problematic case is a tuple as only argument, everything else
should work. So there are three possible solutions for backward compatibility:

1) document the change :o)
2) check for a dictionary as first argument
3) check for a tuple as first argument

Since the tuple is the only problematic thing you could hand in, I
personally feel a preference for making that the "exception" (i.e. I vote
for 3).


I would prefer (2), as the change is catering for the specific use
case of a single dictionary argument. This is what is currently in
CVS.

Regards,
Vinay Sajip
Jul 18 '05 #10
Vinay Sajip wrote:
The only problematic case is a tuple as only argument, everything else
should work. So there are three possible solutions for backward compatibility:

1) document the change :o)
2) check for a dictionary as first argument
3) check for a tuple as first argument

Since the tuple is the only problematic thing you could hand in, I
personally feel a preference for making that the "exception" (i.e. I vote
for 3).

I would prefer (2), as the change is catering for the specific use
case of a single dictionary argument. This is what is currently in
CVS.

Regards,
Vinay Sajip


Couldn't we just use keyword arguments to indicate dictionary-style
substitution? That seems to make a simpler, more elegant solution to me:

if args:
msg = msg % args
else:
msg = msg % kwargs

Or am I (as I so often do) missing something?

regards
Steve
--
http://www.holdenweb.com
http://pydish.holdenweb.com
Holden Web LLC +1 800 494 3119
Jul 18 '05 #11
Vinay Sajip wrote:
I thought a logging API was usually trying to do costly things only when
it knows it has to. You /could/ do that, but since it doesn't come for
free - why not first do the check if you will actually log this message?

Right.

I'd simply say that constructing a log message from arguments is such a
common thing that it is worth providing efficient API support for it. And
I think it's truely counter-pythonic to prevent the usage of dictionaries
at this point.

Patch checked into CVS. Now:

logging.warn('%(string)s and the %(number)d dwarves', {'string':


'Snow White', 'number':7})

gives

WARNING:root:Snow White and the 7 dwarves

Regards,
Vinay


But how much simpler

logging.want("%(string)s and the %(number)d dwarves",
string="Snow White", number=7)

seems (to me). Use kwargs!

regards
Steve
--
http://www.holdenweb.com
http://pydish.holdenweb.com
Holden Web LLC +1 800 494 3119
Jul 18 '05 #12
Steve Holden <st***@holdenweb.com> wrote in message news:<ROXdd.8976$SW3.2862@fed1read01>...
But how much simpler

logging.want("%(string)s and the %(number)d dwarves",
string="Snow White", number=7)


logging.want? ;-)

Well, it was a trivial example. In real use cases, the dict would be a
variable rather than a literal. The problem with using kwargs is that
it could interfere with keyword arguments passed to the logging method
(currently only exc_info, but there may be others in future).

Regards,
Vinay Sajip
Jul 18 '05 #13
Vinay Sajip wrote:
Steve Holden <st***@holdenweb.com> wrote in message news:<ROXdd.8976$SW3.2862@fed1read01>...
But how much simpler

logging.want("%(string)s and the %(number)d dwarves",
string="Snow White", number=7)
logging.want? ;-)

Wishful thinking :-)
Well, it was a trivial example. In real use cases, the dict would be a
variable rather than a literal. The problem with using kwargs is that
it could interfere with keyword arguments passed to the logging method
(currently only exc_info, but there may be others in future).


Indeed, my ignorance of the current API misled me. But *otherwise* it
would have been a good idea, no?

regards
Steve
--
http://www.holdenweb.com
http://pydish.holdenweb.com
Holden Web LLC +1 800 494 3119
Jul 18 '05 #14
On Fri, 22 Oct 2004 07:52:31 -0400, Steve Holden <st***@holdenweb.com> wrote:
Vinay Sajip wrote:
Steve Holden <st***@holdenweb.com> wrote in message news:<ROXdd.8976$SW3.2862@fed1read01>...
But how much simpler

logging.want("%(string)s and the %(number)d dwarves",
string="Snow White", number=7)

logging.want? ;-)

Wishful thinking :-)
Well, it was a trivial example. In real use cases, the dict would be a
variable rather than a literal. The problem with using kwargs is that
it could interfere with keyword arguments passed to the logging method
(currently only exc_info, but there may be others in future).


Indeed, my ignorance of the current API misled me. But *otherwise* it
would have been a good idea, no?


Jumping into unknown wider context here, but (as you know, I'm sure)
as of recent versions you can easily pass a dict arg using keywords:
def foo(*args,**kw): print args, kw ... foo('here:', dict(string="Snow White", number=7),'<-is a dict', this='is a kwarg') ('here:', {'number': 7, 'string': 'Snow White'}, '<-is a dict') {'this': 'is a kwarg'}
^__args tuple_____________________________________________ ____^ ^___kw arg___________^

Less confusing perhaps, the dict arg isolated:
dict(string="Snow White", number=7)

{'number': 7, 'string': 'Snow White'}

Regards,
Bengt Richter
Jul 18 '05 #15
Bengt Richter wrote:
On Fri, 22 Oct 2004 07:52:31 -0400, Steve Holden <st***@holdenweb.com> wrote:

Vinay Sajip wrote:

Steve Holden <st***@holdenweb.com> wrote in message news:<ROXdd.8976$SW3.2862@fed1read01>...
But how much simpler

logging.want("%(string)s and the %(number)d dwarves",
string="Snow White", number=7)
[...]


Jumping into unknown wider context here, but (as you know, I'm sure)
as of recent versions you can easily pass a dict arg using keywords:
>>> def foo(*args,**kw): print args, kw ... >>> foo('here:', dict(string="Snow White", number=7),'<-is a dict', this='is a kwarg') ('here:', {'number': 7, 'string': 'Snow White'}, '<-is a dict') {'this': 'is a kwarg'}
^__args tuple_____________________________________________ ____^ ^___kw arg___________^

Less confusing perhaps, the dict arg isolated:
>>> dict(string="Snow White", number=7)

{'number': 7, 'string': 'Snow White'}

Regards,
Bengt Richter


That strikes me as an acceptable compromise.

regards
Steve
--
http://www.holdenweb.com
http://pydish.holdenweb.com
Holden Web LLC +1 800 494 3119
Jul 18 '05 #16

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

0
by: Robert.Schmitt | last post by:
I found that the configuration system of the new logging package of Python 2.3 has some unintuitive idiosyncracies that are worth mentioning because they can cost you quite some development time...
1
by: j vickroy | last post by:
My system: MSW XP professional Python 2.3.3 logging package: 0.4.9.2 My problem: The log_test3.py script, provided with the logging package distribution, generates an unexpected message: No...
0
by: robert | last post by:
As more and more python packages are starting to use the bloomy (Java-ish) 'logging' module in a mood of responsibility and as I am not overly happy with the current "thickener" style of usage, I...
8
by: akameswaran | last post by:
I wrote up a quick little set of tests, I was acutally comparing ways of doing "case" behavior just to get some performance information. Now two of my test cases had almost identical results which...
6
by: daniel | last post by:
I use a simple program to illustrate the problem: import logging def foo() : raise ValueError("foo") if __name__ == "__main__" : try : foo()
3
by: seb | last post by:
Hi, I am writing to a file some basic information using the logging module. It is working but in the log file some line are printed several time. I had put some print debugging messages in the...
9
by: Jim | last post by:
Hello, I'm trying to write exception-handling code that is OK in the presence of unicode error messages. I seem to have gotten all mixed up and I'd appreciate any un-mixing that anyone can...
6
by: Larry Bates | last post by:
Every time I look at the logging module (up until now) I've given up and continue to use my home-grown logger that I've been using for years. I'm not giving up this time ;-) I find that I...
4
by: Alexandru Mosoi | last post by:
why doesn't logging throw any exception when it should? how do I configure logging to throw exceptions? .... logging.fatal('asdf %d', '123') .... except: .... print 'this line is never...
0
by: DolphinDB | last post by:
Tired of spending countless mintues downsampling your data? Look no further! In this article, you’ll learn how to efficiently downsample 6.48 billion high-frequency records to 61 million...
0
by: ryjfgjl | last post by:
ExcelToDatabase: batch import excel into database automatically...
0
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 6 Mar 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:15 (7.15PM). In this month's session, we are pleased to welcome back...
1
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 6 Mar 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:15 (7.15PM). In this month's session, we are pleased to welcome back...
0
by: Vimpel783 | last post by:
Hello! Guys, I found this code on the Internet, but I need to modify it a little. It works well, the problem is this: Data is sent from only one cell, in this case B5, but it is necessary that data...
0
by: jfyes | last post by:
As a hardware engineer, after seeing that CEIWEI recently released a new tool for Modbus RTU Over TCP/UDP filtering and monitoring, I actively went to its official website to take a look. It turned...
1
by: PapaRatzi | last post by:
Hello, I am teaching myself MS Access forms design and Visual Basic. I've created a table to capture a list of Top 30 singles and forms to capture new entries. The final step is a form (unbound)...
1
by: Shællîpôpï 09 | last post by:
If u are using a keypad phone, how do u turn on JavaScript, to access features like WhatsApp, Facebook, Instagram....
0
by: Faith0G | last post by:
I am starting a new it consulting business and it's been a while since I setup a new website. Is wordpress still the best web based software for hosting a 5 page website? The webpages will be...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.