472,811 Members | 1,321 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 472,811 software developers and data experts.

reliable unit test logging

Hi all!

I have many many many python unit test, which are used for testing some
remote web service.

The most important issue here is logging of test execution process and
result. I strongly need following:
1. start/end timestamp for each test case (most important)
2. immediate report about exceptions (stacktrace)
3. it will be nice to use logging module for output

I have investigated some extension for standard unittest module, e.g.
testoob, nose, etc. They have very nice features but they does not
satisfy first requirement in my list - test execution logs not doesn't
contain timestamps.

Can someone explain my why so simple feature like logging of timestamps
during test execution was not implemented in any extension?

Also i need some advices about how i can implement timestamps logging
myself. I think proper way is develop customized TextTestRunner and use
logging module instead of "print"s. Is it right way or there is more simply?

Oct 1 '07 #1
6 5601
Vyacheslav Maslov <vm*****@swsoft.comwrites:
I have many many many python unit test, which are used for testing
some remote web service.
Part of your confusion comes from the fact that "test a remote
service" isn't what a unit test does.

A unit test is one that executes a very *limited* part of the code: it
tests a code unit, not the whole system, and makes a simple set of
assertions about the result. If there are accesses to remote services
over the network, that's far beyond the scope of a unit test.

I don't doubt that you may be using the Python standard library module
'unittest' to perform these tests. But they're not unit tests; they're
integration tests, or system tests, or performance tests, or something
else.
Can someone explain my why so simple feature like logging of
timestamps during test execution was not implemented in any
extension?
Probably because the most important thing to know for the purpose of a
unit test is whether the yes/no assertions were violated. Knowing when
the tests start and finish isn't interesting. If start and finish
times *are* interesting for your tests, you're *not* doing unit
testing, but some other form of testing like performance tests.
That said, you *can* certainly instrument the unittest.TestCase with
timings if you want to use it for a performance test. Every instance
of a TestCase will invoke its setUp method to set up test fixtures,
and its tearDown method to tear down fixtures. You can use that hook
to implement logging.

Untested code, that should give you enough to try it out yourself:

import unittest
import logging

logging.basicConfig(level=logging.INFO,
datefmt="%Y-%m-%d %H:%M:%S",
format="%(asctime)s:%(levelname)s:%(message)s")

class TimedTestCase(unittest.TestCase):
""" A test case that will log its start and end times """

def setUp(self):
""" Set up test fixtures """
logging.info("test case setup")

def tearDown(self):
""" Tear down test fixtures """
logging.info("test case teardown")

class Test_FooBar(TimedTestCase):
""" Test cases for FooBar """

def test_slices_spam(self):
""" FooBar should slice spam """
self.failUnless(sends_spam)

def test_eats_eggs(self):
""" FooBar should eat eggs """
self.failUnless(eats_eggs)

--
\ "The World is not dangerous because of those who do harm but |
`\ because of those who look at it without doing anything." |
_o__) —Albert Einstein |
Ben Finney
Oct 1 '07 #2
Ben Finney wrote:
Vyacheslav Maslov <vm*****@swsoft.comwrites:
>I have many many many python unit test, which are used for testing
some remote web service.

Part of your confusion comes from the fact that "test a remote
service" isn't what a unit test does.

A unit test is one that executes a very *limited* part of the code: it
tests a code unit, not the whole system, and makes a simple set of
assertions about the result. If there are accesses to remote services
over the network, that's far beyond the scope of a unit test.

I don't doubt that you may be using the Python standard library module
'unittest' to perform these tests. But they're not unit tests; they're
integration tests, or system tests, or performance tests, or something
else.
>Can someone explain my why so simple feature like logging of
timestamps during test execution was not implemented in any
extension?

Probably because the most important thing to know for the purpose of a
unit test is whether the yes/no assertions were violated. Knowing when
the tests start and finish isn't interesting. If start and finish
times *are* interesting for your tests, you're *not* doing unit
testing, but some other form of testing like performance tests.
I understand your opinion, you are right, i use unit tests for some
other kind of work. But anyway it works and produce good results for
project.
Untested code, that should give you enough to try it out yourself:
Thanks i will look into this.

Oct 2 '07 #3
Vyacheslav Maslov <vm*****@swsoft.comwrites:
I understand your opinion
Hopefully you mean "explanation", not "opinion". I gave what appear to
me to be facts, not opinion, about the definition of a unit test.
you are right, i use unit tests for some other kind of work.
More accurately: you use the Python 'unittest' framework for some
tests that are not unit tests.
But anyway it works and produce good results for project.
Indeed, there's nothing wrong with using the module this way. The only
trouble in this case is confused terminology, that has led you to
believe the module is deficient, when actually it's doing the job it's
meant to do.
Thanks i will look into this.
Glad to help.

--
\ "Holy uncanny photographic mental processes, Batman!" -- Robin |
`\ |
_o__) |
Ben Finney
Oct 2 '07 #4
Ben Finney wrote:
Vyacheslav Maslov <vm*****@swsoft.comwrites:
>I understand your opinion

Hopefully you mean "explanation", not "opinion". I gave what appear to
me to be facts, not opinion, about the definition of a unit test.
Yes, i meant "explanation".

I have one more question related to logging module, not unit test. I use
FileHandler to append information to file log, in fact location of log
file depends on some external factor and is calculated during
initialization. Furthermore i want to use configuration file because it
is comfortable way. So i need way to define in configuration file some
variable which should evaluated during logging system initialization, i
try following way:

[handler_hand02]
class=FileHandler
level=NOTSET
formatter=form01
args=(logFileDir+"myfile.log","a",)
logFileDir is defined in scope of module which call
logging.config.fileConfig()

and it produces "name 'logFileDir' is not defined" exception. As i
understand this happens because inside logging module variable
logFileDir is not visible. How i can avoid this?

Thanks!

--
Oct 3 '07 #5
Vyacheslav Maslov <vm*****@swsoft.comwrites:
I have one more question related to logging module, not unit test.
Please do readers a favour, then, and start a new thread (i.e. compose
a new message, not a reply in an existing thread) for unrelated
questions.

--
\ "Philosophy is questions that may never be answered. Religion |
`\ is answers that may never be questioned." —anonymous |
_o__) |
Ben Finney
Oct 4 '07 #6
En Wed, 03 Oct 2007 11:37:57 -0300, Vyacheslav Maslov <vm*****@swsoft.com>
escribi�:
I have one more question related to logging module, not unit test. I use
FileHandler to append information to file log, in fact location of log
file depends on some external factor and is calculated during
initialization. Furthermore i want to use configuration file because it
is comfortable way. So i need way to define in configuration file some
variable which should evaluated during logging system initialization, i
try following way:

[handler_hand02]
class=FileHandler
level=NOTSET
formatter=form01
args=(logFileDir+"myfile.log","a",)
logFileDir is defined in scope of module which call
logging.config.fileConfig()

and it produces "name 'logFileDir' is not defined" exception. As i
understand this happens because inside logging module variable
logFileDir is not visible. How i can avoid this?
logging.config uses the ConfigParser class; ConfigParser has some
interpolation mechanism.
I think this should work (but I've not tested it):

[handler_hand02]
class=FileHandler
level=NOTSET
formatter=form01
args=("%(logFileDir)s"+"myfile.log","a",)

and call it using logging.config.fileConfig({"logFileDir":
"your/desired/path"})

--
Gabriel Genellina

Oct 7 '07 #7

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

14
by: | last post by:
Hi! I'm looking for unit-testing tools for .NET. Somthing like Java has --> http://www.junit.org regards, gicio
15
by: Enrique | last post by:
Question I am posting this question again (3rd time) because some issues with my no spam alias. Here it is the question: I have not been able to run unit tests for a VSTO (2005) project. I...
72
by: Jacob | last post by:
I have compiled a set og unit testing recommendations based on my own experience on the concept. Feedback and suggestions for improvements are appreciated: ...
5
by: shuisheng | last post by:
Dear All, I was told that unit test is a powerful tool for progamming. If I am writing a GUI code, is it possible to still using unit test? I have a little experience in using unittest++. But...
176
by: nw | last post by:
Hi, I previously asked for suggestions on teaching testing in C++. Based on some of the replies I received I decided that best way to proceed would be to teach the students how they might write...
1
by: rich_sposato | last post by:
I released version 2.0 of C++ Unit Test Library. You can download it from SourceForget.Net at http://sourceforge.net/projects/cppunittest/ .. I wrote this unit test library because other unit...
1
by: Richard Lewis Haggard | last post by:
We're using VS05 and today the units tests have stopped working in our development environment. I'm sure that it is something really silly and simple but I'll be darned if I can figure out what it...
27
by: brad | last post by:
Does anyone else feel that unittesting is too much work? Not in general, just the official unittest module for small to medium sized projects? It seems easier to write some quick methods that are...
5
by: Ben Finney | last post by:
Howdy all, PEP 299 <URL:http://www.python.org/dev/peps/pep-0299details an enhancement for entry points to Python programs: a module attribute (named '__main__') that will be automatically called...
0
linyimin
by: linyimin | last post by:
Spring Startup Analyzer generates an interactive Spring application startup report that lets you understand what contributes to the application startup time and helps to optimize it. Support for...
0
by: erikbower65 | last post by:
Here's a concise step-by-step guide for manually installing IntelliJ IDEA: 1. Download: Visit the official JetBrains website and download the IntelliJ IDEA Community or Ultimate edition based on...
0
by: kcodez | last post by:
As a H5 game development enthusiast, I recently wrote a very interesting little game - Toy Claw ((http://claw.kjeek.com/))。Here I will summarize and share the development experience here, and hope it...
2
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 6 Sept 2023 starting at 18:00 UK time (6PM UTC+1) and finishing at about 19:15 (7.15PM) The start time is equivalent to 19:00 (7PM) in Central...
14
DJRhino1175
by: DJRhino1175 | last post by:
When I run this code I get an error, its Run-time error# 424 Object required...This is my first attempt at doing something like this. I test the entire code and it worked until I added this - If...
5
by: DJRhino | last post by:
Private Sub CboDrawingID_BeforeUpdate(Cancel As Integer) If = 310029923 Or 310030138 Or 310030152 Or 310030346 Or 310030348 Or _ 310030356 Or 310030359 Or 310030362 Or...
0
by: lllomh | last post by:
Define the method first this.state = { buttonBackgroundColor: 'green', isBlinking: false, // A new status is added to identify whether the button is blinking or not } autoStart=()=>{
0
by: lllomh | last post by:
How does React native implement an English player?
2
by: DJRhino | last post by:
Was curious if anyone else was having this same issue or not.... I was just Up/Down graded to windows 11 and now my access combo boxes are not acting right. With win 10 I could start typing...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.