473,396 Members | 1,608 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,396 software developers and data experts.

reliable unit test logging

Hi all!

I have many many many python unit test, which are used for testing some
remote web service.

The most important issue here is logging of test execution process and
result. I strongly need following:
1. start/end timestamp for each test case (most important)
2. immediate report about exceptions (stacktrace)
3. it will be nice to use logging module for output

I have investigated some extension for standard unittest module, e.g.
testoob, nose, etc. They have very nice features but they does not
satisfy first requirement in my list - test execution logs not doesn't
contain timestamps.

Can someone explain my why so simple feature like logging of timestamps
during test execution was not implemented in any extension?

Also i need some advices about how i can implement timestamps logging
myself. I think proper way is develop customized TextTestRunner and use
logging module instead of "print"s. Is it right way or there is more simply?

Oct 1 '07 #1
6 5660
Vyacheslav Maslov <vm*****@swsoft.comwrites:
I have many many many python unit test, which are used for testing
some remote web service.
Part of your confusion comes from the fact that "test a remote
service" isn't what a unit test does.

A unit test is one that executes a very *limited* part of the code: it
tests a code unit, not the whole system, and makes a simple set of
assertions about the result. If there are accesses to remote services
over the network, that's far beyond the scope of a unit test.

I don't doubt that you may be using the Python standard library module
'unittest' to perform these tests. But they're not unit tests; they're
integration tests, or system tests, or performance tests, or something
else.
Can someone explain my why so simple feature like logging of
timestamps during test execution was not implemented in any
extension?
Probably because the most important thing to know for the purpose of a
unit test is whether the yes/no assertions were violated. Knowing when
the tests start and finish isn't interesting. If start and finish
times *are* interesting for your tests, you're *not* doing unit
testing, but some other form of testing like performance tests.
That said, you *can* certainly instrument the unittest.TestCase with
timings if you want to use it for a performance test. Every instance
of a TestCase will invoke its setUp method to set up test fixtures,
and its tearDown method to tear down fixtures. You can use that hook
to implement logging.

Untested code, that should give you enough to try it out yourself:

import unittest
import logging

logging.basicConfig(level=logging.INFO,
datefmt="%Y-%m-%d %H:%M:%S",
format="%(asctime)s:%(levelname)s:%(message)s")

class TimedTestCase(unittest.TestCase):
""" A test case that will log its start and end times """

def setUp(self):
""" Set up test fixtures """
logging.info("test case setup")

def tearDown(self):
""" Tear down test fixtures """
logging.info("test case teardown")

class Test_FooBar(TimedTestCase):
""" Test cases for FooBar """

def test_slices_spam(self):
""" FooBar should slice spam """
self.failUnless(sends_spam)

def test_eats_eggs(self):
""" FooBar should eat eggs """
self.failUnless(eats_eggs)

--
\ "The World is not dangerous because of those who do harm but |
`\ because of those who look at it without doing anything." |
_o__) —Albert Einstein |
Ben Finney
Oct 1 '07 #2
Ben Finney wrote:
Vyacheslav Maslov <vm*****@swsoft.comwrites:
>I have many many many python unit test, which are used for testing
some remote web service.

Part of your confusion comes from the fact that "test a remote
service" isn't what a unit test does.

A unit test is one that executes a very *limited* part of the code: it
tests a code unit, not the whole system, and makes a simple set of
assertions about the result. If there are accesses to remote services
over the network, that's far beyond the scope of a unit test.

I don't doubt that you may be using the Python standard library module
'unittest' to perform these tests. But they're not unit tests; they're
integration tests, or system tests, or performance tests, or something
else.
>Can someone explain my why so simple feature like logging of
timestamps during test execution was not implemented in any
extension?

Probably because the most important thing to know for the purpose of a
unit test is whether the yes/no assertions were violated. Knowing when
the tests start and finish isn't interesting. If start and finish
times *are* interesting for your tests, you're *not* doing unit
testing, but some other form of testing like performance tests.
I understand your opinion, you are right, i use unit tests for some
other kind of work. But anyway it works and produce good results for
project.
Untested code, that should give you enough to try it out yourself:
Thanks i will look into this.

Oct 2 '07 #3
Vyacheslav Maslov <vm*****@swsoft.comwrites:
I understand your opinion
Hopefully you mean "explanation", not "opinion". I gave what appear to
me to be facts, not opinion, about the definition of a unit test.
you are right, i use unit tests for some other kind of work.
More accurately: you use the Python 'unittest' framework for some
tests that are not unit tests.
But anyway it works and produce good results for project.
Indeed, there's nothing wrong with using the module this way. The only
trouble in this case is confused terminology, that has led you to
believe the module is deficient, when actually it's doing the job it's
meant to do.
Thanks i will look into this.
Glad to help.

--
\ "Holy uncanny photographic mental processes, Batman!" -- Robin |
`\ |
_o__) |
Ben Finney
Oct 2 '07 #4
Ben Finney wrote:
Vyacheslav Maslov <vm*****@swsoft.comwrites:
>I understand your opinion

Hopefully you mean "explanation", not "opinion". I gave what appear to
me to be facts, not opinion, about the definition of a unit test.
Yes, i meant "explanation".

I have one more question related to logging module, not unit test. I use
FileHandler to append information to file log, in fact location of log
file depends on some external factor and is calculated during
initialization. Furthermore i want to use configuration file because it
is comfortable way. So i need way to define in configuration file some
variable which should evaluated during logging system initialization, i
try following way:

[handler_hand02]
class=FileHandler
level=NOTSET
formatter=form01
args=(logFileDir+"myfile.log","a",)
logFileDir is defined in scope of module which call
logging.config.fileConfig()

and it produces "name 'logFileDir' is not defined" exception. As i
understand this happens because inside logging module variable
logFileDir is not visible. How i can avoid this?

Thanks!

--
Oct 3 '07 #5
Vyacheslav Maslov <vm*****@swsoft.comwrites:
I have one more question related to logging module, not unit test.
Please do readers a favour, then, and start a new thread (i.e. compose
a new message, not a reply in an existing thread) for unrelated
questions.

--
\ "Philosophy is questions that may never be answered. Religion |
`\ is answers that may never be questioned." —anonymous |
_o__) |
Ben Finney
Oct 4 '07 #6
En Wed, 03 Oct 2007 11:37:57 -0300, Vyacheslav Maslov <vm*****@swsoft.com>
escribi�:
I have one more question related to logging module, not unit test. I use
FileHandler to append information to file log, in fact location of log
file depends on some external factor and is calculated during
initialization. Furthermore i want to use configuration file because it
is comfortable way. So i need way to define in configuration file some
variable which should evaluated during logging system initialization, i
try following way:

[handler_hand02]
class=FileHandler
level=NOTSET
formatter=form01
args=(logFileDir+"myfile.log","a",)
logFileDir is defined in scope of module which call
logging.config.fileConfig()

and it produces "name 'logFileDir' is not defined" exception. As i
understand this happens because inside logging module variable
logFileDir is not visible. How i can avoid this?
logging.config uses the ConfigParser class; ConfigParser has some
interpolation mechanism.
I think this should work (but I've not tested it):

[handler_hand02]
class=FileHandler
level=NOTSET
formatter=form01
args=("%(logFileDir)s"+"myfile.log","a",)

and call it using logging.config.fileConfig({"logFileDir":
"your/desired/path"})

--
Gabriel Genellina

Oct 7 '07 #7

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

14
by: | last post by:
Hi! I'm looking for unit-testing tools for .NET. Somthing like Java has --> http://www.junit.org regards, gicio
15
by: Enrique | last post by:
Question I am posting this question again (3rd time) because some issues with my no spam alias. Here it is the question: I have not been able to run unit tests for a VSTO (2005) project. I...
72
by: Jacob | last post by:
I have compiled a set og unit testing recommendations based on my own experience on the concept. Feedback and suggestions for improvements are appreciated: ...
5
by: shuisheng | last post by:
Dear All, I was told that unit test is a powerful tool for progamming. If I am writing a GUI code, is it possible to still using unit test? I have a little experience in using unittest++. But...
176
by: nw | last post by:
Hi, I previously asked for suggestions on teaching testing in C++. Based on some of the replies I received I decided that best way to proceed would be to teach the students how they might write...
1
by: rich_sposato | last post by:
I released version 2.0 of C++ Unit Test Library. You can download it from SourceForget.Net at http://sourceforge.net/projects/cppunittest/ .. I wrote this unit test library because other unit...
1
by: Richard Lewis Haggard | last post by:
We're using VS05 and today the units tests have stopped working in our development environment. I'm sure that it is something really silly and simple but I'll be darned if I can figure out what it...
27
by: brad | last post by:
Does anyone else feel that unittesting is too much work? Not in general, just the official unittest module for small to medium sized projects? It seems easier to write some quick methods that are...
5
by: Ben Finney | last post by:
Howdy all, PEP 299 <URL:http://www.python.org/dev/peps/pep-0299details an enhancement for entry points to Python programs: a module attribute (named '__main__') that will be automatically called...
0
by: ryjfgjl | last post by:
In our work, we often receive Excel tables with data in the same format. If we want to analyze these data, it can be difficult to analyze them because the data is spread across multiple Excel files...
0
by: emmanuelkatto | last post by:
Hi All, I am Emmanuel katto from Uganda. I want to ask what challenges you've faced while migrating a website to cloud. Please let me know. Thanks! Emmanuel
1
by: nemocccc | last post by:
hello, everyone, I want to develop a software for my android phone for daily needs, any suggestions?
1
by: Sonnysonu | last post by:
This is the data of csv file 1 2 3 1 2 3 1 2 3 1 2 3 2 3 2 3 3 the lengths should be different i have to store the data by column-wise with in the specific length. suppose the i have to...
0
marktang
by: marktang | last post by:
ONU (Optical Network Unit) is one of the key components for providing high-speed Internet services. Its primary function is to act as an endpoint device located at the user's premises. However,...
0
Oralloy
by: Oralloy | last post by:
Hello folks, I am unable to find appropriate documentation on the type promotion of bit-fields when using the generalised comparison operator "<=>". The problem is that using the GNU compilers,...
0
jinu1996
by: jinu1996 | last post by:
In today's digital age, having a compelling online presence is paramount for businesses aiming to thrive in a competitive landscape. At the heart of this digital strategy lies an intricately woven...
0
by: Hystou | last post by:
Overview: Windows 11 and 10 have less user interface control over operating system update behaviour than previous versions of Windows. In Windows 11 and 10, there is no way to turn off the Windows...
0
agi2029
by: agi2029 | last post by:
Let's talk about the concept of autonomous AI software engineers and no-code agents. These AIs are designed to manage the entire lifecycle of a software development project—planning, coding, testing,...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.