473,287 Members | 1,659 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,287 software developers and data experts.

PEP 299 and unit testing

Howdy all,

PEP 299 <URL:http://www.python.org/dev/peps/pep-0299details an
enhancement for entry points to Python programs: a module attribute
(named '__main__') that will be automatically called if the module is
run as a program.

The PEP has status "Rejected", citing backward-compatibility issues,
and Guido's pronouncement that "It's not worth the change (in docs,
user habits, etc.) and there's nothing particularly broken."

I don't deny the backward-compatibility issues in the cited
discussion, but I'd like to point out one thing that is broken by
this: unit testing of program modules.
Unit tests need to import a module and introspectively test small
units from the module to verify their behaviour in isolation. The
boundary of a unit test is the code that's actually in the module
under test: any functional code in that module needs to be tested by
the module's unit test, any code not in that module is outside the
scope of that unit test module.

The logical extension of this is to put *all* functional code into
discrete units, including the "main line" code that gets executed when
the module is run as a program. This leads to code of the type
discussed in PEP 299:

def main(argv):
""" Do the main stuff of this program """
parse_commandline(argv)
try:
do_interesting_things()
except SystemExit, e:
exitcode = e.code
return exitcode

if __name__ == "__main__":
import sys
exitcode = main(sys.argv)
sys.exit(exitcode)

This allows the module's 'main' function to be called as a discrete
unit from the unit test module; the unit test passes in 'argv' as
desired, and fakes out other units that aren't being tested.

What it doesn't allow is for the testing of the 'if __name__ ==
"__main__":' clause itself. No matter how simple we make that, it's
still functional code that can contain errors, be they obvious or
subtle; yet it's code that *can't* be touched by the unit test (by
design, it doesn't execute when the module is imported), leading to
errors that won't be caught as early or easily as they might.

So, I'd argue that "nothing particularly broken" isn't true: unit
testing is flawed in this scenario. It means that even the simple
metric of statement-level test coverage can't ever get to 100%, which
is a problem since it defeats a simple goal of "get all functional
code covered by unit tests".
On the other hand, if PEP 299 *were* implemented (and the
backward-compatibility issues solved), the above could be written as:

def __main__(argv):
""" Do the main stuff of this program """
parse_commandline(argv)
try:
do_interesting_things()
except SystemExit, e:
exitcode = e.code
return exitcode

with no module-level 'if __name__' test at all, and therefore no
functional code unreachable by the unit test module. The effect of the
program is the same, but the invocation of the '__main__' function
isn't left to be implemented in every single program, separately and
subject to error in every case. Instead, it becomes part of the
*external* environment of the module, and is trivially outside the
scope of a unit test module for that program.

--
\ "What I have to do is see, at any rate, that I do not lend |
`\ myself to the wrong which I condemn." -- Henry Thoreau, _Civil |
_o__) Disobedience_ |
Ben Finney
Oct 29 '07 #1
5 2212
Ben Finney wrote:
What it doesn't allow is for the testing of the 'if __name__ ==
"__main__":' clause itself. No matter how simple we make that, it's
still functional code that can contain errors, be they obvious or
subtle; yet it's code that *can't* be touched by the unit test (by
design, it doesn't execute when the module is imported), leading to
errors that won't be caught as early or easily as they might.
You could always use runpy.run_module.

STeVe
Oct 29 '07 #2
Steven Bethard <st************@gmail.comwrites:
Ben Finney wrote:
What it doesn't allow is for the testing of the 'if __name__ ==
"__main__":' clause itself. No matter how simple we make that,
it's still functional code that can contain errors, be they
obvious or subtle; yet it's code that *can't* be touched by the
unit test (by design, it doesn't execute when the module is
imported), leading to errors that won't be caught as early or
easily as they might.

You could always use runpy.run_module.
For values of "always" that include Python 2.5, of course. (I'm still
coding to Python 2.4, until 2.5 is more widespread.)

Thanks! I was unaware of that module. It does seem to nicely address
the issue I discussed.

--
\ "Pinky, are you pondering what I'm pondering?" "I think so, |
`\ Brain, but Zero Mostel times anything will still give you Zero |
_o__) Mostel." -- _Pinky and The Brain_ |
Ben Finney
Oct 29 '07 #3
Ben Finney wrote:
Steven Bethard <st************@gmail.comwrites:
>Ben Finney wrote:
>>What it doesn't allow is for the testing of the 'if __name__ ==
"__main__":' clause itself. No matter how simple we make that,
it's still functional code that can contain errors, be they
obvious or subtle; yet it's code that *can't* be touched by the
unit test (by design, it doesn't execute when the module is
imported), leading to errors that won't be caught as early or
easily as they might.
You could always use runpy.run_module.

For values of "always" that include Python 2.5, of course. (I'm still
coding to Python 2.4, until 2.5 is more widespread.)

Thanks! I was unaware of that module. It does seem to nicely address
the issue I discussed.
You might try the runpy module as-is with Python 2.4. I don't know if
it works, but it's pure Python so it's worth a try.

STeVe
Oct 29 '07 #4
Steven Bethard <st************@gmail.comwrites:
Ben Finney wrote:
Thanks! I was unaware of that module. It does seem to nicely
address the issue I discussed.

You might try the runpy module as-is with Python 2.4. I don't know
if it works, but it's pure Python so it's worth a try.
Drat. It uses (by explicit design) "the standard import mechanism" to
load the module, which means it doesn't work for exactly the thing I'm
trying to do: load a program file *not* named with a '.py' suffix.

I've long been able to load my program modules from no-suffix
filenames (or indeed any non-standard filenames) with this function::

def make_module_from_file(module_name, file_name):
""" Make a new module object from the code in specified file """

from types import ModuleType
module = ModuleType(module_name)

module_file = open(file_name, 'r')
exec module_file in module.__dict__
sys.modules[module_name] = module

return module

Unfortunately, it seems that "module is already present with name
'foo' in 'sys.modules'" is insufficient for the Python import
mechanism. The module loader used by 'runpy' still complains that it
can't find the module, which is no surprise because its filename is
not that of a library module.

Perhaps I need to delve into the details of the import mechanism
myself :-(

--
\ "With Lisp or Forth, a master programmer has unlimited power |
`\ and expressiveness. With Python, even a regular guy can reach |
_o__) for the stars." -- Raymond Hettinger |
Ben Finney
Oct 29 '07 #5
Ben Finney <bi****************@benfinney.id.auwrites:
Steven Bethard <st************@gmail.comwrites:
Ben Finney wrote:
What it doesn't allow is for the testing of the 'if __name__ ==
"__main__":' clause itself. No matter how simple we make that,
it's still functional code that can contain errors, be they
obvious or subtle; yet it's code that *can't* be touched by the
unit test (by design, it doesn't execute when the module is
imported), leading to errors that won't be caught as early or
easily as they might.
You could always use runpy.run_module.

Thanks! I was unaware of that module. It does seem to nicely address
the issue I discussed.
Thinking about it further: I don't think it does address the issue.

Running the *entire* module code again in a single step (as
'run_module' seems to do) would happily overwrite any instrumented
faked attributes of the module that were inserted for the purpose of
unit testing, rendering it useless for unit test purposes.

The issue here is that there is an irreducible amount of functional
code inside the module that cannot be unit tested without running the
entire program with all its side effects.

PEP 299 promises to make that specific small-but-significant code
become an implementation detail in the language runtime, which would
mean it would no longer be prone to errors in the modules themselves,
and thus no longer the topic of a unit test on those modules. I think
100% statement coverage is not possible in Python programs without
this, or something that achieves the same thing.

--
\ "We spend the first twelve months of our children's lives |
`\ teaching them to walk and talk and the next twelve years |
_o__) telling them to sit down and shut up." -- Phyllis Diller |
Ben Finney
Oct 29 '07 #6

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

4
by: Hugh Cowan | last post by:
Hello, I don't program full-time (anymore), but I do try and stay on-top of the latest technologies and like most are always trying to upgrade my skills and remain current (as much as is...
11
by: rhat | last post by:
Hi Everyone, I've recently been reading some articles about unit-testing in Python , but I am a bit confused: where do I go to get started with this? I tried googling for "unittest" but all I've...
14
by: | last post by:
Hi! I'm looking for unit-testing tools for .NET. Somthing like Java has --> http://www.junit.org regards, gicio
4
by: Peter Rilling | last post by:
Does VS.NET 2005 Professional support integrated unit testing, or is that only with the team system?
72
by: Jacob | last post by:
I have compiled a set og unit testing recommendations based on my own experience on the concept. Feedback and suggestions for improvements are appreciated: ...
4
by: Dat AU DUONG | last post by:
Hi, I am new to Unit testing, could you tell me where I could find information (hopefully step by step) and what is the benefit of unit testing. I am a sole developer in a company, therefore I...
5
by: shuisheng | last post by:
Dear All, I was told that unit test is a powerful tool for progamming. If I am writing a GUI code, is it possible to still using unit test? I have a little experience in using unittest++. But...
176
by: nw | last post by:
Hi, I previously asked for suggestions on teaching testing in C++. Based on some of the replies I received I decided that best way to proceed would be to teach the students how they might write...
48
by: Ark Khasin | last post by:
Unit testing is an integral component of both "formal" and "agile" models of development. Alas, it involves a significant amount of tedious labor. There are test automation tools out there but...
0
by: DolphinDB | last post by:
The formulas of 101 quantitative trading alphas used by WorldQuant were presented in the paper 101 Formulaic Alphas. However, some formulas are complex, leading to challenges in calculation. Take...
0
by: Aftab Ahmad | last post by:
So, I have written a code for a cmd called "Send WhatsApp Message" to open and send WhatsApp messaage. The code is given below. Dim IE As Object Set IE =...
0
by: ryjfgjl | last post by:
ExcelToDatabase: batch import excel into database automatically...
0
by: marcoviolo | last post by:
Dear all, I would like to implement on my worksheet an vlookup dynamic , that consider a change of pivot excel via win32com, from an external excel (without open it) and save the new file into a...
1
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 6 Mar 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:15 (7.15PM). In this month's session, we are pleased to welcome back...
0
by: Vimpel783 | last post by:
Hello! Guys, I found this code on the Internet, but I need to modify it a little. It works well, the problem is this: Data is sent from only one cell, in this case B5, but it is necessary that data...
0
by: jfyes | last post by:
As a hardware engineer, after seeing that CEIWEI recently released a new tool for Modbus RTU Over TCP/UDP filtering and monitoring, I actively went to its official website to take a look. It turned...
0
by: ArrayDB | last post by:
The error message I've encountered is; ERROR:root:Error generating model response: exception: access violation writing 0x0000000000005140, which seems to be indicative of an access violation...
1
by: PapaRatzi | last post by:
Hello, I am teaching myself MS Access forms design and Visual Basic. I've created a table to capture a list of Top 30 singles and forms to capture new entries. The final step is a form (unbound)...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.