I am getting a strange error with this script:
$ cat doctest-threads.py
""" import time, threading def example():
.... thread.out = []
.... while thread.running:
.... time.sleep(.01)
.... thread.out.append(".") thread = threading.Thread(None, example) thread.running = True; thread.start() time.sleep(.1) thread.running = False print thread.out
['.', '.', '.', '.', '.', '.', '.', '.', '.']
"""
if __name__ == "__main__":
import doctest; doctest.testmod()
$ python doctest-threads.py
Exception in thread Thread-1:
Traceback (most recent call last):
File "/usr/lib/python2.4/threading.py", line 442, in __bootstrap
self.run()
File "/usr/lib/python2.4/threading.py", line 422, in run
self.__target(*self.__args, **self.__kwargs)
File "<doctest __main__[1]>", line 5, in example
NameError: global name 'thread' is not defined
I have found out a workaround, putting 'thread' in the main program
(i.e.
in the globals):
$ cat doctest-threads2.py
""" thread.running = True thread.start() time.sleep(.1) thread.running = False print thread.out
['.', '.', '.', '.', '.', '.', '.', '.', '.']
"""
import time, threading
def example():
thread.out = []
while thread.running:
time.sleep(.01)
thread.out.append(".")
thread = threading.Thread(None, example)
if __name__ == "__main__":
import doctest; doctest.testmod()
However this is strange, since replacing in the first script
globals()["thread"] = threading.Thread(None, example)
does NOT work, so it is not just putting stuff in the globals.
Also, it seems that I cannot reproduce the same error in absense of
threads.
Any idea of what is happening?
Thanks for sharing,
Michele Simionato 5 1781
I don't see the problem with your doctest usage, but what makes you believe that
the code you show below produces exactly 9 dots?
strangely enough, re-working the code to this import time, threading def example():
.... thread.out = []
.... for i in range(9): thread.out.append(".") thread = threading.Thread(None, example) thread.running = True; thread.start() time.sleep(.1) thread.running = False print thread.out
['.', '.', '.', '.', '.', '.', '.', '.', '.']
makes the test "succeed" (though it can fail for some of the same reasons the
original test isn't guaranteed to give 9 dots either).
Jeff
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.6 (GNU/Linux)
iD8DBQFC90uvJd01MZaTXX0RAjbkAJ0WYacQkPI59tXsexlr9k H3kNCFtQCgh2YL
y5ha9MglIY5w/cg5glN1Zgc=
=67II
-----END PGP SIGNATURE-----
[Michele Simionato] I am getting a strange error with this script:
$ cat doctest-threads.py """ import time, threading def example(): ... thread.out = [] ... while thread.running: ... time.sleep(.01) ... thread.out.append(".") thread = threading.Thread(None, example) thread.running = True; thread.start() time.sleep(.1) thread.running = False print thread.out ['.', '.', '.', '.', '.', '.', '.', '.', '.'] """ if __name__ == "__main__": import doctest; doctest.testmod() $ python doctest-threads.py Exception in thread Thread-1: Traceback (most recent call last): File "/usr/lib/python2.4/threading.py", line 442, in __bootstrap self.run() File "/usr/lib/python2.4/threading.py", line 422, in run self.__target(*self.__args, **self.__kwargs) File "<doctest __main__[1]>", line 5, in example NameError: global name 'thread' is not defined
It looks like pure thread-race accident to me. The main program does
nothing to guarantee that the thread is finished before it prints
`thread.out`, neither anything to guarantee that Python doesn't exit
while the thread is still running. Stuff, e.g., a time.sleep(5) after
"thread.running = False", and it's much more likely to work the way
you intended (but still not guaranteed).
A guarantee requires explicit synchronization; adding thread.join()
after "thread.running = False" should be sufficient. That ensures two things:
1. The `example` thread is done before thread.out gets printed.
2. The *main* thread doesn't exit (and Python doesn't start tearing itself
down) while the `example` thread is still running.
The exact output depends on OS scheduling accidents, but I expect
you'll see 10 dots most often.
BTW, trying to coordinate threads with sleep() calls is usually a Bad
Idea; you can't generally expect more from an OS than that it will
treat sleep's argument as a lower bound on the elapsed time the
sleeper actually yields the CPU.
Thank you for your replies Jeff & Tim. The snippet I submitted is
unfortunate,
since I was writing an example (for a Python course I am going to give
in
September) to show that you cannot reliably assume that you will get
exactly 9 dots, because of the limitations of 'sleep'. Mindlessly, I
have
cut & pasted that snippet, but my real question was not "how many dots
I get", it was:
"why the error message talks about 'thread' not being in the globals?"
It's true that I can avoid it with a thread.join() (which I had
forgot),
but still I really cannot understand the reason for such message. Why
it
is so misleading? Can something be done about it?
TIA,
Michele Simionato
[Michele Simionato] Thank you for your replies Jeff & Tim. The snippet I submitted is unfortunate, since I was writing an example (for a Python course I am going to give in September) to show that you cannot reliably assume that you will get exactly 9 dots, because of the limitations of 'sleep'. Mindlessly, I have cut & pasted that snippet, but my real question was not "how many dots I get", it was: "why the error message talks about 'thread' not being in the globals?" It's true that I can avoid it with a thread.join() (which I had forgot), but still I really cannot understand the reason for such message.
Because the program is buggy: synchronizing threads isn't a "do it if
you feel like it" thing, it's essential to correct threaded behavior.
If you're going to show students bad thread practice, they're going to
get mysteries a lot deeper and more damaging than this one <0.5 wink>.
Add some more prints:
""" import time, threading def example():
.... thread.out = []
.... while thread.running:
.... time.sleep(.01)
.... print [11]
.... thread.out.append(".")
.... print [12]
.... print [13] thread = threading.Thread(None, example) thread.running = True; thread.start() time.sleep(.1) thread.running = False print thread.out
['.', '.', '.', '.', '.', '.', '.', '.', '.']
"""
if __name__ == "__main__":
import doctest
doctest.testmod()
print [2]
Here's a typical run on my box:
File "blah.py", line 13, in __main__
Failed example:
time.sleep(.1)
Expected nothing
Got:
[11]
[12]
[11]
[12]
[11]
[12]
[11]
[12]
[11]
[12]
[11]
[12]
[11]
[12]
[11]
[12]
[11]
[12]
************************************************** ********************
1 items had failures:
1 of 7 in __main__
***Test Failed*** 1 failures.
[2]
[11]
Exception in thread Thread-1:
Traceback (most recent call last):
File "C:\Code\python\lib\threading.py", line 444, in __bootstrap
self.run()
File "C:\Code\python\lib\threading.py", line 424, in run
self.__target(*self.__args, **self.__kwargs)
File "<doctest __main__[1]>", line 6, in example
NameError: global name 'thread' is not defined
Note that [2] is printed _while_ the spawned thread is still running
([13] is never printed): the call to doctest.testmod() is completely
finished, but you're still letting a thread spawned _by_ doctest run.
The environment doctest set up for that thread is gone too. Although
it doesn't actually matter in this specific example, because the main
thread (not just doctest) is also entirely done, the Python
interpreter starts tearing itself down. Why that last doesn't matter
in this example would take some time to explain; I don't think it's
needed here, because the test case got into mortal trouble for an
earlier reason.
Why it is so misleading?
Simply because bad thread programming has allowed a thread to run
beyond the time resources it relies on have vanished. It may sound
harsh, but this is tough love <wink>: it's pilot error.
Can something be done about it?
Properly synchronize the thread, to enforce what the code requires but
cannot hope to obtain by blind luck. All it takes is the
thread.join() I suggested. I don't understand why you're fighting
that, because it's basic proper thread practice -- it's not like I
suggested an obscure expert-level hack here. If a student doesn't
know to join() a thread before they rely on that thread being done,
their thread career will be an endless nightmare.
All that said, this specific failure would _happen_ to go away too, if
in doctest's DocTestRunner.run(), the final 'test.globs.clear()" were
removed. If you feel it's necessary to let threads spawned by a
doctest run beyond the time doctest completes, you can arrange to
invoke DocTestRunner.run() with clear_globs=False. That's not an
intended use case, but it will work. The intended use case is
explained in run's docstring:
The examples are run in the namespace `test.globs`. If
`clear_globs` is true (the default), then this namespace will
be cleared after the test runs, to help with garbage
collection. If you would like to examine the namespace after
the test completes, then use `clear_globs=False`.
Tim Peters wrote: Because the program is buggy: synchronizing threads isn't a "do it if you feel like it" thing, it's essential to correct threaded behavior. If you're going to show students bad thread practice, they're going to get mysteries a lot deeper and more damaging than this one <0.5 wink>.
Well, yes, but here I am using bad practices *on purpose*, trying
to figure out the most likely mistakes of my students. I want to show
them
what they should NOT do, and what happens if they do. I personally
don't have
that much practice with threads (for instance, I knew about .join() but
I forgot to use it) so it is also good for me if I do some mistake.
Luckily
the course is not about threads, but I might cover them as an optional
topic.
Properly synchronize the thread, to enforce what the code requires but cannot hope to obtain by blind luck. All it takes is the thread.join() I suggested. I don't understand why you're fighting that, because it's basic proper thread practice -- it's not like I suggested an obscure expert-level hack here.
I am not fighting .join() at all; but I want to know what I should
expect
in case I do a mistake. This is pretty useful in case I had to debug a
threaded program written by my students.
If a student doesn't know to join() a thread before they rely on that thread being done, their thread career will be an endless nightmare.
This is exactly the reason why I want to show them what happens if they
don't use it ;)
All that said, this specific failure would _happen_ to go away too, if in doctest's DocTestRunner.run(), the final 'test.globs.clear()" were removed. If you feel it's necessary to let threads spawned by a doctest run beyond the time doctest completes, you can arrange to invoke DocTestRunner.run() with clear_globs=False.
Perfect, this answers my question and gives me an useful tip about
doctest
globals.
Thanks a lot!
Michele Simionato This thread has been closed and replies have been disabled. Please start a new discussion. Similar topics
by: Pierre Rouleau |
last post by:
I have a problem writing self-testable modules using doctest when these
modules have internationalized strings using gettext _('...').
- The main module of an application (say app.py) calls...
|
by: Alan G Isaac |
last post by:
> python doctest.py -v
Running doctest.__doc__
Trying: .remove(42)
Expecting:
Traceback (most recent call last):
File "<stdin>", line 1, in ?
ValueError: list.remove(x): x not in list
ok...
|
by: Michele Simionato |
last post by:
Some time ago I hacked a custom solution to run doctests
on text files containing documentation. The solution
involved this kind of game:
tester=doctest.Tester(globs={},verbose=1)...
|
by: David MacKay |
last post by:
Hello, I'm a python-list newbie. I've got a question about doctest; perhaps
a bug report.
I really like doctest, but sometimes doctest gives a failure when the output
looks absolutely fine to me...
|
by: Runsun Pan |
last post by:
I intend to use the doctect heavily. For this I am thinking of
coding a class that comes with a built-in doctest functionality.
I'd like to seek for input before I start.
The idea is to have a...
|
by: p.lavarre |
last post by:
From: http://docs.python.org/lib/doctest-soapbox.html ...
Can I somehow tell doctest that it's time to quit?
I ask because not all doctest examples are created equal. Some
failures are...
|
by: Eric Mahurin |
last post by:
Noob here. Just got into python a little over a week ago...
One of the (unique?) things I really like about python is the concept
of doctesting. But, now I want more! Here's what I'd like to...
|
by: thomas.guest |
last post by:
I'm not making progress with the following and would appreciate any
help.
Here's an interpreted Python session.
....
Traceback (most recent call last):
File "<stdin>", line 1, in <module>...
|
by: Bzyczek |
last post by:
Hello,
I have problems with running doctests if I use czech national
characters in UTF-8 encoding.
I have Python script, which begin with encoding definition:
# -*- coding: utf-8 -*-
I...
|
by: Charles Arthur |
last post by:
How do i turn on java script on a villaon, callus and itel keypad mobile phone
|
by: emmanuelkatto |
last post by:
Hi All, I am Emmanuel katto from Uganda. I want to ask what challenges you've faced while migrating a website to cloud.
Please let me know.
Thanks!
Emmanuel
|
by: BarryA |
last post by:
What are the essential steps and strategies outlined in the Data Structures and Algorithms (DSA) roadmap for aspiring data scientists? How can individuals effectively utilize this roadmap to progress...
|
by: nemocccc |
last post by:
hello, everyone, I want to develop a software for my android phone for daily needs, any suggestions?
|
by: Hystou |
last post by:
Most computers default to English, but sometimes we require a different language, especially when relocating. Forgot to request a specific language before your computer shipped? No problem! You can...
|
by: Oralloy |
last post by:
Hello folks,
I am unable to find appropriate documentation on the type promotion of bit-fields when using the generalised comparison operator "<=>".
The problem is that using the GNU compilers,...
|
by: jinu1996 |
last post by:
In today's digital age, having a compelling online presence is paramount for businesses aiming to thrive in a competitive landscape. At the heart of this digital strategy lies an intricately woven...
|
by: tracyyun |
last post by:
Dear forum friends,
With the development of smart home technology, a variety of wireless communication protocols have appeared on the market, such as Zigbee, Z-Wave, Wi-Fi, Bluetooth, etc. Each...
|
by: agi2029 |
last post by:
Let's talk about the concept of autonomous AI software engineers and no-code agents. These AIs are designed to manage the entire lifecycle of a software development project—planning, coding, testing,...
| |