Interesting Thread Gotcha

I thought I would share this nasty little gotcha with the group.

Consider the following code fragment:

<start>
print 'starting kbd thread'
keyboard_thread = thread.start_new_thread(kbd_driver (port_q,kbd_q))
print 'starting main loop'
error = Mainloop(s,port_q,active_q_list)
<end>

It produces, as output, the following:

starting kbd thread
we get here - a

It does not print 'starting main loop', the Mainloop routine
is never executed, and no exceptions are raised.

Here is the offending routine that seems to capture the control:

<start>
def kbd_driver(out_q,in_q):
"""
thread to look for keyboard input and to put it on the queue out_q
also looks for replies on in_q and prints them
"""

kbdname = '/dev/stdin'

kbd = open(kbdname,'r+',1) # Reading, line buffered

unblock(kbd) # Call the magic to unblock keyboard
print 'we get here - a'
while True:

try:
d = kbd.readline() # see if any kbd input
except:
IOError
try:
msg=in_q.get(block=False)
except Queue.Empty:
time.sleep(0.1)
continue
print msg
time.sleep(0.1)
continue
d = d.rstrip() # get rid of line feed
out_q.put([d + '\r',in_q]) # add a carriage return and return q and send
to port
<end>

The unblock is a routine that unblocks a port using fcntl - it
is not the problem. In case you don't believe me, here it is:

def unblock(f):
"""Given file 'f', sets its unblock flag to true."""

fcntl.fcntl(f.fileno(), fcntl.F_SETFL, os.O_NONBLOCK)

I will post the solution tomorrow when I read my mail,
if no one has spotted it by then.

- Hendrik

>>keyboard_thread = thread.start_new_thread(kbd_driver (port_q,kbd_q))

>>keyboard_thread = thread.start_new_thread(kbd_driver, (port_q,kbd_q))

>keyboard_thread = thread.start_new_thread(kbd_driver (port_q,kbd_q))

Needs to be

>keyboard_thread = thread.start_new_thread(kbd_driver, (port_q,kbd_q))

Commas are important!

-Dan

It would have been nice, however, to have gotten something like:

TypeError - This routine needs a tuple.

instead of the silent in line calling of the routine in question,
while failing actually to start a new thread.

>
It seems to act no different from plain old:

kbd_driver (port_q,kbd_q)

Is it worth the trouble of learning how to submit a bug report?

Absolutely! - well spotted!

It would have been nice, however, to have gotten something like:

TypeError - This routine needs a tuple.

instead of the silent in line calling of the routine in question,
while failing actually to start a new thread.

Is it worth the trouble of learning how to submit a bug report?

"Dan" <the,,,ail.comwrote:

>>keyboard_thread = thread.start_new_thread(kbd_driver (port_q,kbd_q))

Needs to be

>>keyboard_thread = thread.start_new_thread(kbd_driver, (port_q,kbd_q))

Commas are important!

-Dan

Absolutely! - well spotted!

As the first correct respondent, you win the freedom to spend a week in
Naboomspruit at your own expense.

It would have been nice, however, to have gotten something like:

TypeError - This routine needs a tuple.

instead of the silent in line calling of the routine in question,
while failing actually to start a new thread.

Hendrik van Rooyen wrote:

"Dan" <the,,,ail.comwrote:

>keyboard_thread = thread.start_new_thread(kbd_driver (port_q,kbd_q))

Needs to be
keyboard_thread = thread.start_new_thread(kbd_driver, (port_q,kbd_q))

Commas are important!

-Dan

Absolutely! - well spotted!

As the first correct respondent, you win the freedom to spend a week in
Naboomspruit at your own expense.

It would have been nice, however, to have gotten something like:

TypeError - This routine needs a tuple.

instead of the silent in line calling of the routine in question,
while failing actually to start a new thread.

You can't prevent the silent inline-calling - otherwise, how would you do
this:

def compute_thread_target():
def target():
pass
return target

thread.start_new_thread(compute_thread_target())

Of course start_new_thread could throw an error if it got nothing callable
as first argument. No idea why it doesn't.

Diez

On Jan 16, 11:06 am, "Diez B. Roggisch" <de...@nospam.web.dewrote:

>Hendrik van Rooyen wrote:

>>"Dan" <the,,,ail.comwrote:
>>keyboard_thread = thread.start_new_thread(kbd_driver (port_q,kbd_q))
Needs to be
>>keyboard_thread = thread.start_new_thread(kbd_driver, (port_q,kbd_q))
Commas are important!
-Dan
Absolutely! - well spotted!
As the first correct respondent, you win the freedom to spend a week in
Naboomspruit at your own expense.
It would have been nice, however, to have gotten something like:
TypeError - This routine needs a tuple.
instead of the silent in line calling of the routine in question,
while failing actually to start a new thread.

You can't prevent the silent inline-calling - otherwise, how would you do
this:

def compute_thread_target():
def target():
pass
return target

thread.start_new_thread(compute_thread_target() )

Of course start_new_thread could throw an error if it got nothing callable
as first argument. No idea why it doesn't.

Diez

Of course, in his case, having start_new_thread throw an error
wouldn't have helped, since he went into an infinite loop while
evaluating the parameters for start_new_thread.

Would it be possible to have pychecker (or some such) warn that there
is an insufficient parameter count to start_new_thread? I guess that
would require knowing the type of thread. . .

Dan schrieb:

On Jan 16, 11:06 am, "Diez B. Roggisch" <de...@nospam.web.dewrote:

Hendrik van Rooyen wrote:
"Dan" <the,,,ail.comwrote:
>keyboard_thread = thread.start_new_thread(kbd_driver (port_q,kbd_q))
Needs to be
>keyboard_thread = thread.start_new_thread(kbd_driver, (port_q,kbd_q))
Commas are important!
-Dan
Absolutely! - well spotted!
As the first correct respondent, you win the freedom to spend a week in
Naboomspruit at your own expense.
It would have been nice, however, to have gotten something like:
TypeError - This routine needs a tuple.
instead of the silent in line calling of the routine in question,
while failing actually to start a new thread.
You can't prevent the silent inline-calling - otherwise, how would you do
this:

def compute_thread_target():
def target():
pass
return target

thread.start_new_thread(compute_thread_target())

Of course start_new_thread could throw an error if it got nothing callable
as first argument. No idea why it doesn't.

Diez

Of course, in his case, having start_new_thread throw an error
wouldn't have helped, since he went into an infinite loop while
evaluating the parameters for start_new_thread.

Would it be possible to have pychecker (or some such) warn that there
is an insufficient parameter count to start_new_thread? I guess that
would require knowing the type of thread. . .

What has this to do with the second argument? It's perfectly legal to
have a function as thread-target that takes no arguments at all, so
enforcing a second argument wouldn't be helpful - all it would do is to
force all developers that don't need an argument tuple to pass the empty
tuple. So there was no insufficient argument count.

And none of these would solve the underlying problem that in python
expressions are evaluated eagerly. Changing that would mean that you end
up with a totally new language.

the only thing that could help to a certain extend would be static
types. Which we don't want here :)

Diez

>>import thread
def bat():

>>thread.start_new_thread(bat)

>>thread.start_new_thread(bat, ())

>>>

On Jan 16, 1:33 pm, "Diez B. Roggisch" <de...@nospam.web.dewrote:

>Dan schrieb:

>>On Jan 16, 11:06 am, "Diez B. Roggisch" <de...@nospam.web.dewrote:
Hendrik van Rooyen wrote:
"Dan" <the,,,ail.comwrote:
>>>>keyboard_thread = thread.start_new_thread(kbd_driver (port_q,kbd_q))
>Needs to be
>>>>keyboard_thread = thread.start_new_thread(kbd_driver, (port_q,kbd_q))
>Commas are important!
>-Dan
Absolutely! - well spotted!
As the first correct respondent, you win the freedom to spend a week in
Naboomspruit at your own expense.
It would have been nice, however, to have gotten something like:
TypeError - This routine needs a tuple.
instead of the silent in line calling of the routine in question,
while failing actually to start a new thread.
You can't prevent the silent inline-calling - otherwise, how would you do
this:
def compute_thread_target():
def target():
pass
return target
thread.start_new_thread(compute_thread_target() )
Of course start_new_thread could throw an error if it got nothing callable
as first argument. No idea why it doesn't.
Diez
Of course, in his case, having start_new_thread throw an error
wouldn't have helped, since he went into an infinite loop while
evaluating the parameters for start_new_thread.
Would it be possible to have pychecker (or some such) warn that there
is an insufficient parameter count to start_new_thread? I guess that
would require knowing the type of thread. . .

What has this to do with the second argument? It's perfectly legal to
have a function as thread-target that takes no arguments at all, so
enforcing a second argument wouldn't be helpful - all it would do is to
force all developers that don't need an argument tuple to pass the empty
tuple. So there was no insufficient argument count.

And none of these would solve the underlying problem that in python
expressions are evaluated eagerly. Changing that would mean that you end
up with a totally new language.

the only thing that could help to a certain extend would be static
types. Which we don't want here :)

Diez

It doesn't seem to be legal in my version of python (or the doc):

>>>import thread
def bat():

print "hello"

>>>thread.start_new_thread(bat)

Traceback (most recent call last):
File "<pyshell#12>", line 1, in <module>
thread.start_new_thread(bat)
TypeError: start_new_thread expected at least 2 arguments, got 1

>>>thread.start_new_thread(bat, ())

2256hello

>Of course start_new_thread could throw an error if it got nothing callable
as first argument. No idea why it doesn't.

>>thread.start_new_thread(None, None)

Absolutely! - well spotted!

It would have been nice, however, to have gotten something like:

TypeError - This routine needs a tuple.

instead of the silent in line calling of the routine in question,
while failing actually to start a new thread.

"Bjoern Schliessmann" <usenet-ourmet.comwrote:

Hendrik van Rooyen wrote:

>Absolutely! - well spotted!

This is no threading problem at all; not even a syntax problem. If
you don't know exactly what start_new_thread and kbd_driver
functions do it's impossible to tell if your code does what is
intended.

>It would have been nice, however, to have gotten something like:

TypeError - This routine needs a tuple.

instead of the silent in line calling of the routine in question,
while failing actually to start a new thread.

Exactly which part of the code should give you this warning?

I am obviously missing something.

My understanding is that, to start a new thread, one does:

NewThreadID = thread.start_new_thread(NameOfRoutineToStart,
(ArgumentToCall_it_with,secondArg,Etc))

This calls start_new_thread with the name and the arguments to pass.

If one omits the comma, then start_new_thread is surely stilled called,
but with an argument that is now a call to the routine in question, which
somehow causes the problem.

So start_new_thread is the code that that is executed, with a bad set of
arguments - one thing, (a call to a routine) instead of two things -
a routine and a tuple of arguments.

Everywhere else in Python if you give a routine the incorrect number of
arguments, you get an exception. Why not here?

>>def f(x): print x

>>f(f(1), f(2), f(3))

>>def g(x):

>>f(f(1), g(2), f(3))

Given that the start_new_thread function never actually got called, what
code exactly do you expect to complain about the absence of a tuple?

Hendrik van Rooyen wrote:

It would have been nice, however, to have gotten something like:

TypeError - This routine needs a tuple.

instead of the silent in line calling of the routine in question,
while failing actually to start a new thread.

You can't prevent the silent inline-calling - otherwise, how would you do
this:

def compute_thread_target():
def target():
pass
return target

thread.start_new_thread(compute_thread_target())
Of course start_new_thread could throw an error if it got nothing callable
as first argument. No idea why it doesn't.

>>a = 42
b = 24
def do_something(c,d):

>>do_something(a,b)

>>def harmless():

>>def evil():

>>do_something(a)

>>do_something(harmless())

>>>do_something(evil())

>>do_something(a,b,harmless())

>>>do_something(a,b,evil())

"Duncan Booth" <dunc...d.invalidwrote:

>Given that the start_new_thread function never actually got called, what
code exactly do you expect to complain about the absence of a tuple?

I don't understand this assertion.

I thought that start_new_thread was called with a missing comma in
its argument list, which had the effect that I am complaining about.

Putting the comma in place solved the problem, without any other
changes, so why do you say that start_new_thread was not called?

Would it be possible to have pychecker (or some such) warn that there
is an insufficient parameter count to start_new_thread? I guess that
would require knowing the type of thread. . .