Bug? asyncore.dispatcher_with_send trouble

[Freddie]

Hi,

I've been mangling python-irclib into an asyncore class, so it fits in
nicely with the rest of my app. I ran into a problem with
asyncore.dispatcher_with_send (Python 2.3.4), though. Not sure if this
is the right place to file a bug, but here goes:
class dispatcher_with_send(dispatcher):

def __init__(self, sock=None):
dispatcher.__init__(self, sock)
self.out_buffer = ''

def initiate_send(self):
num_sent = 0
num_sent = dispatcher.send(self, self.out_buffer[:512])
self.out_buffer = self.out_buffer[num_sent:]

def handle_write(self):
self.initiate_send()

def writable(self):
return (not self.connected) or len(self.out_buffer)

def send(self, data):
if self.debug:
self.log_info('sending %s' % repr(data))
self.out_buffer = self.out_buffer + data
self.initiate_send()
I assumed that disp.send('chunkofdata') would merely put it in the out
buffer until the socket is writable, but this isn't what happens. It
tries to send() the data straight away, which can raise an exception...
and since we're not inside the asyncore.write() function, handle_error
is never called.

Here's the fixed version that I've cobbled together:
class buffered_dispatcher(asyncore.dispatcher):
def __init__(self, sock=None):
asyncore.dispatcher.__init__(self, sock)
self.out_buffer = ''

# We only want to be writable if we're connecting, or something is
in our
# buffer.
def writable(self):
return (not self.connected) or len(self.out_buffer)

# Send some data from our buffer when we can write
def handle_write(self):
sent = asyncore.dispatcher.send(self, self.out_buffer)
self.out_buffer = self.out_buffer[sent:]

# We want buffered output, duh
def send(self, data):
self.out_buffer += data
Hopefully this saves someone else half an hour of annoyance at some point :)

Freddie

[Steve Holden]

Freddie wrote:

Hi,

I've been mangling python-irclib into an asyncore class, so it fits in
nicely with the rest of my app. I ran into a problem with
asyncore.dispatcher_with_send (Python 2.3.4), though. Not sure if this
is the right place to file a bug, but here goes:
No, it isn't. The right place to file a bug would be

http://sourceforge.net/tracker/?grou...70&atid=105470

class dispatcher_with_send(dispatcher):

def __init__(self, sock=None):
dispatcher.__init__(self, sock)
self.out_buffer = ''

def initiate_send(self):
num_sent = 0
num_sent = dispatcher.send(self, self.out_buffer[:512])
self.out_buffer = self.out_buffer[num_sent:]

def handle_write(self):
self.initiate_send()

def writable(self):
return (not self.connected) or len(self.out_buffer)

def send(self, data):
if self.debug:
self.log_info('sending %s' % repr(data))
self.out_buffer = self.out_buffer + data
self.initiate_send()
I assumed that disp.send('chunkofdata') would merely put it in the out
buffer until the socket is writable, but this isn't what happens. It
tries to send() the data straight away, which can raise an exception...
and since we're not inside the asyncore.write() function, handle_error
is never called.

Here's the fixed version that I've cobbled together:
class buffered_dispatcher(asyncore.dispatcher):
def __init__(self, sock=None):
asyncore.dispatcher.__init__(self, sock)
self.out_buffer = ''

# We only want to be writable if we're connecting, or something is
in our
# buffer.
def writable(self):
return (not self.connected) or len(self.out_buffer)

# Send some data from our buffer when we can write
def handle_write(self):
sent = asyncore.dispatcher.send(self, self.out_buffer)
self.out_buffer = self.out_buffer[sent:]

# We want buffered output, duh
def send(self, data):
self.out_buffer += data
Hopefully this saves someone else half an hour of annoyance at some
point :)

Freddie

I haven't actually studied your code in detail, but I have trouble
understanding how something so apparently reliable as asyncore is
suddenly revealed to have fatal flaws in it. Sam Rushing is a pretty
experienced programmer.

Under what circumstances does calling .send(data) result in exceptions
being raised?

regards
Steve

[Freddie]

Steve Holden wrote:

Freddie wrote:

Hi,

I've been mangling python-irclib into an asyncore class, so it fits in
nicely with the rest of my app. I ran into a problem with
asyncore.dispatcher_with_send (Python 2.3.4), though. Not sure if this
is the right place to file a bug, but here goes:
No, it isn't. The right place to file a bug would be

http://sourceforge.net/tracker/?grou...70&atid=105470

Ah. Thanks.

<snip>

I haven't actually studied your code in detail, but I have trouble
understanding how something so apparently reliable as asyncore is
suddenly revealed to have fatal flaws in it. Sam Rushing is a pretty
experienced programmer.

Under what circumstances does calling .send(data) result in exceptions
being raised?

regards
Steve

I'm actually being slightly crazy and using a single dispatcher for the
entire 'life' of that IRC connection. It can be connected/disconnected
multiple times over it's life. Calling send() can sometimes happen when
the socket is closed/broken/whatever, so calling initiate_send()
straight away in there can and will blow up... and it's outside the
normal handle_error() exception trap (read/write methods in
asyncore.py). This is not how I thought a 'buffered output' dispatcher
would work. Having a 'buffered' send() that can actually block, or even
raise an exception that you have to catch yourself (instead of
handle_error being called like every other exception) seems broken to me :|

Freddie