469,909 Members | 1,617 Online
Bytes | Developer Community
New Post

Home Posts Topics Members FAQ

Post your question to a community of 469,909 developers. It's quick & easy.

socket.sendall(), non-blocking sockets, and multi-threaded socket sending

My application requires sending a large piece (~2MB) of data to
several devices on a network via TCP sockets. I have experimented with
different methods for doing this and this has raised some questions
about the implementation of Python sockets.

(both methods use blocking sockets)

Method 1: Calls socket.sendall(data) for each device in sequence, all
in a single thread.

Method 2: Each socket has its own thread that calls
socket.send(datachunk) iteratively until all data is sent.

So Method 1 sends all data to device1, then send all data to device2,
etc... until all data is sent to all devices. Method 2 sends chunks of
data to all devices in parallel. What I am seeing is that Method 1 is
faster than Method 2. Given that the bottleneck is the actual sending
of data on the sockets and not some latency in processing the message
at the other end, this result makes sense to me. Method 2 involves
sending the data in smaller chunks, whereas with Method 1 the
chunksize is selected by the TCP/IP stack implementation, which is
probably more efficient. Also, Method 2 entails context switching
between the send threads.

It turns out that using Method 1, all sendall() calls return
immediately. I am assuming at this point that my app is in the hands
of the TCP/IP stack implementation on my machine (win32-XP). Can
anyone explain what happens inside sendall() for Win32?

Also, I would like to get advice and opinions on what is the most
efficient way to broadcast data to several devices on a network.

Thanks,
T
Jul 18 '05 #1
1 6078
[Tim Black]
My application requires sending a large piece (~2MB) of data to
several devices on a network via TCP sockets. I have experimented with
different methods for doing this and this has raised some questions
about the implementation of Python sockets.

(both methods use blocking sockets)

Method 1: Calls socket.sendall(data) for each device in sequence, all
in a single thread.

Method 2: Each socket has its own thread that calls
socket.send(datachunk) iteratively until all data is sent.

So Method 1 sends all data to device1, then send all data to device2,
etc... until all data is sent to all devices. Method 2 sends chunks of
data to all devices in parallel. What I am seeing is that Method 1 is
faster than Method 2. Given that the bottleneck is the actual sending
of data on the sockets and not some latency in processing the message
at the other end, this result makes sense to me. Method 2 involves
sending the data in smaller chunks, whereas with Method 1 the
chunksize is selected by the TCP/IP stack implementation, which is
probably more efficient. Also, Method 2 entails context switching
between the send threads.

It turns out that using Method 1, all sendall() calls return
immediately. I am assuming at this point that my app is in the hands
of the TCP/IP stack implementation on my machine (win32-XP). Can
anyone explain what happens inside sendall() for Win32?

Also, I would like to get advice and opinions on what is the most
efficient way to broadcast data to several devices on a network.


I'd love to have enough time to get into discussing your findings
about sockets. But unfortunately, I don't :-(

But I did want to point out a module that you may not have come across
that makes the job of distributing information peer-to-peer over LANs
very easy: the spread module. I highly recommend that you take a look
over it: it could save you a lot of wheel-reinvention.

http://www.python.org/other/spread/

HTH,

--
alan kennedy
------------------------------------------------------
check http headers here: http://xhaus.com/headers
email alan: http://xhaus.com/contact/alan
Jul 18 '05 #2

This discussion thread is closed

Replies have been disabled for this discussion.

Similar topics

12 posts views Thread by lothar | last post: by
25 posts views Thread by Yves Glodt | last post: by
32 posts views Thread by Adrian Herscu | last post: by
22 posts views Thread by Steve - DND | last post: by
14 posts views Thread by Patrick Kowalzick | last post: by
399 posts views Thread by =?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?= | last post: by
1 post views Thread by Waqarahmed | last post: by
By using this site, you agree to our Privacy Policy and Terms of Use.