By using this site, you agree to our updated Privacy Policy and our Terms of Use. Manage your Cookies Settings.
444,199 Members | 1,077 Online
Bytes IT Community
+ Ask a Question
Need help? Post your question and get tips & solutions from a community of 444,199 IT Pros & Developers. It's quick & easy.

Socket Performance

P: n/a
Can anyone explain why socket performance (throughput) varies
depending on the amount of data send and recv are called with?

For example: try creating a local client/server (running on the same
computer) where the server sends the client a fixed amount of data.
Using method A, recv(8192) and sendall( ) with 8192 bytes worth of
data. Do this 100 times. Using method B, recv(1) and sendall( ) with 1
byte worth of data. Do this 819200 times.

If you time both methods, method A has much greater throughput than
method B.

Server:

import socket
import random
import string
import time

HOST = 'localhost'
PORT = 50023
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.bind((HOST, PORT))
s.listen(1)
conn, addr = s.accept()
print 'Connected by', addr

string_1 = 'A'
string_8192 = ''.join([random.choice(string.letters + string.digits)
for i in range(8192)])

conn.sendall('Start')

start = time.clock()
total_data = 0
for i in range(0,100):
conn.sendall(string_8192)
total_data += len(string_8192)
print 'Send Speed (Long String): ' + str( total_data / (time.clock() -
start) / 1024 / 1024 ) + ' MB/sec\n\n'

start = time.clock()
total_data = 0
for i in range(0,819200):
conn.sendall(string_1)
total_data += len(string_1)
print 'Send Speed (Short String): ' + str( total_data / (time.clock()
- start) / 1024 / 1024 ) + ' MB/sec'

conn.close()
Client:

import socket
import time

HOST = 'localhost' # The remote host
PORT = 50023 # The same port as used by the server
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.bind((HOST, 50026))
s.connect((HOST, PORT))

data = s.recv(5)
print 'From Server: ' + data

start = time.clock()
total_data = 0
while total_data < 819200:
data = s.recv(8192)
total_data += len(data)
print 'Receive Speed (Long String): ' + str( total_data /
(time.clock() - start) / 1024 / 1024 ) + ' MB/sec\n\n'

start = time.clock()
total_data = 0
while total_data < 819200:
data = s.recv(1)
total_data += len(data)
print 'Receive Speed (Short String): ' + str( total_data /
(time.clock() - start) / 1024 / 1024 ) + ' MB/sec'
s.close()
Mar 13 '08 #1
Share this Question
Share on Google+
5 Replies


P: n/a
sl****@gmail.com wrote:
Sent: Wednesday, March 12, 2008 9:47 PM
To: py*********@python.org
Subject: Socket Performance

Can anyone explain why socket performance (throughput) varies
depending on the amount of data send and recv are called with?

For example: try creating a local client/server (running on the same
computer) where the server sends the client a fixed amount of data.
Using method A, recv(8192) and sendall( ) with 8192 bytes
worth of data. Do this 100 times. Using method B, recv(1) and
sendall( ) with 1 byte worth of data. Do this 819200 times.

If you time both methods, method A has much greater
throughput than method B.
Why is it faster to drink a liter of water a cupful at a time than to
drink it out of an eyedropper?

- Brian

Mar 13 '08 #2

P: n/a
ca********@gmail.com wrote:
Gabriel Genellina wrote:
>No need to reinvent the wheel. socket objects already have a makefile
method returning a file-like object, which behaves like a buffered socket.
That wheel is far from round, and needs some reinvention. Python's
file-like objects do not play nice with lower level calls, which
would be tolerable if they supported some well-defiend high-level
asynchronous I/O, but they do not.
Newbie question: Can you write to the 'file-like object' a pickle,
and receive it intact-- as one string with nothing else?
Yes, but there's a world of gotcha's. Sockets do not recognize
record boundaries, and Python's 'pickle' has holes one's enemies
could drive a truck through. Still, you can pickle, write, read,
un-pickle, and get back your data intact.
I want to know because I want to send two pickles.
"Two pickles" sounds like a tasty snack, but also suggests you may
be playing hopscotch in a minefield. This is a helpful group. Give
us more to go on, and you are likely to receive thousands of
dollars worth of consulting for free.
--
--Bryan
Mar 15 '08 #3

P: n/a
On Mar 15, 8:18*am, Bryan Olson <fakeaddr...@nowhere.orgwrote:
castiro...@gmail.com wrote:
Gabriel Genellina wrote:
No need to reinvent the wheel. socket objects already have a makefile *
method returning a file-like object, which behaves like a buffered socket.

That wheel is far from round, and needs some reinvention. Python's
file-like objects do not play nice with lower level calls, which
would be tolerable if they supported some well-defiend high-level
asynchronous I/O, but they do not.
Newbie question: *Can you write to the 'file-like object' a pickle,
and receive it intact-- as one string with nothing else?

Yes, but there's a world of gotcha's. Sockets do not recognize
record boundaries, and Python's 'pickle' has holes one's enemies
could drive a truck through. Still, you can pickle, write, read,
un-pickle, and get back your data intact.
I want to know because I want to send two pickles.

"Two pickles" sounds like a tasty snack, but also suggests you may
be playing hopscotch in a minefield. This is a helpful group. Give
us more to go on, and you are likely to receive thousands of
dollars worth of consulting for free.
It depends on the situation. How generally applicable is this:

fun ListenerGeneric( port, factory, arrivedfun ):

which calls 'factory' on socketA.accept (and loops again), then
arrivedfun( stringA ) on message complete detection. ?. It should
start itself in a separate thread.

Or is this any better:

for x in connections():
startnewthreadwith x:
for y in messages( x ):
arrivedfun( y )
Mar 15 '08 #4

P: n/a
En Sat, 15 Mar 2008 20:08:05 -0200, <ca********@gmail.comescribi´┐Ż:
On Mar 15, 8:18┬*am, Bryan Olson <fakeaddr...@nowhere.orgwrote:
>castiro...@gmail.com wrote:
Newbie question: ┬*Can you write to the 'file-like object' a pickle,
and receive it intact-- as one string with nothing else?

Yes, but there's a world of gotcha's. Sockets do not recognize
record boundaries, and Python's 'pickle' has holes one's enemies
could drive a truck through. Still, you can pickle, write, read,
un-pickle, and get back your data intact.
I want to know because I want to send two pickles.

"Two pickles" sounds like a tasty snack, but also suggests you may
be playing hopscotch in a minefield. This is a helpful group. Give
us more to go on, and you are likely to receive thousands of
dollars worth of consulting for free.

It depends on the situation. How generally applicable is this:

fun ListenerGeneric( port, factory, arrivedfun ):

which calls 'factory' on socketA.accept (and loops again), then
arrivedfun( stringA ) on message complete detection. ?. It should
start itself in a separate thread.

Or is this any better:

for x in connections():
startnewthreadwith x:
for y in messages( x ):
arrivedfun( y )
This looks like a SocketServer + ThreadingMixIn + a RequestHandler (your
factory).
But as B. Olson already pointed, pickles are unsafe. Worse, it's not that
someone could send a specially crafted pickle that could execute some
arbitrary code: you're blindy executing whatever you receive!
xmlrpc may be a good alternative in some cases.

--
Gabriel Genellina

Mar 16 '08 #5

P: n/a
On Mar 16, 1:29┬*pm, "Gabriel Genellina" <gagsl-...@yahoo.com.ar>
wrote:
En Sat, 15 Mar 2008 20:08:05 -0200, <castiro...@gmail.comescribi´┐Ż:


On Mar 15, 8:18┬*am, Bryan Olson <fakeaddr...@nowhere.orgwrote:
castiro...@gmail.com wrote:
Newbie question: ┬*Can you write to the 'file-like object' a pickle,
and receive it intact-- as one string with nothing else?
Yes, but there's a world of gotcha's. Sockets do not recognize
record boundaries, and Python's 'pickle' has holes one's enemies
could drive a truck through. Still, you can pickle, write, read,
un-pickle, and get back your data intact.
I want to know because I want to send two pickles.
"Two pickles" sounds like a tasty snack, but also suggests you may
be playing hopscotch in a minefield. This is a helpful group. Give
us more to go on, and you are likely to receive thousands of
dollars worth of consulting for free.
It depends on the situation. ┬*How generally applicable is this:
fun ListenerGeneric( port, factory, arrivedfun ):
which calls 'factory' on socketA.accept (and loops again), then
arrivedfun( stringA ) on message complete detection. ┬*?. ┬*It should
start itself in a separate thread.
Or is this any better:
for x in connections():
┬* ┬*startnewthreadwith x:
┬* ┬* ┬* for y in messages( x ):
┬* ┬* ┬* ┬* ┬*arrivedfun( y )

This looks like a SocketServer + ThreadingMixIn + a RequestHandler (your ┬*
factory).
But as B. Olson already pointed, pickles are unsafe. Worse, it's not that ┬*
someone could send a specially crafted pickle that could execute some ┬*
arbitrary code: you're blindy executing whatever you receive!
xmlrpc may be a good alternative in some cases.
I see. Just transmit a binary and execute that. Make sure to have
handles to your program publicly accessible too. I execute this.
Mar 17 '08 #6

This discussion thread is closed

Replies have been disabled for this discussion.