473,326 Members | 2,012 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,326 software developers and data experts.

Limitate speed of a socket-based data transferring

Hi all. I'm writing a TCP-based application that I will use to trasfer
binary files through the network. This piece of code represents how do
I get a file from a remote peer and save it on my local hard drive:

file_obj = open('downloaded.ext', 'wb')
while 1:
buf = sock.recv(2048)
if len(buf) == 0:
break
file_obj.write(buf)
file_obj.close()
sock.close()

I would like to know how could be possible to limit the file transfer
speed (for example: don't write more than 50 Kb/sec).
Some ideas?

Best regards

Sep 14 '06 #1
6 8993
billie schrieb:
Hi all. I'm writing a TCP-based application that I will use to trasfer
binary files through the network. This piece of code represents how do
I get a file from a remote peer and save it on my local hard drive:

file_obj = open('downloaded.ext', 'wb')
while 1:
buf = sock.recv(2048)
if len(buf) == 0:
break
file_obj.write(buf)
file_obj.close()
sock.close()

I would like to know how could be possible to limit the file transfer
speed (for example: don't write more than 50 Kb/sec).
Some ideas?
If you are on unix, use trickle. If you must limit it from within your
own code, I can only assume that computing the transfer rate so far and
introducing timeouts might help - but I never did such a thing, nor do I
know what that means for example for the network stack.

But maybe even then trickle may help you to get an idea, as it is a
user-space program AFAIK. So they might have some information (at least
the code... ) out there that could be of use.
Diez
Sep 14 '06 #2
On 14/09/06, Diez B. Roggisch <de***@nospam.web.dewrote:
billie schrieb:
Hi all. I'm writing a TCP-based application that I will use to trasfer
binary files through the network. This piece of code represents how do
I get a file from a remote peer and save it on my local hard drive:

file_obj = open('downloaded.ext', 'wb')
while 1:
buf = sock.recv(2048)
if len(buf) == 0:
break
file_obj.write(buf)
file_obj.close()
sock.close()

I would like to know how could be possible to limit the file transfer
speed (for example: don't write more than 50 Kb/sec).
Some ideas?

If you are on unix, use trickle. If you must limit it from within your
own code, I can only assume that computing the transfer rate so far and
introducing timeouts might help - but I never did such a thing, nor do I
know what that means for example for the network stack.

But maybe even then trickle may help you to get an idea, as it is a
user-space program AFAIK. So they might have some information (at least
the code... ) out there that could be of use.
You could wrap buf = sock.recv(xxx) in a data counter and sleep loop
so that you burst to no more than 50KB/s average.

Completely untestest and for illustration only :)

file_obj = open('downloaded.ext', 'wb')
interval = 1.0 # seconds eg. 0.5 or 2.0
# smaller the interval, the less bursty and smoother the throughput
max_speed = 51200 # 50k * 1024 = bytes
data_count = 0 # keep track of the amount of data transferred
time_next = time.time() + interval
while 1:
buf = sock.recv(512) # smaller chunks = smoother, more accurate
if len(buf) == 0:
break
data_count += len(buf)
if data_count >= max_speed * interval:
data_count = 0
sleep_for = time_next - time.time()
if sleep_for 0:
time.sleep(sleep_for)
time_next = time.time() + interval
file_obj.write(buf)
file_obj.close()
sock.close()
Sep 14 '06 #3
On 2006-09-15, Dennis Lee Bieber <wl*****@ix.netcom.comwrote:
On 14 Sep 2006 04:54:48 -0700, "billie" <gn****@gmail.comdeclaimed the
following in comp.lang.python:
>Hi all. I'm writing a TCP-based application that I will use to trasfer
binary files through the network. This piece of code represents how do
I get a file from a remote peer and save it on my local hard drive:
<code snipped>
>>
I would like to know how could be possible to limit the file transfer
speed (for example: don't write more than 50 Kb/sec).
Some ideas?
I'm not sure you /can/ limit the /receive/ speed.
Sure you can. Just read data from the socket at the max speed
you want to receive. The receive buffer for that socket will
fill up and the TCP window will start close up and throttle the
sender.
The sender will send at whatever rate they are capable of, so
packets may just become backlogged on your receiving socket
When that happens, the sending end of the socket will throttle
down to match the rate at which data is being read from the
socket.
waiting for you to read them if you add some sort of delay to
your reading loop.
--
Grant Edwards grante Yow! .. the HIGHWAY is
at made out of LIME JELLO and
visi.com my HONDA is a barbequed
OYSTER! Yum!
Sep 15 '06 #4
Grant Edwards wrote:
On 2006-09-15, Dennis Lee Bieber <wl*****@ix.netcom.comwrote:
>>On 14 Sep 2006 04:54:48 -0700, "billie" <gn****@gmail.comdeclaimed the
following in comp.lang.python:

>>>Hi all. I'm writing a TCP-based application that I will use to trasfer
binary files through the network. This piece of code represents how do
I get a file from a remote peer and save it on my local hard drive:

<code snipped>
>>>I would like to know how could be possible to limit the file transfer
speed (for example: don't write more than 50 Kb/sec).
Some ideas?

I'm not sure you /can/ limit the /receive/ speed.


Sure you can. Just read data from the socket at the max speed
you want to receive. The receive buffer for that socket will
fill up and the TCP window will start close up and throttle the
sender.

>>The sender will send at whatever rate they are capable of, so
packets may just become backlogged on your receiving socket


When that happens, the sending end of the socket will throttle
down to match the rate at which data is being read from the
socket.

>>waiting for you to read them if you add some sort of delay to
your reading loop.

Of course this depends crucially on the window size. Since the addition
of the window scaling TCP option it's been possible to specify very
large windows, which are useful over high-bandwidth high-delay links.

The remote (send) throttling will only start to cut in when the window
is full (since the whole point of the sliding window mechanism is to
allow continuous transmission in the face of acknowledgment delay).

regards
Steve
--
Steve Holden +44 150 684 7255 +1 800 494 3119
Holden Web LLC/Ltd http://www.holdenweb.com
Skype: holdenweb http://holdenweb.blogspot.com
Recent Ramblings http://del.icio.us/steve.holden

Sep 15 '06 #5
On 2006-09-15, Steve Holden <st***@holdenweb.comwrote:
>>>The sender will send at whatever rate they are capable of, so
packets may just become backlogged on your receiving socket


When that happens, the sending end of the socket will throttle
down to match the rate at which data is being read from the
socket.

Of course this depends crucially on the window size. Since the
addition of the window scaling TCP option it's been possible
to specify very large windows, which are useful over
high-bandwidth high-delay links.
True. If the window size is large compared to the amount of
data being transferred, then the throttling won't happen.
The remote (send) throttling will only start to cut in when
the window is full (since the whole point of the sliding
window mechanism is to allow continuous transmission in the
face of acknowledgment delay).
Yup.

--
Grant Edwards grante Yow! -- I have seen the
at FUN --
visi.com
Sep 15 '06 #6
On 15/09/06, Grant Edwards <gr****@visi.comwrote:
On 2006-09-15, Steve Holden <st***@holdenweb.comwrote:

Of course this depends crucially on the window size. Since the
addition of the window scaling TCP option it's been possible
to specify very large windows, which are useful over
high-bandwidth high-delay links.

True. If the window size is large compared to the amount of
data being transferred, then the throttling won't happen.
The remote (send) throttling will only start to cut in when
the window is full (since the whole point of the sliding
window mechanism is to allow continuous transmission in the
face of acknowledgment delay).

Yup.
If the OP is also writing the client, the throttling can be at the
client side to control the speed of writing to the socket. But if
multiple clients connect to the server concurrently then this may not
help either!

:)
Sep 17 '06 #7

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

1
by: Tor Erik Sønvisen | last post by:
Hi For an online game I'm developing I need some advice concerning tcp-sockets, and especially which socket options to set and not. What I want is a connection where nothing is buffered (but are...
2
by: chrisben | last post by:
Hi All Has anyone done any experiment on comparison between C++ and C# on socket rec/send speed Conditions in Windows system, same CPU for senders or receivers. Can you see how much faster a...
1
by: Manuel Costa | last post by:
Hi, i was testing a network application that i've been working on which use .net socket components. To read from a socket i use the networkstream beginread. If the other side dies, beginread...
1
by: aplaton101 | last post by:
What is the best way to create a TCP Server which receives connections from several sockets and stores information that will be used for the next connection from each of the sockets. How can I...
4
by: request | last post by:
Hi I am trying server request/response program using socket class. here i couldn't send headers with the request to the proxy server.when i send the request i should able to get the response from...
5
by: Justin Creasy | last post by:
If this is the wrong group for this posting please let me know and I'll move it. I have an application that has an ArrayList of sockets to clients. For standard one-to-one messages my...
2
by: WTH | last post by:
with a C# client (and/or server, but server not important)? I've got a scalable high speed (uses completion ports) C++ TCP/IP communication server but I'd like to write a C# client that other C#...
14
by: DaTurk | last post by:
I am makeing a Multicast server client setup and was wondering what the difference is between Socket.Connect, and Socket.Bind. It may be a stupid question, but I was just curious. Because I...
1
by: bruno.wouters | last post by:
Hi, I have a a
2
by: Thorben Krueger | last post by:
Do you see this too? Mor information and testcase here: http://bugs.python.org/issue3766 I would also be interested in the profiler output under windows. All the best Thorben
0
by: DolphinDB | last post by:
Tired of spending countless mintues downsampling your data? Look no further! In this article, you’ll learn how to efficiently downsample 6.48 billion high-frequency records to 61 million...
0
by: ryjfgjl | last post by:
ExcelToDatabase: batch import excel into database automatically...
0
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 6 Mar 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:15 (7.15PM). In this month's session, we are pleased to welcome back...
0
by: jfyes | last post by:
As a hardware engineer, after seeing that CEIWEI recently released a new tool for Modbus RTU Over TCP/UDP filtering and monitoring, I actively went to its official website to take a look. It turned...
0
by: ArrayDB | last post by:
The error message I've encountered is; ERROR:root:Error generating model response: exception: access violation writing 0x0000000000005140, which seems to be indicative of an access violation...
1
by: PapaRatzi | last post by:
Hello, I am teaching myself MS Access forms design and Visual Basic. I've created a table to capture a list of Top 30 singles and forms to capture new entries. The final step is a form (unbound)...
1
by: Defcon1945 | last post by:
I'm trying to learn Python using Pycharm but import shutil doesn't work
0
by: Faith0G | last post by:
I am starting a new it consulting business and it's been a while since I setup a new website. Is wordpress still the best web based software for hosting a 5 page website? The webpages will be...
0
isladogs
by: isladogs | last post by:
The next Access Europe User Group meeting will be on Wednesday 3 Apr 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM). In this session, we are pleased to welcome former...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.