socket problem

Gordon Wetzstein

Hello everyone,

I have a problem with python sockets.
I broadcast a lot of pickled objects with socket. sendto(...), that works.
Ireceive them on the other site with socket.recvfrom (16384)
The pickled objects are smaller (like 10000 bytes) than the max bytes.

The problem appears if I send too many pickled objects very quickly one
after another,
then I only receive a part of the sent objects.
I print them before I send them, they are there but they never reach the
destination.
When I do a time.sleep(0.1) (but not lesser than 0.1) after socket.send()
all is good but it's too slow!

Does anyone know what the problem is? And maybe a solution!

thanks gordon

--
+++ GMX - Mail, Messaging & more http://www.gmx.net +++

Jetzt ein- oder umsteigen und USB-Speicheruhr als Prämie sichern!

Jul 18 '05 #1

Subscribe Reply

3799

Alan Kennedy

Gordon Wetzstein wrote:

I have a problem with python sockets.
I broadcast a lot of pickled objects with socket. sendto(...), that
works.
Ireceive them on the other site with socket.recvfrom (16384)
The pickled objects are smaller (like 10000 bytes) than the max bytes.

The problem appears if I send too many pickled objects very quickly one
after another,
then I only receive a part of the sent objects.

As Jp Calderone pointed out, UDP is an unreliable protocol. Therefore,
it *will* drop packets occasionally. Worse, it doesn't guarantee
ordering of delivery of packets, meaning that your pickles could
arrive all jumbled up and unusable.

You have three main choices to get around this problem.

1. Use TCP, which guarantees delivery to the destination address once
and only once, guarantees that packets will be received in the order
in which they were sent. However, you can't broadcast with TCP: it's a
point to point protocol.

2. Implement your own guaranteed delivery protocol over UDP. This is
usually done by assigning packets a sequence number as they are
transmitted from the publisher. All consumers then keep track of the
sequence numbers they receive, and re-request any packets they missed.
Implementing your own such protocol can get quite tricky, depending on
how complex your requirements.

3. Use a pre-made, highly efficient python library which is custom
designed for the job: spread. Spread is designed to solve all of the
problems of reliable broadcasting over UDP, and gives you a wide range
of options for how to balance efficiency versus reliability. And more
importantly, it's industrial-strength robust, stable and platform
independent. Also, it's configurable to work with a wide range of
network topologies. More info from

http://www.python.org/other/spread/

If I were you, I'd seriously consider spending an hour getting up and
running with Spread: could be the most productive hour you'll spend in
this area.

HTH,

--
alan kennedy
-----------------------------------------------------
check http headers here: http://xhaus.com/headers
email alan: http://xhaus.com/mailto/alan

Jul 18 '05 #2

Gordon Wetzstein

I have a problem with python sockets.
I broadcast a lot of pickled objects with socket. sendto(...), that
works.
Ireceive them on the other site with socket.recvfrom (16384)
The pickled objects are smaller (like 10000 bytes) than the max bytes.

The problem appears if I send too many pickled objects very quickly one
after another,
then I only receive a part of the sent objects.

As Jp Calderone pointed out, UDP is an unreliable protocol. Therefore,
it *will* drop packets occasionally. Worse, it doesn't guarantee
ordering of delivery of packets, meaning that your pickles could
arrive all jumbled up and unusable.

You have three main choices to get around this problem.

1. Use TCP, which guarantees delivery to the destination address once
and only once, guarantees that packets will be received in the order
in which they were sent. However, you can't broadcast with TCP: it's a
point to point protocol.

2. Implement your own guaranteed delivery protocol over UDP. This is
usually done by assigning packets a sequence number as they are
transmitted from the publisher. All consumers then keep track of the
sequence numbers they receive, and re-request any packets they missed.
Implementing your own such protocol can get quite tricky, depending on
how complex your requirements.

3. Use a pre-made, highly efficient python library which is custom
designed for the job: spread. Spread is designed to solve all of the
problems of reliable broadcasting over UDP, and gives you a wide range
of options for how to balance efficiency versus reliability. And more
importantly, it's industrial-strength robust, stable and platform
independent. Also, it's configurable to work with a wide range of
network topologies. More info from

http://www.python.org/other/spread/

If I were you, I'd seriously consider spending an hour getting up and
running with Spread: could be the most productive hour you'll spend in
this area.

HTH,

--
alan kennedy

thanks, i will try it.
gordon

--
+++ GMX - Mail, Messaging & more http://www.gmx.net +++

Jetzt ein- oder umsteigen und USB-Speicheruhr als Prämie sichern!

Jul 18 '05 #3

Similar topics

2642

Problem with socket

by: Thomas Hervé | last post by:

My problem is not really python specific but as I do my implementation in python I hope someone here can help me. I have two programs that talk through a socket. Here is the code : <server> sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM) sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1) sock.settimeout(5.0)

Python

11733

Fixing socket.makefile()

by: Bryan Olson | last post by:

Here's the problem: Suppose we use: import socket f = some_socket.makefile() Then: f.read() is efficient, but verbose, and incorrect (or at least does not play will with others);

Python

2256

Handling exceptions in Socket Callbacks

by: faktujaa | last post by:

Hi, I am having some problem with callback used in socket implementation. private static void Connect(string strPrtrIPAddr, int intPrtrPort, ref Socket rsocClient) { try { // Create remote end point. System.Net.IPAddress IPAddress = System.Net.IPAddress.Parse(strPrtrIPAddr); System.Net.IPEndPoint IPEndPoint = new...

.NET Framework

7070

question about multithread c# socket server

by: zbcong | last post by:

Hello: I write a multithread c# socket server,it is a winform application,there is a richtextbox control and button,when the button is click,the server begin to listen the socket port,waiting for a incoming connection,the relative code snipprt as following:: private IPAddress myIP=IPAddress.Parse("127.0.0.1"); private IPEndPoint myServer;...

C# / C Sharp

18092

Does Socket.Shutdown or Socket.Close Block and when?

by: Chris Tanger | last post by:

Context: C# System.Net.Sockets Socket created with constructor prarmeters Internetwork, Stream and TCP everything else is left at the default parameters and options except linger may be changed as I find appropriate. I am using the socket asynchronously by calling the BeingSend and BeginReceive calls. I would like to be able to call...

C# / C Sharp

2336

Socket communication problem

by: Sa¹o Zagoranski | last post by:

Hi! I'm writing a simple 3D First person shooter game. It is a multiplayer game, where all the players connect to one server.

C# / C Sharp

8584

Async socket & active connections

by: atlaste | last post by:

Hi, In an attempt to create a full-blown webcrawler I've found myself writing a wrapper around the Socket class in an attempt to make it completely async, supporting timeouts and some scheduling mechanisms. I use a non-blocking approach for this, using the call to 'poll' to support the async mechanism - rather than the 'begin' and 'end'...

C# / C Sharp

3658

socket.makefile() buggy?

by: ahlongxp | last post by:

socket.makefile() may lose data when "connection reset by peer". and socket.recv() will never lose the data. change the "1" to "0" in the client code to see the difference. confirmed on both windows and linux. so I guess there is a problem with makefile(). # Echo server program

Python

3555

VB/C# - Network: Problem with socket receiving data

by: george585 | last post by:

Hello! I am new to network programming, and understand just basics. Using some sample code, and having read documentation, I managed to create a simple app in C# and VB.NET. The application is supposed to do the following: monitor ALL INCOMING TCP traffic on the local computer, and save certain parts of it as files - not log files though, but...

.NET Framework

3834

SOCKET problem

by: =?iso-8859-1?q?|-|e|=5F|=5F_B0=DD?= | last post by:

hi all! I got a problem. I declared a SOCKET var in my C program but when i compiled the program it displayed like *--------------------------------------------------------------* *'SOCKET': illegal use of this type as an expression * * C:\..\..\..\include\winsock2.h: see declaration of 'SOCKET' *...

C / C++

7486

What is ONU?

by: marktang | last post by:

ONU (Optical Network Unit) is one of the key components for providing high-speed Internet services. Its primary function is to act as an endpoint device located at the user's premises. However, people are often confused as to whether an ONU can Work As a Router. In this blog post, we’ll explore What is ONU, What Is Router, ONU & Router’s main...

General

7442

The easy way to turn off automatic updates for Windows 10/11

by: Hystou | last post by:

Overview: Windows 11 and 10 have less user interface control over operating system update behaviour than previous versions of Windows. In Windows 11 and 10, there is no way to turn off the Windows Update option using the Control Panel or Settings app; it automatically checks for updates and installs any it finds, whether you like it or not. For...

Windows Server

7776

Discussion: How does Zigbee compare with other wireless protocols in smart home applications?

by: tracyyun | last post by:

Dear forum friends, With the development of smart home technology, a variety of wireless communication protocols have appeared on the market, such as Zigbee, Z-Wave, Wi-Fi, Bluetooth, etc. Each protocol has its own unique characteristics and advantages, but as a user who is planning to build a smart home system, I am a bit confused by the...

General

6001

AI Job Threat for Devs

by: agi2029 | last post by:

Let's talk about the concept of autonomous AI software engineers and no-code agents. These AIs are designed to manage the entire lifecycle of a software development project—planning, coding, testing, and deployment—without human intervention. Imagine an AI that can take a project description, break it down, write the code, debug it, and then...

Career Advice

4965

Couldn’t get equations in html when convert word .docx file to html file in C#.

by: conductexam | last post by:

I have .net C# application in which I am extracting data from word file and save it in database particularly. To store word all data as it is I am converting the whole word file firstly in HTML and then checking html paragraph one by one. At the time of converting from word file to html my equations which are in the word document file was convert...

C# / C Sharp

3473

Trying to create a lan-to-lan vpn between two differents networks

by: TSSRALBI | last post by:

Hello I'm a network technician in training and I need your help. I am currently learning how to create and manage the different types of VPNs and I have a question about LAN-to-LAN VPNs. The last exercise I practiced was to create a LAN-to-LAN VPN between two Pfsense firewalls, by using IPSEC protocols. I succeeded, with both firewalls in...

Networking - Hardware / Configuration

3456

Windows Forms - .Net 8.0

by: adsilva | last post by:

A Windows Forms form does not have the event Unload, like VB6. What one acts like?

Visual Basic .NET

1032

How to add payments to a PHP MySQL app.

by: muto222 | last post by:

How can i add a mobile payment intergratation into php mysql website.

PHP

729

Comprehensive Guide to Website Development in Toronto: Expert Insights from BSMN Consultancy

by: bsmnconsultancy | last post by:

In today's digital era, a well-designed website is crucial for businesses looking to succeed. Whether you're a small business owner or a large corporation in Toronto, having a strong online presence can significantly impact your brand's success. BSMN Consultancy, a leader in Website Development in Toronto offers valuable insights into creating...

General