sending pickled dictionary over a socket

[Ivan Dewolf]

I saw this thread:
http://bytes.com/topic/python/answers/553952-sending-dictionary-via-network

which works nicely for tiny dictionaries, but my sample pickled dictionary is about 500,000 characters long, and this method doesn't work. I tried splitting the dictionary on \n and then joining it, but the sockets hang after a coupla thousand lines.

Surely, passing a dictionary over a socket must be a fairly common request? how is it done?

[Ivan Dewolf]

also, security is not an issue, I am aware of the security issues.

Here's some code fragments:

pkl_dict = cPickle.dumps(heir_dict)
pkl_split = pkl_dict.split("\n")
print len(pkl_split)
i = 0
for line in pkl_split:
    i += 1
    print str(i)+" "+line
    mysocket.send(line)
 

my sample pickle returns len(pkl_split) = 45499
and hangs when i = 2707

[Ivan Dewolf]

hokay, solved it. Brak it into chunks and send 'em.

The server launches the client .py

server snippet:

mysocket = socket(AF_INET,SOCK_DGRAM)
mysocket.bind(addr)
#launch client.py
clientthread().start()
pkl_str = ""
Dlen = mysocket.recv(buf)
for i in range(0,int(int(Dlen)/buf)+1):
    data = mysocket.recv(buf)
    pkl_str +=data
dict = pickle.loads(pkl_str)
 

client snippet:

### Create socket
mysocket = socket(AF_INET,SOCK_DGRAM)
 
### Send pickle length
mysocket.sendto(str(len( pkl_dict)),addr)
i = 0
j = buf
while(i<len(pkl_dict)):
    if j>(len(pkl_dict)-1):
        j=(len(pkl_dict))
    ###send pickle chunks
    mysocket.sendto(pkl_dict[i:j],addr)
    i += buf
    j += buf
 

[Oralloy]

Ivan,

Good on you!

Is there a reason you're using UDP, and not IP?

Cheers!

[Ivan Dewolf]

um.... out of ignorance?
to be honest, I'm a bit fuzzy as to how to choose between the two.

[Oralloy]

Ivan,

I know that unknowing feeling.

UDP Datagrams are non-guaranteed delivery. They're great for sending very short messages when you don't know if you'll get a response.

IP Datastreams are called "reliable", as they have retransmit and sequencing built into their delivery.

In general I suggest using IP (TCP/IP) for "conversations" between computers, like FTP or Telnet.

Datagrams are good for quick messages where losing information is Ok. One common use of datagrams is protocols like DHCP. The client system (e.g. your desktop) sends a broadcast datagram asking for a valid address or identifier token. One or more servers may reply, even though the client only needs one.

Does that help?