help on packet format for tcp/ip programming

[rattan]

I want a specific packet format for packet exchange between a client
server across the network. For example frame format
as a python class could be:
class Frame:
def __init__(self, buffer=None, count=0, offset=0):
self.buffer = buffer
self.count = count
self.offset = offset
the question is how to convert it to a byte stream so that format of
count and offset also becomes a sequence of bytes. I tried
the pickle module as:
a = Frame()
dat = pickle.dumps(a)
but the size of dat is quite large almost to 4 X the size of a, which
probably is an overkill for the numbers of bytes exchanged. Is there a
simpler solution (similar to C language -- set aside a buffer and fill
it with bytes and network byteorder values for count and offset and
send it out, there is no increase in byte count in the outgoing
packet)?

Any pointers will be appreciated.
-ishwar

[Bjoern Schliessmann]

ra****@cps.cmich.edu wrote:

I want a specific packet format for packet exchange between a
client server across the network. For example frame format
as a python class could be:
class Frame:
def __init__(self, buffer=None, count=0, offset=0):
self.buffer = buffer
self.count = count
self.offset = offset
the question is how to convert it to a byte stream so that format
of count and offset also becomes a sequence of bytes.

Try struct.

Regards,
Björn

--
BOFH excuse #208:

Your mail is being routed through Germany ... and they're censoring
us.

[rattan]

On Feb 8, 1:43 am, Bjoern Schliessmann <usenet-
[e-mail]:

[e-mail] wrote:

I want a specific packet format for packet exchange between a
client server across the network. For example frame format
as a python class could be:
class Frame:
def __init__(self, buffer=None, count=0, offset=0):
self.buffer = buffer
self.count = count
self.offset = offset
the question is how to convert it to a byte stream so that format
of count and offset also becomes a sequence of bytes.

Try struct.

Regards,

Björn

--
BOFH excuse #208:

Your mail is being routed through Germany ... and they're censoring
us.

struct module pack and unpack will only work for fixed size buffer :
pack('>1024sIL', buffer, count. offset) but the buffer size can vary
from one packet to the next :-(

-ishwar

[Felipe Almeida Lessa]

On 7 Feb 2007 19:14:13 -0800, ra****@cps.cmich.edu <ra****@cps.cmich.edu>

struct module pack and unpack will only work for fixed size buffer :
pack('>1024sIL', buffer, count. offset) but the buffer size can vary
from one packet to the next :-(

Then send the size of the buffer before the buffer, so the recipient
can expect that many bytes.

--
Felipe.

[Gabriel Genellina]

En Thu, 08 Feb 2007 00:14:13 -0300, <ra****@cps.cmich.eduescribió:

On Feb 8, 1:43 am, Bjoern Schliessmann <usenet-
[e-mail]:

>[e-mail] wrote:

I want a specific packet format for packet exchange between a
client server across the network. For example frame format
as a python class could be:
class Frame:
def __init__(self, buffer=None, count=0, offset=0):
self.buffer = buffer
self.count = count
self.offset = offset
the question is how to convert it to a byte stream so that format
of count and offset also becomes a sequence of bytes.

Try struct.

struct module pack and unpack will only work for fixed size buffer :
pack('>1024sIL', buffer, count. offset) but the buffer size can vary
from one packet to the next :-(

Use struct to pack count and offset into a fixed size field (2 or 4 bytes
depending on range); send buffer after that.
buffer must come *after* you send its size, else it's a lot harder to
retrieve at the other end.

--
Gabriel Genellina

[Grant Edwards]

On 2007-02-08, ra****@cps.cmich.edu <ra****@cps.cmich.eduwrote:

struct module pack and unpack will only work for fixed size buffer :
pack('>1024sIL', buffer, count. offset) but the buffer size can vary
from one packet to the next :-(

Oh for Pete's sake...

struct.pack('>%dsIL' % len(buffer), buffer, count, offset)

--
Grant Edwards grante Yow! I want the presidency
at so bad I can already taste
visi.com the hors d'oeuvres.

[rattan]

On Feb 8, 3:40 am, Grant Edwards <[e-mail]:

On 2007-02-08, [e-mail] <[e-mail]:

struct module pack and unpack will only work for fixed size buffer :
pack('>1024sIL', buffer, count. offset) but the buffer size can vary
from one packet to the next :-(

Oh for Pete's sake...

struct.pack('>%dsIL' % len(buffer), buffer, count, offset)

--
Grant Edwards grante Yow! I want the presidency
at so bad I can already taste
visi.com the hors d'oeuvres.

that is great but how does one unpack on the other side?

-ishwar

[Diez B. Roggisch]

ra****@cps.cmich.edu wrote:

On Feb 8, 3:40 am, Grant Edwards <[e-mail]:

>On 2007-02-08, [e-mail] <[e-mail]:

struct module pack and unpack will only work for fixed size buffer :
pack('>1024sIL', buffer, count. offset) but the buffer size can vary
from one packet to the next :-(

Oh for Pete's sake...

struct.pack('>%dsIL' % len(buffer), buffer, count, offset)

--
Grant Edwards grante Yow! I want the
presidency
at so bad I can already
taste
visi.com the hors d'oeuvres.

that is great but how does one unpack on the other side?

By peeking into the header, determining the number of bytes necessary?

But why do you need this anyway - if you know that you will have python on
both ends, use Pyro or xmlrpc.

Diez

[Grant Edwards]

On 2007-02-08, ra****@cps.cmich.edu <ra****@cps.cmich.eduwrote:

On Feb 8, 3:40 am, Grant Edwards <[e-mail]:

>On 2007-02-08, [e-mail] <[e-mail]:

struct module pack and unpack will only work for fixed size buffer :
pack('>1024sIL', buffer, count. offset) but the buffer size can vary
from one packet to the next :-(

Oh for Pete's sake...

struct.pack('>%dsIL' % len(buffer), buffer, count, offset)

that is great but how does one unpack on the other side?

struct.unpack('>%dsIL' % buflen ,packet)

--
Grant Edwards grante Yow! Yow! Did something
at bad happen or am I in a
visi.com drive-in movie??

[Grant Edwards]

On 2007-02-08, Diez B. Roggisch <de***@nospam.web.dewrote:

>>>struct module pack and unpack will only work for fixed size buffer :
pack('>1024sIL', buffer, count. offset) but the buffer size can vary
from one packet to the next :-(

Oh for Pete's sake...

struct.pack('>%dsIL' % len(buffer), buffer, count, offset)

that is great but how does one unpack on the other side?

By peeking into the header, determining the number of bytes necessary?

Better yet, use a protocol that's not brain-dead: send the
buffer size _before_ you send the buffer.

But why do you need this anyway - if you know that you will
have python on both ends, use Pyro or xmlrpc.

--
Grant Edwards grante Yow! A dwarf is passing
at out somewhere in Detroit!
visi.com