By using this site, you agree to our updated Privacy Policy and our Terms of Use. Manage your Cookies Settings.
446,176 Members | 1,101 Online
Bytes IT Community
+ Ask a Question
Need help? Post your question and get tips & solutions from a community of 446,176 IT Pros & Developers. It's quick & easy.

struct unpack newline

P: n/a
Hi All,

I am pretty new to python and am having a problem
intepreting binary data using struct.unpack.
I am reading a file containing binary packed data
using open with "rb". All the values are coming through
fine when using (integer1,) = struct.unpack('l', line[86:90])
except when line[86:90] contains "carriage-return" "linefeed"
which are valid binary packed values. Error = unpack
string size dows not match format. It seems that
struct, instead of reading 4 bytes for line[86:90]
only reads 2 bytes if the second byte is CR or LF.

Thanks
Grant

Jul 19 '05 #1
Share this Question
Share on Google+
5 Replies


P: n/a
gr***@idscape.co.za wrote:
I am pretty new to python and am having a problem
intepreting binary data using struct.unpack.
I am reading a file containing binary packed data
using open with "rb". All the values are coming through
fine when using (integer1,) = struct.unpack('l', line[86:90])
except when line[86:90] contains "carriage-return" "linefeed"
which are valid binary packed values. Error = unpack
string size dows not match format. It seems that
struct, instead of reading 4 bytes for line[86:90]
only reads 2 bytes if the second byte is CR or LF.


verifying that struct doesn't care about newlines is of course
pretty trivial:
import struct
struct.unpack("l", "\0\0\0\0") (0,) struct.unpack("l", "\0\r\0\0") (3328,) struct.unpack("l", "\0\n\0\0") (2560,)

have you verified that len(line) really is what you think?
struct.unpack("l", "\0\r\n\0") (658688,) struct.unpack("l", "\0\n\0")

Traceback (most recent call last):
File "<stdin>", line 1, in ?
struct.error: unpack str size does not match format

</F>

Jul 19 '05 #2

P: n/a

<gr***@idscape.co.za> wrote in message news:11**********************@g14g2000cwa.googlegr oups.com...
except when line[86:90] contains "carriage-return" "linefeed"
which are valid binary packed values.


You probably don't want to be reading binary data a
line at a time, if that's what you're doing.
Jul 19 '05 #3

P: n/a
Hi ,

Thanks for the tip regarding checking the length of line. I discovered
that on the problem record it was short by a few bytes. After changing
the read method from "for line in.." to "infile.read(n)" my problem was
solved,
what concerns me though is that although the file is opened in binary
mode,
"for line.." has a problem reading the file correctly.

Thanks
Grant
Fredrik Lundh wrote:
gr***@idscape.co.za wrote:
I am pretty new to python and am having a problem
intepreting binary data using struct.unpack.
I am reading a file containing binary packed data
using open with "rb". All the values are coming through
fine when using (integer1,) = struct.unpack('l', line[86:90])
except when line[86:90] contains "carriage-return" "linefeed"
which are valid binary packed values. Error = unpack
string size dows not match format. It seems that
struct, instead of reading 4 bytes for line[86:90]
only reads 2 bytes if the second byte is CR or LF.


verifying that struct doesn't care about newlines is of course
pretty trivial:
import struct
struct.unpack("l", "\0\0\0\0") (0,) struct.unpack("l", "\0\r\0\0") (3328,) struct.unpack("l", "\0\n\0\0") (2560,)

have you verified that len(line) really is what you think?
struct.unpack("l", "\0\r\n\0") (658688,) struct.unpack("l", "\0\n\0")

Traceback (most recent call last):
File "<stdin>", line 1, in ?
struct.error: unpack str size does not match format

</F>


Jul 19 '05 #4

P: n/a
gr***@idscape.co.za wrote:
what concerns me though is that although the file is opened in binary
mode,
"for line.." has a problem reading the file correctly.


There is _no_ correct way of splitting a file containing binary data in
lines because binary data may contain newline bytes that do not indicate a
new line. E. g.
struct.unpack("l", "\r\n\r\n")

(168626701,)

how should Python know whether it just encountered two empty lines or the
integer 168626701?

Peter

Jul 19 '05 #5

P: n/a
Good point. Hadn't thouhgt of that.

Thanks
Grant

Jul 19 '05 #6

This discussion thread is closed

Replies have been disabled for this discussion.