piping out binaries properly

[Andy Leszczynski]

I have got following program:

import sys
import binascii
from string import *
sys.stdout.write(binascii.unhexlify("41410A4141"))
when I run under Unix I got:

$ python u.py > u.bin
$ od -t x1 u.bin
0000000 41 41 0a 41 41

and under Windows/Cygwin following:

$ python u.py > u.bin
$ od -t x1 u.bin
0000000 41 41 0d 0a 41 41
0000006
The question is how can I pipe out binary content properly and platform
independently?
A.

[Mike Meyer]

Andy Leszczynski <leszczynscyATnospam.yahoo.com.nospam> writes:

I have got following program:

import sys
import binascii
from string import *
sys.stdout.write(binascii.unhexlify("41410A4141"))
when I run under Unix I got:

$ python u.py > u.bin
$ od -t x1 u.bin
0000000 41 41 0a 41 41

and under Windows/Cygwin following:

$ python u.py > u.bin
$ od -t x1 u.bin
0000000 41 41 0d 0a 41 41
0000006

The question is how can I pipe out binary content properly and platform
independently?

It's not normal to write binary content to stdout - you normally write
it to a file. Open the file with open(name, 'wb') to write binaries.

There doesn't appear to be any way to retroactively change the mode on
a file. Which is probably a good thing.

<mike
--
Mike Meyer <mw*@mired.org> http://www.mired.org/home/mwm/
Independent WWW/Perforce/FreeBSD/Unix consultant, email for more information.

[Andy Leszczynski]

Mike Meyer wrote:

It's not normal to write binary content to stdout - you normally write

Well, I grew up in the Unix world and it is normal over there.

I am still curious which layer adds that 0xd. Is it python, cygwin,
windows ...

Thx for reply, Andy

[marduk]

On Wed, 2005-10-12 at 00:16 -0400, Mike Meyer wrote:
[...]

It's not normal to write binary content to stdout - you normally write
it to a file. Open the file with open(name, 'wb') to write binaries.

It is interesting that as a "Unix consultant" you should make that
claim. Especially since

sys.stdout

<open file '<stdout>', mode 'w' at 0x2aaaaaac9198>
^^^^

Indeed there would be a lot of broken Unix systems out there if that
were not the case.

As for the OP, you may want to check how stdout is opened on your
system. In Windows there are two "write" modes for files, 'w', and 'wb'
where apparently 'w' is text mode and 'wb' is binary. You may want to
check what mode your stdout is in. I don't have a Windows box handy
right now to verify.
There doesn't appear to be any way to retroactively change the mode on
a file. Which is probably a good thing.

<mike--
Mike Meyer <mw*@mired.org> http://www.mired.org/home/mwm/ Independent WWW/Perforce/FreeBSD/Unix consultant, email for more

information.

[Mike Meyer]

Andy Leszczynski <leszczynscyATnospam.yahoo.com.nospam> writes:

Mike Meyer wrote:

It's not normal to write binary content to stdout - you normally write
Well, I grew up in the Unix world and it is normal over there.

I watched the Unix world grow up, and it ain't normal to me. I don't
think I've ever written a program that wrote binary data to standard
out, not in nearly 30 years of unix programming. I've written lots of
things whose standard out was designed specifically to be read by
another program, but never as binary data.
I am still curious which layer adds that 0xd. Is it python, cygwin,
windows ...

It's Python. You tell Python whether or not it needs to do
platform-dependent newline mangling when you open the file.

<mike
--
Mike Meyer <mw*@mired.org> http://www.mired.org/home/mwm/
Independent WWW/Perforce/FreeBSD/Unix consultant, email for more information.

[Erik Max Francis]

Mike Meyer wrote:

I watched the Unix world grow up, and it ain't normal to me.
Since there's no distinction between a file opened in binary mode and in
text mode on Unix, there is no difference.
I don't
think I've ever written a program that wrote binary data to standard
out, not in nearly 30 years of unix programming. I've written lots of
things whose standard out was designed specifically to be read by
another program, but never as binary data.

Plenty of applications use that functionality and depend on it. See
cjpeg, djpeg, the pbmplus library, and so forth.

--
Erik Max Francis && ma*@alcyone.com && http://www.alcyone.com/max/
San Jose, CA, USA && 37 20 N 121 53 W && AIM erikmaxfrancis
Every human being is a problem in search of a solution.
-- Ashley Montagu

[Mike Meyer]

marduk <us****@marduk.letterboxes.org> writes:

On Wed, 2005-10-12 at 00:16 -0400, Mike Meyer wrote:
[...]

It's not normal to write binary content to stdout - you normally write
it to a file. Open the file with open(name, 'wb') to write binaries.

It is interesting that as a "Unix consultant" you should make that
claim. Especially since

sys.stdout

<open file '<stdout>', mode 'w' at 0x2aaaaaac9198>
^^^^

Indeed there would be a lot of broken Unix systems out there if that
were not the case.

Ain't ambiguity wonderful? That we use the same word to donote a
collection of bytes on a disk and a software entity that one can write
bytes to allows for all kind of mistaken interpretations.

<mike
--
Mike Meyer <mw*@mired.org> http://www.mired.org/home/mwm/
Independent WWW/Perforce/FreeBSD/Unix consultant, email for more information.

[Paul Rubin]

Erik Max Francis <ma*@alcyone.com> writes:

I've written lots of things whose standard out was designed
specifically to be read by another program, but never as binary data.

Plenty of applications use that functionality and depend on it. See
cjpeg, djpeg, the pbmplus library, and so forth.

Also "tar cf files... | compress > tarball.Z" used to be a standard
idiom, now replaced by "tar cfz ..." which does about the same thing under
the covers.

[Fredrik Lundh]

Andy Leszczynski wrote:

when I run under Unix I got:

$ python u.py > u.bin
$ od -t x1 u.bin
0000000 41 41 0a 41 41

and under Windows/Cygwin following:

$ python u.py > u.bin
$ od -t x1 u.bin
0000000 41 41 0d 0a 41 41
0000006

The question is how can I pipe out binary content properly and platform
independently?

$ python -h
usage: python [option] ... [-c cmd | file | -] [arg] ...
Options and arguments (and corresponding environment variables):
....
-u : unbuffered binary stdout and stderr (also PYTHONUNBUFFERED=x)
....

</F>

[Paul Watson]

"Andy Leszczynski" <leszczynscyATnospam.yahoo.com.nospam> wrote in message
news:o_********************@comcast.com...

I have got following program:

import sys
import binascii
from string import *
sys.stdout.write(binascii.unhexlify("41410A4141"))
when I run under Unix I got:

$ python u.py > u.bin
$ od -t x1 u.bin
0000000 41 41 0a 41 41

and under Windows/Cygwin following:

$ python u.py > u.bin
$ od -t x1 u.bin
0000000 41 41 0d 0a 41 41
0000006
The question is how can I pipe out binary content properly and platform
independently?
A.

try:
import msvcrt, os
msvcrt.setmode(sys.stdout.fileno(), os.O_BINARY)
except:
pass