WSGI/wsgiref: modifying output on windows ?

[tjollans]

Hi,

I am currently trying to port my web software AFoC <http://afoc.k.vu>
to Windows and have hit a strange problem: it seams that binary files
(PNG images in this case) get distorted by wsgiref. (I have tried both
the CGIHandler with both Xitami and Apache2 and WSGIServer). My
application is returning a list of one item (the image as a string).
The image is intact at that point. (tested by writing it to disk and
comparing); The client gets a version with a few added \r characters
and something missing at the end, but of the same overall length. Here
are specific examples:

>>f1 = file('afoc_bak', 'rb')
f2 = file('box_top.png', 'rb')
s1 = f1.read()
s2 = f2.read()
s1 == s2

False

>>s1

"\x89PNG\r\n\x1a\n\x00\x00\x00\rIHDR
\x00\x00\x00\x96\x00\x00\x00\x14\x08\x06\x00
\x00\x00oy\xc41\x00\x00\x00\x04sBIT\x08\x08\x08\x0 8|\x08d
\x88\x00\x00\x00\x19tEX
tSoftware\x00www.inkscape.org\x9b\xee<\x1a\x00\x00 \x01\x95IDATh\x81\xed
\xd9?K#A\
x18\xc7\xf1\xef\xecnv\xb3FQA\x8b\xf3@\xc1?\xe0)*\x da\xd9(v\xd7+r
\xb6\x82\xef\xc3
\xce\x17\xe1\xab\xb0\x15k\x0b\x1b\x1b\xe1@\x8eK\xa 1\x85\x044F
\x8d1\x9a8\x16\x93\
x80\xc4F]u\x89\xfc>\xcd\xb0\x0fS<\xc5\x8fyfw\x8d\xb5\x16\x9 1\x8f
\xe6\xa5\xdd\x80
|O\n\x96|\n\x05K>\x85\x01\x06\xd2nB\xda\xc6\xe3\xb 3\xb5\x06<\x00Uk\xed}
\xeb\xc6\
x00\xf8\xf1\x85\x8dI{\xab7\xd6f\xb0\xee\x81\xaa1\x a6\x02\x94\xad
\xb5\x95\xe6\xc6
\x00\x9dX\xf2zU\xc0\xc7\x05\xac\xd6x\xbe\x03n\x81\ xd8\x18s\x03\\Yk\xeb:
\xb1\xe4-
n\x80\x08\x17\xac\x07\\\xb0n\x812\x90\x05B\xc07\xc 6\x94\x82\xf9\xa5\xad
\xb5\xd4\
xda\x94\xb6\xe0y\xbe\x0fP*\xfe?>\xc9\xef\x15J\xc5\ x7f\x97\xb81x
\x87\x0bUL#T\xb8{
\xbb5\xab\xeb\x07\xfaB*oR)\x17\xce\x8f\x0e\xb7w\xf 3\xc7;\x7f\x81k
\xe0\n(\x01\x97
@\x11\xb8\xf0'g76S\xecQ\xdaP&\xccu\xfc\x1c\\\x98\n \xa3n\xef\xect\xbf
\xd0([\xdc\x
a5\xbe\x0e\xd4\xf4\x1dK\xdem\xf4\xd7\xca\xc2\xc8\x f8\xf24\xd0\x05t
\x02\x1d\xb8\x
b1\x18+X\xf2n\xc6xfbf}\x11\x17\xa8,\xeeb\x1f\x01\x a1\x82%
\x89\xc4\xb9\xfe\x9e\xb
1\xc9?\x11\xee2_\xc7\x8dC\xab`IbC\xc3\xbf\xe7p\xa3 \xb0\xf9f\xa8\x7f
\x85\x92\\\x9
4\xed\xedk\xad)X\x92X&\xcc\xf5\xb6\xd6\x14,I,\x08\ xe2\x9e\xd6\x9a\x82%
\x89y~&\xf
b\xa2\x96F#\xf2\xfd=\x01\x19c^\xe3KN&\x89\x00\x00\ x00\x00IEND\xaeB`
\x82"

>>s2

"\x89PNG\r\r\n\x1a\r\n\x00\x00\x00\rIHDR
\x00\x00\x00\x96\x00\x00\x00\x14\x08\x06
\x00\x00\x00oy\xc41\x00\x00\x00\x04sBIT\x08\x08\x0 8\x08|\x08d
\x88\x00\x00\x00\x1
9tEXtSoftware\x00www.inkscape.org\x9b\xee<\x1a\x00 \x00\x01\x95IDATh
\x81\xed\xd9?
K#A\x18\xc7\xf1\xef\xecnv\xb3FQA\x8b\xf3@\xc1?\xe0 )*\xda\xd9(v\xd7+r
\xb6\x82\xef
\xc3\xce\x17\xe1\xab\xb0\x15k\x0b\x1b\x1b\xe1@\x8e K\xa1\x85\x044F
\x8d1\x9a8\x16\
x93\x80\xc4F]u\x89\xfc>\xcd\xb0\x0fS<\xc5\x8fyfw\x8d\xb5\x16\x9 1\x8f
\xe6\xa5\xdd
\x80|O\r\n\x96|\r\n\x05K>\x85\x01\x06\xd2nB\xda\xc 6\xe3\xb3\xb5\x06<
\x00Uk\xed}\
xeb\xc6\x00\xf8\xf1\x85\x8dI{\xab7\xd6f\xb0\xee
\x81\xaa1\xa6\x02\x94\xad\xb5\x95
\xe6\xc6\x00\x9dX\xf2zU\xc0\xc7\x05\xac\xd6x\xbe\x 03n\x81\xd8\x18s\x03\
\Yk\xeb:\
xb1\xe4-n\x80\x08\x17\xac\x07\\\xb0n\x812\x90\x05B
\xc07\xc6\x94\x82\xf9\xa5\xad\
xb5\xd4\xda\x94\xb6\xe0y\xbe\x0fP*\xfe?>\xc9\xef\x 15J\xc5\x7f\x97\xb81x
\x87\x0bU
L#T\xb8{\xbb5\xab\xeb\x07\xfaB*oR)\x17\xce\x8f\x0e \xb7w\xf3\xc7;\x7f
\x81k\xe0\r\
n(\x01\x97@\x11\xb8\xf0'g76S\xecQ\xdaP&\xccu\xfc\x 1c\\\x98\r\n\xa3n\xef
\xect\xbf
\xd0([\xdc\xa5\xbe\x0e\xd4\xf4\x1dK\xdem\xf4\xd7\xca
\xc2\xc8\xf8\xf24\xd0\x05t\x
02\x1d\xb8\xb1\x18+X\xf2n\xc6xfbf}\x11\x17\xa8,\xe eb\x1f\x01\xa1\x82%
\x89\xc4\xb
9\xfe\x9e\xb1\xc9?\x11\xee2_\xc7\x8dC\xab`IbC\xc3\ xbf\xe7p\xa3\xb0\xf9f
\xa8\x7f\
x85\x92\\\x94\xed\xedk\xad)X\x92X&\xcc\xf5\xb6\xd6 \x14,I,\x08\xe2\x9e
\xd6\x9a\x8
2%\x89y~&\xfb\xa2\x96F#\xf2\xfd=\x01\x19c^\xe3KN&
\x89\x00\x00\x00\x00IE"

>>>

a thought that springs to mind is that this may be related to the
barmy DOS relic of binary/ASCII file destinction; however, this should
not be a problem if I understand PEP 333 correctly:

"""The server or gateway should treat the yielded strings as binary
byte sequences: in particular, it should ensure that line endings are
not altered."""

Might this be a bug in wsgiref ? I will hopefully be able to do some
more testing, e.g. with the FLUP FastCGI back-end, in the next few
days.

Kind regards,

Thomas Jollans

[tjollans]

On Jun 3, 10:11 pm, "[e-mail]"
<[e-mail]:

Hi,

I am currently trying to port my web software AFoC <http://afoc.k.vu>
to Windows and have hit a strange problem: it seams that binary files
(PNG images in this case) get distorted by wsgiref. (I have tried both
the CGIHandler with both Xitami and Apache2 and WSGIServer).

I have just realised that the WSGIServer works after all, I do not
know why I thought it didn't.

[tjollans]

On Jun 3, 10:11 pm, "[e-mail]"
<[e-mail]:

Might this be a bug in wsgiref ? I will hopefully be able to do some
more testing, ...

The following simple CGI script should, AFAIK, on any platform, output
exactly the file specified in code. It does not on Apache 2 on
Windows; I believe this to be a bug in wsgiref.handlers.CGIHandler.
Can someone test this in a similar environment (Windows+a web server?)
to confirm this ? (The first two lines need to be edited depending on
the site)

#!c:\python25\python.exe
fname, type = r'c:\hdr.png', 'image/png'

from wsgiref.handlers import CGIHandler

def wsgiapp(env, start_response):
f = file(fname, 'rb')
start_response('200 OK', [('Content-Type', type)])
return [f.read()]
CGIHandler().run(wsgiapp)