How to bypass Windows 'cooking' the I/O? (One more time, please)

[norseman]

I know I saw the answer recently, as in since February '08, but I can't
re-find it. :( I tried the mail archives and such and my own
collections but the piece I saw still eludes me.
Problem: (sos=same old s...) Microsoft insists the world work it's way
even when the Microsoft way was proven wrong decades ago. In this case
it's (still) 'cooking' the writes even with 'rwb' and O_RDWR|O_BINARY in
(proper respective) use.

Specific: python created and inspected binary file ends:
00460: 0D 1A (this is correct)

after a write
os.lseek(target, -1, 2)
os.write(target,record)
the expected result would be:
00460: 0D 20 .....data bytes.... 1A

BUT I get:
00460: 20 .... data bytes... 1A

It is one byte off!!! And the 0D has to be there. Signifies the end of
the header.
Same python program runs as expected in Linux. Maybe because that's
where it was written?! :)
What I seek is the way to slap Microsoft up side the head and make it
work correctly. OK, well, at least in this situation.
Note: Things like this justify Python implementers bypassing OS calls
(data fetch, data write) and using the BIOS direct. Remember, the CPU
understands bit patterns only. It has no comprehension of 'text',
'program', 'number', 'pointer', blah blah blah.... All that is totally
beyond it's understanding. A given bit pattern means 'do that'. The CPU
is 100% binary. Memory, storage and the rest is just bits-on, bits-off.
Patterns. Proper binary I/O is mandatory for the machine to function.
Anyway - if whoever mentioned the flags and such to 'over ride'
Microsoft's BS would re-send that piece I would be very appreciative.
Steve
no******@hughes.net

[Larry Bates]

norseman wrote:

>

I know I saw the answer recently, as in since February '08, but I can't
re-find it. :( I tried the mail archives and such and my own
collections but the piece I saw still eludes me.
Problem: (sos=same old s...) Microsoft insists the world work it's way
even when the Microsoft way was proven wrong decades ago. In this case
it's (still) 'cooking' the writes even with 'rwb' and O_RDWR|O_BINARY in
(proper respective) use.

Specific: python created and inspected binary file ends:
00460: 0D 1A (this is correct)

after a write
os.lseek(target, -1, 2)
os.write(target,record)
the expected result would be:
00460: 0D 20 .....data bytes.... 1A

BUT I get:
00460: 20 .... data bytes... 1A

It is one byte off!!! And the 0D has to be there. Signifies the end of
the header.
Same python program runs as expected in Linux. Maybe because that's
where it was written?! :)
What I seek is the way to slap Microsoft up side the head and make it
work correctly. OK, well, at least in this situation.
Note: Things like this justify Python implementers bypassing OS calls
(data fetch, data write) and using the BIOS direct. Remember, the CPU
understands bit patterns only. It has no comprehension of 'text',
'program', 'number', 'pointer', blah blah blah.... All that is totally
beyond it's understanding. A given bit pattern means 'do that'. The CPU
is 100% binary. Memory, storage and the rest is just bits-on, bits-off.
Patterns. Proper binary I/O is mandatory for the machine to function.
Anyway - if whoever mentioned the flags and such to 'over ride'
Microsoft's BS would re-send that piece I would be very appreciative.
Steve
no******@hughes.net

Since you didn't show any code context, I'll try my mind reading superpowers.....

You may be the victim of buffering (not calling .flush() or .close() to commit
your write to disk). Why aren't you using the file object to do you seek and write?

Normal file I/O sequence:

fp = open(target, 'wb')

fp.seek(-1, 2)

fp.write(record)

by going through os. methods instead of the file instance I think you are
accessing the file through 2 different I/O buffers. I could be all wrong here.

-Larry

[Ulrich Eckhardt]

norseman wrote:

In this case it's [MS Windows] (still) 'cooking' the writes even
with 'rwb' and O_RDWR|O_BINARY in (proper respective) use.

Specific: python created and inspected binary file ends:
00460: 0D 1A (this is correct)

I'm not actually sure what the 0x1a is supposed to do there. The 0x0d is
a '\r' which looks like a MacOS line ending, but I also seem to remember
0x1a being use as EOF signal in some old system's text files. Anyway, you
mentioned some "binary file" (aren't they all?) so this might not matter
after all.

after a write
os.lseek(target, -1, 2)

That should be os.SEEK_END and not 2. In any case, it moves the write
position to the last byte (-1 offset to the end of the file).

os.write(target,record)
the expected result would be:
00460: 0D 20 .....data bytes.... 1A

I'm sorry, but I don't see that. write() will not insert the bytes but first
overwrite till EOF and then append further output, i.e. the 0x1a at the end
is only present if it was present in the record.

BUT I get:
00460: 20 .... data bytes... 1A

It is one byte off!!! And the 0D has to be there. Signifies the end of
the header.

Okay, this looks strange, both the offset and the fact that it seems to
insert bytes. How about providing a minimal example?

Anyway - if whoever mentioned the flags and such to 'over ride'
Microsoft's BS would re-send that piece I would be very appreciative.

You are not one with the Tao, which is why you programs don't run. ;)

Uli

--
Sator Laser GmbH
Geschäftsführer: Thorsten Föcking, Amtsgericht Hamburg HR B62 932