By using this site, you agree to our updated Privacy Policy and our Terms of Use. Manage your Cookies Settings.
446,361 Members | 2,131 Online
Bytes IT Community
+ Ask a Question
Need help? Post your question and get tips & solutions from a community of 446,361 IT Pros & Developers. It's quick & easy.

strange file.write() behavior on windows: $ConvertToNonresident, $ReplaceAttribute2

P: n/a
while taking some rough disk performance measures on windows machines,
and snooping with FileMon, i've noticed some odd behavior

here's the little nul-writer i'm running:
def writeTest(nBlocks, blockSize):
"""write nBlocks*blockSize nuls to a file"""
f = open("writeTest.out", "wb")
nulBlock = '\0'*blockSize
while nBlocks:
f.write(nulBlock)
nBlocks -= 1
f.close()

when i invoke this with a blockSize of 64K or less, no surprises.
here's a row of FileMon output for the 64K block size:
2:44:48 PM python.exe:3064 WRITE C:\writeTest.out SUCCESS Offset:
214106112 Length: 65536

when i invoke this with larger blocksizes (say, 655360), FileMon output
looks like this:
2:37:46 PM python.exe:3064 WRITE C:\writeTest.out Offset: 140902400
Length: 655360
2:37:46 PM python.exe:3064 WRITE C:\$ReplaceAttribute2 SUCCESS Offset:
140902400 Length: 65536
....(the $ReplaceAttribute2 line repeats several times, writing 65536
and advancing the offset by 65536 each time)

sometimes, depending on the phase of the moon, instead of
$ReplaceAttribute2 the write goes to $ConvertToNonresident. within a
single run of writeTest the write always goes to the same one,
whichever one that is. these runs are always slower (sometimes greatly)
than writing the same with 64K blocks, even though many fewer
file.write()'s are being issued because of the larger block size.

finally, where you'd expect an even 10 of the WRITE
C:\$ReplaceAttribute2 lines per WRITE C:\writeTest.out lines in the
example above, instead FileMon reports 8 lines for the first, 6 for the
second, 8 for the third, 6 for the fourth, etc... i've no idea if this
means FileMon is missing messages, but this pattern is absolutely
consistent across every run i've made, on both xp and win2k3 server
machines.

a look at the python fileobject.c source shed no light for me, anyone
know what could be going on? the "equivalent" c version of writeTest
using fwrite() shows a succession of 1K blocks writing to the named
file (even when fwrite is given 64K blocks), and no mysterious
$ReplaceAttribute2 lines

-- w

Nov 22 '05 #1
Share this Question
Share on Google+
4 Replies


P: n/a
This is really just a snide joke at the expense of Microsoft, but have you
checked the MSDN documentation about ConvertToNonresident or ReplaceAttribute2?

Your search - site:msdn.microsoft.com ConvertToNonresident OR
ReplaceAttribute2 - did not match any documents.
-- google.com

Internally, Python also simply calls fwrite(). Are you using the same C
runtime for both sets of tests?

Jeff

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.1 (GNU/Linux)

iD8DBQFDe8alJd01MZaTXX0RAjpDAKCPpv7J3sJ21YszxDjknK G0dEvjRwCfQfP5
oU/o2FKjl0cDqqr3dDufeyU=
=C/YG
-----END PGP SIGNATURE-----

Nov 22 '05 #2

P: n/a
This is really just a snide joke at the expense of Microsoft, but have you
checked the MSDN documentation about ConvertToNonresident or ReplaceAttribute2?

Your search - site:msdn.microsoft.com ConvertToNonresident OR
ReplaceAttribute2 - did not match any documents.
-- google.com

Internally, Python also simply calls fwrite(). Are you using the same C
runtime for both sets of tests?

Jeff

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.1 (GNU/Linux)

iD8DBQFDe8alJd01MZaTXX0RAjpDAKCPpv7J3sJ21YszxDjknK G0dEvjRwCfQfP5
oU/o2FKjl0cDqqr3dDufeyU=
=C/YG
-----END PGP SIGNATURE-----

Nov 22 '05 #3

P: n/a
"welch" <we***************@gmail.com> wrote:
while taking some rough disk performance measures on windows machines,
and snooping with FileMon, i've noticed some odd behavior sometimes, depending on the phase of the moon, instead of
$ReplaceAttribute2 the write goes to $ConvertToNonresident. within a
single run of writeTest the write always goes to the same one,
whichever one that is. these runs are always slower (sometimes greatly)
than writing the same with 64K blocks, even though many fewer
file.write()'s are being issued because of the larger block size.

finally, where you'd expect an even 10 of the WRITE
C:\$ReplaceAttribute2 lines per WRITE C:\writeTest.out lines in the
example above, instead FileMon reports 8 lines for the first, 6 for the
second, 8 for the third, 6 for the fourth, etc... i've no idea if this
means FileMon is missing messages, but this pattern is absolutely
consistent across every run i've made, on both xp and win2k3 server
machines.

a look at the python fileobject.c source shed no light for me, anyone
know what could be going on? the "equivalent" c version of writeTest
using fwrite() shows a succession of 1K blocks writing to the named
file (even when fwrite is given 64K blocks), and no mysterious
$ReplaceAttribute2 lines


you're seeing activities by the Windows caching system (c:\$ stuff is various
NTFS data streams). unless you're writing low-level drivers, you don't really
have to care about what it does, and when it does what...

</F>

Nov 22 '05 #4

P: n/a
"welch" <we***************@gmail.com> wrote:
while taking some rough disk performance measures on windows machines,
and snooping with FileMon, i've noticed some odd behavior sometimes, depending on the phase of the moon, instead of
$ReplaceAttribute2 the write goes to $ConvertToNonresident. within a
single run of writeTest the write always goes to the same one,
whichever one that is. these runs are always slower (sometimes greatly)
than writing the same with 64K blocks, even though many fewer
file.write()'s are being issued because of the larger block size.

finally, where you'd expect an even 10 of the WRITE
C:\$ReplaceAttribute2 lines per WRITE C:\writeTest.out lines in the
example above, instead FileMon reports 8 lines for the first, 6 for the
second, 8 for the third, 6 for the fourth, etc... i've no idea if this
means FileMon is missing messages, but this pattern is absolutely
consistent across every run i've made, on both xp and win2k3 server
machines.

a look at the python fileobject.c source shed no light for me, anyone
know what could be going on? the "equivalent" c version of writeTest
using fwrite() shows a succession of 1K blocks writing to the named
file (even when fwrite is given 64K blocks), and no mysterious
$ReplaceAttribute2 lines


you're seeing activities by the Windows caching system (c:\$ stuff is various
NTFS data streams). unless you're writing low-level drivers, you don't really
have to care about what it does, and when it does what...

</F>

Nov 22 '05 #5

This discussion thread is closed

Replies have been disabled for this discussion.