By using this site, you agree to our updated Privacy Policy and our Terms of Use. Manage your Cookies Settings.
464,360 Members | 1,313 Online
Bytes IT Community
+ Ask a Question
Need help? Post your question and get tips & solutions from a community of 464,360 IT Pros & Developers. It's quick & easy.

Windows process ownership trouble

P: n/a
Am currently being very confused over the following code on Windows

import subprocess, os

file = open("filename", "w")
try:
proc = subprocess.Popen("nosuchprogram", stdout=file)
except OSError:
file.close()
os.remove("filename")

This produces the following exception:

Traceback (most recent call last):
File "C:\processown.py", line 10, in <module>
os.remove("filename")
WindowsError: [Error 32] The process cannot access the file because it
is being used by another process: 'filename'

How can it be in use by another process? The process didn't even
start, right?

Would appreciate some help: is this a Python bug, or a Windows bug, or
just me being confused...?
Jun 27 '08 #1
Share this Question
Share on Google+
7 Replies

P: n/a
geoffbache wrote:
Am currently being very confused over the following code on Windows

import subprocess, os

file = open("filename", "w")
try:
proc = subprocess.Popen("nosuchprogram", stdout=file)
except OSError:
file.close()
os.remove("filename")

This produces the following exception:

Traceback (most recent call last):
File "C:\processown.py", line 10, in <module>
os.remove("filename")
WindowsError: [Error 32] The process cannot access the file because it
is being used by another process: 'filename'

How can it be in use by another process? The process didn't even
start, right?
[slight aside: best not to use "file" as an identifier; it shadows the
builtin "file" factory function. Not fatal, but might trip you up
somewhere later.]

This is awkward. What's happening behind the scenes is that,
as it creates the Popen object, the subprocess module gets
hold of the Windows handle behind your file. Then it calls
the CreateProcess API to create the process. If that succeeds,
it does whatever it does, closes the file handles and returns.
If it fails -- and here it fails, obviously -- it reraises the error
as a WindowsError. WindowsError is a subclass of OSError so
is trapped by your exception handler.

But -- and this is the point -- this exception is raised before the
the internal handles are closed, and altho' I don't know exactly
what effect this will have it clearly prevents the file from being
removed. And, as far as I can see, will continue to prevent its
being removed until the whole Python process exits and its
handles released.

I'm happy to be corrected on this, but it looks to me like a
bug in the subprocess error handling. If I'm right, I'm a little
surprised this hasn't bitten someone before but a quick search
of the bugs database doesn't seem to turn anything up.

TJG
Jun 27 '08 #2

P: n/a
geoffbache wrote:
Am currently being very confused over the following code on Windows

import subprocess, os

file = open("filename", "w")
try:
proc = subprocess.Popen("nosuchprogram", stdout=file)
except OSError:
file.close()
os.remove("filename")
Forgot to say: slightly awkward, but you can work around
it like this:

<code>
import os
import subprocess

f = open ("filename", "w")
try:
proc = subprocess.Popen ("blah", stdout=f)
except OSError:
os.close (f.fileno ())

os.remove ("filename")

</code>

TJG
Jun 27 '08 #3

P: n/a
Thanks Tim, very helpful again.

I've now reported this as http://bugs.python.org/issue3210
and implemented your suggested workaround.

Regards,
Geoff

On Jun 25, 9:19 pm, Tim Golden <m...@timgolden.me.ukwrote:
geoffbache wrote:
Am currently being very confused over the following code on Windows
import subprocess, os
file = open("filename", "w")
try:
proc = subprocess.Popen("nosuchprogram", stdout=file)
except OSError:
file.close()
os.remove("filename")

Forgot to say: slightly awkward, but you can work around
it like this:

<code>
import os
import subprocess

f = open ("filename", "w")
try:
proc = subprocess.Popen ("blah", stdout=f)
except OSError:
os.close (f.fileno ())

os.remove ("filename")

</code>

TJG
Jun 27 '08 #4

P: n/a

Tim,

Unfortunately my previous message was premature, it seems your
workaround doesn't work
either on my system (Windows XP, Python 2.5.1) I get the following
printed out

Traceback (most recent call last):
File "C:\TextTest\processown.py", line 12, in <module>
os.remove ("filename")
WindowsError: [Error 32] The process cannot access the file because it
is being used by another process: 'filename'
close failed: [Errno 9] Bad file descriptor

Any ideas?

Geoff
Jun 27 '08 #5

P: n/a
geoffbache wrote:
Tim,

Unfortunately my previous message was premature, it seems your
workaround doesn't work
either on my system (Windows XP, Python 2.5.1) I get the following
printed out

Traceback (most recent call last):
File "C:\TextTest\processown.py", line 12, in <module>
os.remove ("filename")
WindowsError: [Error 32] The process cannot access the file because it
is being used by another process: 'filename'
close failed: [Errno 9] Bad file descriptor
Strange. I've just double-checked and it works perfectly
well for me, although I haven't traced all the way through
the code paths of the subprocess module: it was a bit of
an educated guess. Just to confirm, I'm talking about this
code running on Python 2.5.2 on Win XP SP2.

<code>
import os
import subprocess

f = open ("filename", "w")
try:
proc = subprocess.Popen ("blah", stdout=f)
except OSError:
os.close (f.fileno ())

os.remove ("filename")
</code>

If I get a chance I'll try to look more closely at the subprocess
code. Could you dump out the code (or the interpreter session
in its entirety) just to make sure there's not a typo or a thinko
getting in the way?

TJG
Jun 27 '08 #6

P: n/a
geoffbache wrote:
Tim,

Unfortunately my previous message was premature, it seems your
workaround doesn't work
either on my system (Windows XP, Python 2.5.1) I get the following
printed out

Traceback (most recent call last):
File "C:\TextTest\processown.py", line 12, in <module>
os.remove ("filename")
WindowsError: [Error 32] The process cannot access the file because it
is being used by another process: 'filename'
close failed: [Errno 9] Bad file descriptor
(Assuming you have the pywin32 extensions installed...)

If you tweak your copy of subprocess.py by switching on
the use of the pywin32 extensions in preference to the
_subprocess module, the problem goes away, I think
because the PyHANDLE object auto-closes. I'm not saying
it won't ever fail -- the interactions of objects, scope,
frames, exceptions and garbage collection are things I've
never looked at too closely. But if you change line 378 from
if 0: to if 1:, then the following works fine:

<code>
import os
import subprocess

f = open ("filename", "w")
try:
p = subprocess.Popen ("blah", stdout=f)
except WindowsError:
f.close ()

os.remove ("filename")

</code>

Note that the os.remove is *outside* the except handler.
This is, I think, because the underlying handle object
is still active until the try/except block closes. At which
point it is, probably, gc-ed and closes. And you can
remove the file.

TJG
Jun 27 '08 #7

P: n/a
Tim Golden wrote:
geoffbache wrote:
>Tim,

Unfortunately my previous message was premature, it seems your
workaround doesn't work
either on my system (Windows XP, Python 2.5.1) I get the following
printed out

Traceback (most recent call last):
File "C:\TextTest\processown.py", line 12, in <module>
os.remove ("filename")
WindowsError: [Error 32] The process cannot access the file because it
is being used by another process: 'filename'
close failed: [Errno 9] Bad file descriptor
[... snip purported solution using pywin32 ...]

..... or maybe I should just stop talking rubbish and go home
since that solution works even *without* the tweak to
subprocess.py.

TJG
Jun 27 '08 #8

This discussion thread is closed

Replies have been disabled for this discussion.