468,110 Members | 1,835 Online
Bytes | Developer Community
New Post

Home Posts Topics Members FAQ

Post your question to a community of 468,110 developers. It's quick & easy.

copyfile avoiding overwrites and race conditions

Here is a code fragment, where I am trying to copy a file, avoiding
overwrites and race conditions. The filename gets a '02','03','04' etc
appended to the end if a file with that name already exists.

I know the writing of the single space is overkill, but I am surprised
I cannot find an example of this floating around the web and newsgroups

my understanding of 'os.open' and 'os.fdopen' is minimal

## start fragment

i = 0
while True:

if i >= 100: raise RanOutFileNamesError(fullpath)
i += 1

if i > 1:
name0 = '%s_%02i%s' % (suggested_name, i, file_extension)
else:
assert i == 1
name0 = '%s%s' % (suggested_name, file_extension)
fullpath = os.path.join(path0, name0)

# create dummy file, force overwrite

try:
fd = os.open(
fullpath,
os.O_CREAT | os.O_EXCL | os.O_WRONLY)
except OSError:
continue

# is this really necessary?
file = os.fdopen(fd, "w")
file.write(' ')
file.close()

shutil.copyfile(original_filepath, fullpath)

return fullpath

## end fragment

Jun 16 '06 #1
2 3367
ma*****@gmail.com wrote:
Here is a code fragment, where I am trying to copy a file, avoiding
overwrites and race conditions. The filename gets a '02','03','04' etc
appended to the end if a file with that name already exists.

I know the writing of the single space is overkill, but I am surprised
I cannot find an example of this floating around the web and newsgroups

my understanding of 'os.open' and 'os.fdopen' is minimal

## start fragment

i = 0
while True:

if i >= 100: raise RanOutFileNamesError(fullpath)
i += 1

if i > 1:
name0 = '%s_%02i%s' % (suggested_name, i, file_extension)
else:
assert i == 1
name0 = '%s%s' % (suggested_name, file_extension)
fullpath = os.path.join(path0, name0)

# create dummy file, force overwrite

try:
fd = os.open(
fullpath,
os.O_CREAT | os.O_EXCL | os.O_WRONLY)
except OSError:
continue

# is this really necessary?
file = os.fdopen(fd, "w")
file.write(' ')
file.close()

shutil.copyfile(original_filepath, fullpath)

return fullpath

## end fragment

I guess my approach would be different. To eliminate any race
conditions, I would keep a small text file that always contained
the next filename that is to be written. Something like:

nextfiletowrite=/path/filename006.dat

I would try to get a lock on this file, read it, extract next
filename, increment the counter portion of the filename,
write it back out and unlock it. Now I have the name of the
file to write that is unique to my instance and I can write it
without worrying about other processes.

Hope this helps.

-Larry Bates
Jun 19 '06 #2
Larry Bates wrote:
I guess my approach would be different. To eliminate any race
conditions, I would keep a small text file that always contained
the next filename that is to be written. Something like:

nextfiletowrite=/path/filename006.dat

I would try to get a lock on this file, read it, extract next
filename, increment the counter portion of the filename,
write it back out and unlock it. Now I have the name of the
file to write that is unique to my instance and I can write it
without worrying about other processes.


Yes, that would work as well

but I get the feeling that

1) most people don't write to an automatically incrementing filename
2) the few who do don't sweat the race condition

Thanks for your reply

Manuel

Jun 19 '06 #3

This discussion thread is closed

Replies have been disabled for this discussion.

Similar topics

5 posts views Thread by GIMME | last post: by
18 posts views Thread by Urs Vogel | last post: by
4 posts views Thread by ad | last post: by
11 posts views Thread by brother52 | last post: by
1 post views Thread by Larry Bates | last post: by
1 post views Thread by Solo | last post: by
By using this site, you agree to our Privacy Policy and Terms of Use.