By using this site, you agree to our updated Privacy Policy and our Terms of Use. Manage your Cookies Settings.
440,440 Members | 1,872 Online
Bytes IT Community
+ Ask a Question
Need help? Post your question and get tips & solutions from a community of 440,440 IT Pros & Developers. It's quick & easy.

copyfile avoiding overwrites and race conditions

P: n/a
Here is a code fragment, where I am trying to copy a file, avoiding
overwrites and race conditions. The filename gets a '02','03','04' etc
appended to the end if a file with that name already exists.

I know the writing of the single space is overkill, but I am surprised
I cannot find an example of this floating around the web and newsgroups

my understanding of 'os.open' and 'os.fdopen' is minimal

## start fragment

i = 0
while True:

if i >= 100: raise RanOutFileNamesError(fullpath)
i += 1

if i > 1:
name0 = '%s_%02i%s' % (suggested_name, i, file_extension)
else:
assert i == 1
name0 = '%s%s' % (suggested_name, file_extension)
fullpath = os.path.join(path0, name0)

# create dummy file, force overwrite

try:
fd = os.open(
fullpath,
os.O_CREAT | os.O_EXCL | os.O_WRONLY)
except OSError:
continue

# is this really necessary?
file = os.fdopen(fd, "w")
file.write(' ')
file.close()

shutil.copyfile(original_filepath, fullpath)

return fullpath

## end fragment

Jun 16 '06 #1
Share this Question
Share on Google+
2 Replies


P: n/a
ma*****@gmail.com wrote:
Here is a code fragment, where I am trying to copy a file, avoiding
overwrites and race conditions. The filename gets a '02','03','04' etc
appended to the end if a file with that name already exists.

I know the writing of the single space is overkill, but I am surprised
I cannot find an example of this floating around the web and newsgroups

my understanding of 'os.open' and 'os.fdopen' is minimal

## start fragment

i = 0
while True:

if i >= 100: raise RanOutFileNamesError(fullpath)
i += 1

if i > 1:
name0 = '%s_%02i%s' % (suggested_name, i, file_extension)
else:
assert i == 1
name0 = '%s%s' % (suggested_name, file_extension)
fullpath = os.path.join(path0, name0)

# create dummy file, force overwrite

try:
fd = os.open(
fullpath,
os.O_CREAT | os.O_EXCL | os.O_WRONLY)
except OSError:
continue

# is this really necessary?
file = os.fdopen(fd, "w")
file.write(' ')
file.close()

shutil.copyfile(original_filepath, fullpath)

return fullpath

## end fragment

I guess my approach would be different. To eliminate any race
conditions, I would keep a small text file that always contained
the next filename that is to be written. Something like:

nextfiletowrite=/path/filename006.dat

I would try to get a lock on this file, read it, extract next
filename, increment the counter portion of the filename,
write it back out and unlock it. Now I have the name of the
file to write that is unique to my instance and I can write it
without worrying about other processes.

Hope this helps.

-Larry Bates
Jun 19 '06 #2

P: n/a
Larry Bates wrote:
I guess my approach would be different. To eliminate any race
conditions, I would keep a small text file that always contained
the next filename that is to be written. Something like:

nextfiletowrite=/path/filename006.dat

I would try to get a lock on this file, read it, extract next
filename, increment the counter portion of the filename,
write it back out and unlock it. Now I have the name of the
file to write that is unique to my instance and I can write it
without worrying about other processes.


Yes, that would work as well

but I get the feeling that

1) most people don't write to an automatically incrementing filename
2) the few who do don't sweat the race condition

Thanks for your reply

Manuel

Jun 19 '06 #3

This discussion thread is closed

Replies have been disabled for this discussion.