[itamaram]

So I have a web server which gets a request for some file, and if the file does not exist, it downloads it from a remote server, and then serve it ie:

Expand|Select|Wrap|Line Numbers

1. def do_GET(self):
2.     try:
3.         if not os.path.exists(self.localpath + self.path):#no such file
4.             if not os.path.exists(self.localpath + self.path[:self.path.rfind('/') + 1]):#no such directory
5.                 os.makedirs(self.path[:self.path.rfind('/') + 1])
6.  

The main problem with this (which I wasn't expecting) occurs when several requests for resources in the same non-existing folder arrive simultanously, which means several folder creation commands for the same folder, which makes my system unhappy. To be exact, I get the following error code:
WindowsError: [Error 183] Cannot create a file when that file already exists

I am assuming putting some sort of a lock before creating a folder is the way to go, but I have no idea where to start.

Any tips/pointers/full code solutions would be greatly appreciated.

Thanks,
Itamar

[edwardrsmith]

Use a global variable or add error handling so that your program deals with the error cleanly.

Edward

[itamaram]

Originally Posted by edwardrsmith

Use a global variable or add error handling so that your program deals with the error cleanly.

Edward

Would you care to elaborate?
I have error handling (hence the 'try' in my code snippet), but it is highly ineffective due to the fact that if 2 threads will try creating the same folder, both will throw an exception, and none will create the folder. However, it is very likely I am doing it wrong.

Global variables is something I haven't tried. Won't the multi-threading make this problematic too?

[edwardrsmith]

I don't know how but it is possible to make variables outside of the program (i.e. stored in system memory). It should also be possible to store the variable in the memory belonging to the program which creates the threads.

As for error handling, I don't see an except statement and all threads but one should throw an error. In order for the error to be thrown the folder should have to have been created by one thread after the others checked but before they attempted to create the folder.

Edward

[itamaram]

Originally Posted by edwardrsmith

I don't know how but it is possible to make variables outside of the program (i.e. stored in system memory). It should also be possible to store the variable in the memory belonging to the program which creates the threads.

Currently trying to do that. will keep you updated.

Originally Posted by edwardrsmith

... and all threads but one should throw an error...

Thats the problem, this does not happen. All threads are trying to create the same folder at once, and then crash half way through, so no folder is created. I can not fully explain why, but it is something in the system level.

[itamaram]

Gah! Found my mistake.
My call for makedirs only gave a local path, not a full path (for which I was checking existence).

So I've created folders on my root, and checked if they existed somewhere completely different. Sigh.

On the up side, it only took me 2 days to notice...