Mod_python

[Lad]

In my web application I use Apache and mod_python.
I allow users to upload huge files( via HTTP FORM , using POST method)
I would like to store the file directly on a hard disk and not to
upload the WHOLE huge file into server's memory first.
Can anyone suggest a solution?
Thank you
LB

[Maxim Sloyko]

Lad wrote:

In my web application I use Apache and mod_python.
I allow users to upload huge files( via HTTP FORM , using POST method)
I would like to store the file directly on a hard disk and not to
upload the WHOLE huge file into server's memory first.
Can anyone suggest a solution?

The only solution you need is Apache and mod_python :)
I mean, Apache won't load a huge POST request into its memory no
matter what. All file uploads will be stored in temporary files. Under
mod_python (provided you use FieldStorage) you'll need to deal only
with 'file' objects.

--
Maxim Sloyko

[Lad]

Maxim Sloyko wrote:

Lad wrote:

In my web application I use Apache and mod_python.
I allow users to upload huge files( via HTTP FORM , using POST method)
I would like to store the file directly on a hard disk and not to
upload the WHOLE huge file into server's memory first.
Can anyone suggest a solution?

The only solution you need is Apache and mod_python :)
I mean, Apache won't load a huge POST request into its memory no
matter what. All file uploads will be stored in temporary files. Under
mod_python (provided you use FieldStorage) you'll need to deal only
with 'file' objects.

Maxim ,
Thank you for your reply.
Here is an example:
If a user uploads 100MB file ,
what will be a consumption of memory, when the upload is complete?

Or does it mean that Apache will read a part of a file , store it in a
temporary file, then read another part and adds this part to the
temporary file and so on until the whole uploaded file is read?
Thank you for the reply.
Lad

[J. Clifford Dyer]

Lad wrote:

Maxim Sloyko wrote:

>Lad wrote:

>>In my web application I use Apache and mod_python.
I allow users to upload huge files( via HTTP FORM , using POST method)
I would like to store the file directly on a hard disk and not to
upload the WHOLE huge file into server's memory first.
Can anyone suggest a solution?

The only solution you need is Apache and mod_python :)
I mean, Apache won't load a huge POST request into its memory no
matter what. All file uploads will be stored in temporary files. Under
mod_python (provided you use FieldStorage) you'll need to deal only
with 'file' objects.

Maxim ,
Thank you for your reply.
Here is an example:
If a user uploads 100MB file ,
what will be a consumption of memory, when the upload is complete?

Or does it mean that Apache will read a part of a file , store it in a
temporary file, then read another part and adds this part to the
temporary file and so on until the whole uploaded file is read?
Thank you for the reply.
Lad

Are you working on *nix? Try this from the shell on your server:

$ top

Then upload a 100MB file and watch the memory usage.

On Windows, I think you can do similar from Ctrl-Alt-Del, but
graphically. Correct me if I'm wrong.

Cheers,
Cliff

[Graham Dumpleton]

Maxim Sloyko wrote:

Lad wrote:

In my web application I use Apache and mod_python.
I allow users to upload huge files( via HTTP FORM , using POST method)
I would like to store the file directly on a hard disk and not to
upload the WHOLE huge file into server's memory first.
Can anyone suggest a solution?

The only solution you need is Apache and mod_python :)
I mean, Apache won't load a huge POST request into its memory no
matter what. All file uploads will be stored in temporary files. Under
mod_python (provided you use FieldStorage) you'll need to deal only
with 'file' objects.

Note though that the default behaviour of FieldStorage is to store
files in whatever directory the 'tempfile' module uses. This may not
always be acceptable, especially with very large files, as the user may
want to have the files actually reside elsewhere in some special
directory and having to make a copy of the file, if a different file
system, may in itself cause issues.

Thus, provided you are using a recent version of mod_python, you still
may have to customise how files are handled using the FieldStorage file
callbacks to have the file stored in the desired location from the
outside.

The other alternative to FieldStorage is to write a custom
Apache/mod_python input filter to process uploads. An example of this
is Tramline (http://www.infrae.com/products/tramline).

Graham