Securely serving files

[Nicholas Sherlock]

Hey all,

People on my website register to be allowed access to certain downloads.
I store these files above the document root so that they can't be
accessed by Apache (Only from PHP). I wrote a file serving script which
dumps the correct headers for the download and calls readfile().

I thought everything was going swimmingly until I realized that my
webhost has Safe Mode turned on, and the script execution time is
limited to 30 seconds. If your file takes longer than that to download,
then too bad :).

So how can I do this? The files that I am serving are no larger than
about 2 megabytes.

It seems that one option would be to make a copy of the file I want to
serve, but below the document root so that it is available from the
browser. I'd give the file a randomly generated name that couldn't just
be guessed, and just write a regular link to the browser. Has anyone
tried this, and did it work out okay for you? How did you go about
cleaning up files after the client has downloaded them? Is there a
better option?

Cheers,
Nicholas Sherlock

[Jerry Stuckle]

Nicholas Sherlock wrote:

Hey all,

People on my website register to be allowed access to certain downloads.
I store these files above the document root so that they can't be
accessed by Apache (Only from PHP). I wrote a file serving script which
dumps the correct headers for the download and calls readfile().

I thought everything was going swimmingly until I realized that my
webhost has Safe Mode turned on, and the script execution time is
limited to 30 seconds. If your file takes longer than that to download,
then too bad :).

So how can I do this? The files that I am serving are no larger than
about 2 megabytes.

It seems that one option would be to make a copy of the file I want to
serve, but below the document root so that it is available from the
browser. I'd give the file a randomly generated name that couldn't just
be guessed, and just write a regular link to the browser. Has anyone
tried this, and did it work out okay for you? How did you go about
cleaning up files after the client has downloaded them? Is there a
better option?

Cheers,
Nicholas Sherlock

Why not just include() the file?

--
==================
Remove the "x" from my email address
Jerry Stuckle
JDS Computer Training Corp.
js*******@attglobal.net
==================

[Dra Jenic]

Yes actually I've done that before. It works just fine, as for cleaning
the files up afterward you could just use crontab. Assuming your using
Linux. Windows probably has something similar.
Nicholas Sherlock wrote:

Hey all,

People on my website register to be allowed access to certain downloads.
I store these files above the document root so that they can't be
accessed by Apache (Only from PHP). I wrote a file serving script which
dumps the correct headers for the download and calls readfile().

I thought everything was going swimmingly until I realized that my
webhost has Safe Mode turned on, and the script execution time is
limited to 30 seconds. If your file takes longer than that to download,
then too bad :).

So how can I do this? The files that I am serving are no larger than
about 2 megabytes.

It seems that one option would be to make a copy of the file I want to
serve, but below the document root so that it is available from the
browser. I'd give the file a randomly generated name that couldn't just
be guessed, and just write a regular link to the browser. Has anyone
tried this, and did it work out okay for you? How did you go about
cleaning up files after the client has downloaded them? Is there a
better option?

Cheers,
Nicholas Sherlock

[Jasen Betts]

On 2006-02-11, Jerry Stuckle <js*******@attglobal.net> wrote:

Why not just include() the file?

not reccomended for binary files - they might have <? in them somewhere...

--

Bye.
Jasen

[Jasen Betts]

On 2006-02-11, Nicholas Sherlock <N.********@gmail.com> wrote:

Hey all,

People on my website register to be allowed access to certain downloads.
I store these files above the document root so that they can't be
accessed by Apache (Only from PHP). I wrote a file serving script which
dumps the correct headers for the download and calls readfile().

I thought everything was going swimmingly until I realized that my
webhost has Safe Mode turned on, and the script execution time is
limited to 30 seconds. If your file takes longer than that to download,
then too bad :).

So how can I do this? The files that I am serving are no larger than
about 2 megabytes.

It seems that one option would be to make a copy of the file I want to
serve, but below the document root so that it is available from the
browser.
if it's a *nix server make a link instread of a copy of the file it's much
faster, and uses much less disk space.
I'd give the file a randomly generated name that couldn't just
be guessed, and just write a regular link to the browser. Has anyone
tried this, and did it work out okay for you? How did you go about
cleaning up files after the client has downloaded them? Is there a
better option?

Store the names and delete any that have been sitting there for more than 30
seconds, or spawn a background script that sleeps for 30 seconds and then
deletes the copy.

With unix the download will continue even after the file is no longer present
windows will ( I think) give an error and not delete the file so you'd have
to find a way to detect that, sleep again, and retry.

Bye.
Jasen

[Jerry Stuckle]

Jasen Betts wrote:

On 2006-02-11, Jerry Stuckle <js*******@attglobal.net> wrote:

Why not just include() the file?

not reccomended for binary files - they might have <? in them somewhere...

He didn't say the were binary files.

And even if they were, he'd have exactly the same problem with
readfile() - which he is currently using.

--
==================
Remove the "x" from my email address
Jerry Stuckle
JDS Computer Training Corp.
js*******@attglobal.net
==================

[Nicholas Sherlock]

Jasen Betts wrote:

if it's a *nix server make a link instread of a copy of the file it's much
faster, and uses much less disk space.
The Symlink() command? Awesome!
Store the names and delete any that have been sitting there for more than 30
seconds, or spawn a background script that sleeps for 30 seconds and then
deletes the copy.

With unix the download will continue even after the file is no longer present
windows will ( I think) give an error and not delete the file so you'd have
to find a way to detect that, sleep again, and retry.

I'm on Unix. Thanks, this is an excellent solution.

Cheers,
Nicholas Sherlock

[Andy Hassall]

On Sat, 11 Feb 2006 16:56:08 -0500, Jerry Stuckle <js*******@attglobal.net>
wrote:

Jasen Betts wrote:

On 2006-02-11, Jerry Stuckle <js*******@attglobal.net> wrote:

Why not just include() the file?

not reccomended for binary files - they might have <? in them somewhere...

He didn't say the were binary files.

And even if they were, he'd have exactly the same problem with
readfile() - which he is currently using.

No, readfile() doesn't execute the file as PHP. include() does. They're very
different..

--
Andy Hassall :: an**@andyh.co.uk :: http://www.andyh.co.uk
http://www.andyhsoftware.co.uk/space :: disk and FTP usage analysis tool

[Jerry Stuckle]

Andy Hassall wrote:

On Sat, 11 Feb 2006 16:56:08 -0500, Jerry Stuckle <js*******@attglobal.net>
wrote:

Jasen Betts wrote:

On 2006-02-11, Jerry Stuckle <js*******@attglobal.net> wrote:
Why not just include() the file?

not reccomended for binary files - they might have <? in them somewhere...

He didn't say the were binary files.

And even if they were, he'd have exactly the same problem with
readfile() - which he is currently using.

No, readfile() doesn't execute the file as PHP. include() does. They're very
different..

Please read the original post. He is not outputing a php file. It is
text - and would not be executed.
--
==================
Remove the "x" from my email address
Jerry Stuckle
JDS Computer Training Corp.
js*******@attglobal.net
==================

[Jasen Betts]

On 2006-02-12, Jerry Stuckle <js*******@attglobal.net> wrote:

Andy Hassall wrote:

On Sat, 11 Feb 2006 16:56:08 -0500, Jerry Stuckle <js*******@attglobal.net>
wrote:

Jasen Betts wrote:

On 2006-02-11, Jerry Stuckle <js*******@attglobal.net> wrote:
>Why not just include() the file?

not reccomended for binary files - they might have <? in them somewhere...

He didn't say the were binary files.

And even if they were, he'd have exactly the same problem with
readfile() - which he is currently using.

No, readfile() doesn't execute the file as PHP. include() does. They're very
different..

Please read the original post. He is not outputing a php file. It is
text - and would not be executed.

php files are (usually) text files...
With include() it would be executed if it contained php tags.
The .php filename extension is only significant to the web server.


--

Bye.
Jasen

[Jasen Betts]

On 2006-02-12, Nicholas Sherlock <N.********@gmail.com> wrote:

Jasen Betts wrote:

if it's a *nix server make a link instread of a copy of the file it's much
faster, and uses much less disk space.

The Symlink() command? Awesome!

symlinks depend on the apache configuration.

hard links (the link() function) are indistinguishable from
real files.

as long as the file you're linking to is on the same partition as the web
site (and is readable by PHP - you already said it was) link() is probably
the best way to go.

Store the names and delete any that have been sitting there for more than 30
seconds, or spawn a background script that sleeps for 30 seconds and then
deletes the copy.

shell_exec (" ( sleep 30 ; rm '$filename' ) &");

but that doesn't work in safe mode... :( someone else may have an
alternative.
Bye.
Jasen

[Nicholas Sherlock]

Jasen Betts wrote:

On 2006-02-12, Nicholas Sherlock <N.********@gmail.com> wrote:

The Symlink() command? Awesome!
symlinks depend on the apache configuration.

hard links (the link() function) are indistinguishable from
real files.

as long as the file you're linking to is on the same partition as the web
site (and is readable by PHP - you already said it was) link() is probably
the best way to go.

I'm using Symlink() right now with success. What is the difference
between hard links and symlinks? Is there any reason to switch?

Store the names and delete any that have been sitting there for more than 30
seconds, or spawn a background script that sleeps for 30 seconds and then
deletes the copy.

shell_exec (" ( sleep 30 ; rm '$filename' ) &");

but that doesn't work in safe mode... :( someone else may have an
alternative.

Since it isn't that critical when the link expires, I just delete all
previous links every time I create a link. Hopefully there won't be any
concurrency issues there.

In safe mode, the folders that you create with mkdir get Apache's ID,
not the ID of the script, and are therefore not accessible from PHP
scripts. The solution was to use the ftp_* functions of PHP to create a
folder, which then has the correct ID for access.

Cheers,
Nicholas Sherlock

Cheers,
Nicholas Sherlock

[Richard Levasseur]

Alternatively, you can use chmod() to change the file attributes to
allow reading/writing of the file. Might be faster than the ftp
functions.

Hard links point to the data (ie: they're just a normal file). When
all the hard links to the data are gone, then the data is deleted.
They also cannot cross volumes.

Symlinks point to a file (which points to the data). The effect being,
if you delete the symlink, the original file is still there. They can
cross volumes as they just have a file path they point to.

Apache can be setup to follow, not follow, and change the owner id of
symlinks. Since you said symlinks are apparently working, I don't
think it matters which you choose in this case.

[Nicholas Sherlock]

Richard Levasseur wrote:

Alternatively, you can use chmod() to change the file attributes to
allow reading/writing of the file. Might be faster than the ftp
functions.
AFAICS, chmod() isn't the problem. The problem is that when using
Mkdir() the owner of the created folder isn't the same as the owner of
the script, and PHP's safe mode therefore doesn't allow you to access
the created folder (You can't even call Chmod() on it). Creating it
through FTP solves this problem.
Apache can be setup to follow, not follow, and change the owner id of
symlinks. Since you said symlinks are apparently working, I don't
think it matters which you choose in this case.

Thanks.

Cheers,
Nicholas Sherlock

[Jerry Stuckle]

Jasen Betts wrote:

On 2006-02-12, Jerry Stuckle <js*******@attglobal.net> wrote:

Andy Hassall wrote:

On Sat, 11 Feb 2006 16:56:08 -0500, Jerry Stuckle <js*******@attglobal.net>
wrote:

Jasen Betts wrote:
>On 2006-02-11, Jerry Stuckle <js*******@attglobal.net> wrote:
>
>
>
>>Why not just include() the file?
>
>not reccomended for binary files - they might have <? in them somewhere...

He didn't say the were binary files.

And even if they were, he'd have exactly the same problem with
readfile() - which he is currently using.
No, readfile() doesn't execute the file as PHP. include() does. They're very
different..

Please read the original post. He is not outputing a php file. It is
text - and would not be executed.

php files are (usually) text files...
With include() it would be executed if it contained php tags.
The .php filename extension is only significant to the web server.

That's true. But this is NOT php code. This is plain text.

Get me a newspaper and show me where articles have tags such as <?php in
them. Or any other plain text file not about PHP.

--
==================
Remove the "x" from my email address
Jerry Stuckle
JDS Computer Training Corp.
js*******@attglobal.net
==================

[Jasen Betts]

On 2006-02-13, Jerry Stuckle <js*******@attglobal.net> wrote:

Jasen Betts wrote:

php files are (usually) text files...
With include() it would be executed if it contained php tags.
The .php filename extension is only significant to the web server.

That's true. But this is NOT php code. This is plain text.

Get me a newspaper and show me where articles have tags such as <?php in
them. Or any other plain text file not about PHP.

If you can guarantee that the files will never contain
and you (or other resposible people control the content <?php (etc) that'd
work... but it still seems like a problem waiting to happen to me.

Bye.
Jasen

[Andy Hassall]

On Sat, 11 Feb 2006 20:13:59 -0500, Jerry Stuckle <js*******@attglobal.net>
wrote:

Andy Hassall wrote:

On Sat, 11 Feb 2006 16:56:08 -0500, Jerry Stuckle <js*******@attglobal.net>
wrote:

Jasen Betts wrote:

On 2006-02-11, Jerry Stuckle <js*******@attglobal.net> wrote:
>Why not just include() the file?

not reccomended for binary files - they might have <? in them somewhere...

He didn't say the were binary files.

And even if they were, he'd have exactly the same problem with
readfile() - which he is currently using.

No, readfile() doesn't execute the file as PHP. include() does. They're very
different..

Please read the original post. He is not outputing a php file. It is
text - and would not be executed.

PHP code _is_ text.

What advantages does include() have over readfile() that makes you suggest it?

It's dangerous and doesn't solve the problem in the original post (which
doesn't actually say anything about what's in the files, or even whether
they're "text" or "binary").

--
Andy Hassall :: an**@andyh.co.uk :: http://www.andyh.co.uk
http://www.andyhsoftware.co.uk/space :: disk and FTP usage analysis tool

[Jerry Stuckle]

Andy Hassall wrote:

On Sat, 11 Feb 2006 20:13:59 -0500, Jerry Stuckle <js*******@attglobal.net>
wrote:

Andy Hassall wrote:

On Sat, 11 Feb 2006 16:56:08 -0500, Jerry Stuckle <js*******@attglobal.net>
wrote:

Jasen Betts wrote:
>On 2006-02-11, Jerry Stuckle <js*******@attglobal.net> wrote:
>
>
>
>>Why not just include() the file?
>
>not reccomended for binary files - they might have <? in them somewhere...

He didn't say the were binary files.

And even if they were, he'd have exactly the same problem with
readfile() - which he is currently using.

No, readfile() doesn't execute the file as PHP. include() does. They're very
different..
Please read the original post. He is not outputing a php file. It is
text - and would not be executed.

PHP code _is_ text.

What advantages does include() have over readfile() that makes you suggest it?

It solves the problem he's having - timeout of the PHP script when
downloading large files.
It's dangerous and doesn't solve the problem in the original post (which
doesn't actually say anything about what's in the files, or even whether
they're "text" or "binary").

Sure it does. And the original post intimated they are text files -
because they are serverd with readfile().

--
==================
Remove the "x" from my email address
Jerry Stuckle
JDS Computer Training Corp.
js*******@attglobal.net
==================

[Jasen Betts]

On 2006-02-14, Jerry Stuckle <js*******@attglobal.net> wrote:

Sure it does. And the original post intimated they are text files -
because they are serverd with readfile().

text/binary makes no difference on unix systems.

<?php
header("Content-type: image/gif");
readfile("example.gif");
?>

Bye.
Jasen

[Jerry Stuckle]

Jasen Betts wrote:

On 2006-02-14, Jerry Stuckle <js*******@attglobal.net> wrote:

Sure it does. And the original post intimated they are text files -
because they are serverd with readfile().

text/binary makes no difference on unix systems.

<?php
header("Content-type: image/gif");
readfile("example.gif");
?>

Bye.
Jasen

Yes, I know there's no difference between text and binary on Linux.
I've probably been using Unix a lot longer than you - and longer than a
lot of people here have been alive.

However - again, he didn't say he was outputting gif's, jpegs or
whatever. Nothing at all about it.

--
==================
Remove the "x" from my email address
Jerry Stuckle
JDS Computer Training Corp.
js*******@attglobal.net
==================

[Richard Levasseur]

Please note that include() does not reset or otherwise prevent the
limit on script execution from being enforced (other things can: system
calls, sleep, etc, but include itself is not one of them).

Aside: Including and executing unknown data would open up a potentially
serious security vulnerability. That would be a Bad Thing (tm).

Additional Aside: include() is for including php code, using it in any
other manner would be bad form.

[Nicholas Sherlock]

Richard Levasseur wrote:

Please note that include() does not reset or otherwise prevent the
limit on script execution from being enforced (other things can: system
calls, sleep, etc, but include itself is not one of them).

Aside: Including and executing unknown data would open up a potentially
serious security vulnerability. That would be a Bad Thing (tm).

To be clear here, I'm serving user uploaded binary files, so Include()
would indeed be the kiss of death for my whole website.

Cheers,
Nicholas Sherlock