By using this site, you agree to our updated Privacy Policy and our Terms of Use. Manage your Cookies Settings.
459,518 Members | 980 Online
Bytes IT Community
+ Ask a Question
Need help? Post your question and get tips & solutions from a community of 459,518 IT Pros & Developers. It's quick & easy.

Where to log?

P: n/a
Ok, I have a nice iface, I have a (hopefully) good user authentication, I
have a DB up-and-running, and I have PHP 4.3.4 (yeah, I know, I will
update...).

What I need is a logging utility. And the question is: where to keep the
logs?

We (me and my buddy) have written everything from scratch (including
sessions support) - in a quest to learn something useful. Now we need to
decide where to keep our logs?

We can either keep them in a DB (already running, so no
setup/configuration needed here), or in a file.
Obviously, the file has an advantage over the DB - no overhead, less
queries. But here's the bad part: there will be ~500 users using this
site. It most certainly *will* happen that a few will use it
simultaneously.

So, we can flock() the file. Great. I can see the queue getting dreadfuly
long, the execution time getting dangerously near the limit and the
users getting mad on the other side of the cable.

Well, we don't have to flock() it. Fantastic - I can just imagine what
will happen if ~10 instaces of the script will try to write to this file
simultaneously...

Ok, we can can even write a "utility" script, that will get invoked by
the other scripts (with fopen(), not include()) and will do the actual
logging. But - isn't that just a other way for flock()? :)

Well, why don't we use the DB? Oh, right - the overhead...

Any ideas? I'm run out of mine...

Cheers
Mike

Jul 17 '05 #1
Share this Question
Share on Google+
8 Replies


P: n/a
On 2005-04-22, Micha³ Wo¼niak
<mikiwoz_remove_this@yahoo_remove_this.co.uk> wrote:
So, we can flock() the file. Great. I can see the queue getting
dreadfuly
long, the execution time getting dangerously near the limit and the
users getting mad on the other side of the cable.


If you only flock() it for the split second it actually is needed to
open and write to the log file you shouldn't see that kind of problem.
Even with 500 concurrent users (that is 500 people visiting your site at
the same time, not 500 concurrent requests) you should have plenty of
capacity to spare.

--
Cheers,
- Jacob Atzen
Jul 17 '05 #2

P: n/a
One quick glance of an experienced eye allowed to understand the blurred
and almost unreadable Jacob Atzen's handwriting:
On 2005-04-22, Micha? Wo?niak
<mikiwoz_remove_this@yahoo_remove_this.co.uk> wrote:
So, we can flock() the file. Great. I can see the queue getting
dreadfuly
long, the execution time getting dangerously near the limit and the
users getting mad on the other side of the cable.


If you only flock() it for the split second it actually is needed to
open and write to the log file you shouldn't see that kind of problem.
Even with 500 concurrent users (that is 500 people visiting your site
at the same time, not 500 concurrent requests) you should have plenty
of capacity to spare.


Still not convinced, as there will be multiple write aactions to the file
per request...

Cheers
Mike
Jul 17 '05 #3

P: n/a
One more idea: syslog().
The only problem is: I don't want to have those logs in
the /var/log/syslog file, I'd rather having them in /some/other/file -
is it possible? Couldn't find anything on this on php.net :/

TIA
Mike

Jul 17 '05 #4

P: n/a
=?ISO-8859-2?Q?Micha=B3_Wo=BCniak?= (mikiwoz_remove_this@yahoo_remove_this.co.uk) wrote:
: Ok, I have a nice iface, I have a (hopefully) good user authentication, I
: have a DB up-and-running, and I have PHP 4.3.4 (yeah, I know, I will
: update...).

: What I need is a logging utility. And the question is: where to keep the
: logs?

: We (me and my buddy) have written everything from scratch (including
: sessions support) - in a quest to learn something useful. Now we need to
: decide where to keep our logs?

: We can either keep them in a DB (already running, so no
: setup/configuration needed here), or in a file.
: Obviously, the file has an advantage over the DB - no overhead, less
: queries. But here's the bad part: there will be ~500 users using this
: site. It most certainly *will* happen that a few will use it
: simultaneously.

: So, we can flock() the file. Great. I can see the queue getting dreadfuly
: long, the execution time getting dangerously near the limit and the
: users getting mad on the other side of the cable.

: Well, we don't have to flock() it. Fantastic - I can just imagine what
: will happen if ~10 instaces of the script will try to write to this file
: simultaneously...

Assuming you are on Linux or Unix.

Open the log file for append, and write entire lines at a time, no locking
is required (note _append_ mode, that's the key).

Up to a large size, (something like 64,000 bytes typical) each such line
is guaranteed to be written as a single atomic unit to the end of the
file, no locking required. This is a documented feature of append mode on
unix, and is done exactly for this kind of situation.

(In practise you rarely even have to write whole lines at a time, because
the program's file write will normally buffer your output and write it one
line at a time even if you don't do that part explicitly your self.)

This is true for linux/unix, but may not be true for windows.
--

This space not for rent.
Jul 17 '05 #5

P: n/a
On 2005-04-22, Malcolm Dew-Jones <yf***@vtn1.victoria.tc.ca> wrote:
Up to a large size, (something like 64,000 bytes typical) each such
line is guaranteed to be written as a single atomic unit to the end of
the file, no locking required. This is a documented feature of append
mode on unix, and is done exactly for this kind of situation.


Where is this documented?

--
Cheers,
- Jacob Atzen
Jul 17 '05 #6

P: n/a
Jacob Atzen (ja***@aub.dk) wrote:
: On 2005-04-22, Malcolm Dew-Jones <yf***@vtn1.victoria.tc.ca> wrote:
: > Up to a large size, (something like 64,000 bytes typical) each such
: > line is guaranteed to be written as a single atomic unit to the end of
: > the file, no locking required. This is a documented feature of append
: > mode on unix, and is done exactly for this kind of situation.

: Where is this documented?

I have no idea anymore where this is _directly_ stated, but here is one
example location in the linux man pages where this capability is _implied_
by mentioning an exception to it in NFS (from
http://www.die.net/doc/linux/man/man2/open.2.html, found by a quick
google)

open(2) - Linux man page
NAME
open, creat - open and possibly create a file or device
SYNOPSIS
#include <sys/types.h>
#include <sys/stat.h>
#include <fcntl.h>
(snip)
O_APPEND

The file is opened in append mode. Before each write, the file
pointer is positioned at the end of the file, as if with lseek.
O_APPEND may lead to corrupted files on NFS file systems if more
than one process appends data to a file at once. This is because
NFS does not support appending to a file, so the client kernel has
to simulate it, which can't be done without a race condition.

Note the caveat that multi processing appending doesn't work properly on
NFS. They only mention this because it is an exception to the normal
state of affairs - where multi processing appending does not lead to
corrupted files.

I'm sure a unix or linux group would have someone who knows the document
where the exact details are mentioned.

--

This space not for rent.
Jul 17 '05 #7

P: n/a
One quick glance of an experienced eye allowed to understand the blurred
and almost unreadable Malcolm Dew-Jones's handwriting:
Up to a large size, (something like 64,000 bytes typical) each such
line is guaranteed to be written as a single atomic unit to the end
of
the file, no locking required. This is a documented feature of
append mode on unix, and is done exactly for this kind of
situation.


That would be great, if true. Doing a google search on this. :)

Thanks
Mike
Jul 17 '05 #8

P: n/a
Michal Wozniak wrote:
Ok, I have a nice iface, I have a (hopefully) good user authentication, I have a DB up-and-running, and I have PHP 4.3.4 (yeah, I know, I will
update...).

What I need is a logging utility. And the question is: where to keep the logs?

<snip>

What to log? Access?? If so, why not apache's log?

--
<?php echo 'Just another PHP saint'; ?>
Email: rrjanbiah-at-Y!com Blog: http://rajeshanbiah.blogspot.com/

Jul 17 '05 #9

This discussion thread is closed

Replies have been disabled for this discussion.