DB -> Flat file, did I waste my time?

[FFMG]

In my site I have a config table, (MySQL), with about 30 entries; the
data is loaded on every single page load. This is not the only call to
the db, (we do a total of about 8 calls to the db).
As with many configurations settings, once the options are set the
values will not change much.

So I thought it would be a good idea to move the data to a flat file.
I have written a small wrapper class to first look for the file, if it
exists then load the data from that file, if not the data is loaded
from the database and then saved to a file.
That way I can copy the database without worries about the cached data
itself.

It all works as expected, but the question is, did I waste my time?
How can I do some load test on my server to see what would be better?
Database call or flat file read.

My personal guess is that the flat file will only help under very heavy
load, (a couple of hundreds/thousand pages a seconds).

What do you think?
Will the flat file help at all?

Simon

[Christoph Burschka]

FFMG schrieb:

In my site I have a config table, (MySQL), with about 30 entries; the
data is loaded on every single page load. This is not the only call to
the db, (we do a total of about 8 calls to the db).
As with many configurations settings, once the options are set the
values will not change much.

So I thought it would be a good idea to move the data to a flat file.
I have written a small wrapper class to first look for the file, if it
exists then load the data from that file, if not the data is loaded
from the database and then saved to a file.
That way I can copy the database without worries about the cached data
itself.

It all works as expected, but the question is, did I waste my time?
How can I do some load test on my server to see what would be better?
Database call or flat file read.

My personal guess is that the flat file will only help under very heavy
load, (a couple of hundreds/thousand pages a seconds).

What do you think?
Will the flat file help at all?

Simon

I would guess that the relational database is better for a great amount
of data that needs to be sorted and changed regularly. The flat file is
better for a small amount of data that gets read extremely often, always
in the same order, but never written.

Almost all PHP/MySQL CMS (eg Drupal and MediaWiki) have their basic
configuration settings in a file - so it probably isn't a bad idea.

--
CB

[P Pulkkinen]

Flat file is very common for settings, but database is popular enough too. I
think both are good solutions. I have used both.

You did brain-stretching when writing the wrapper, and exercise is always
good. It is waste of time to even think anymore, if you wasted your time or
not.

So, now stop thinking that and continue to do something else. Remember to be
happy.

[Jim Carlock]

"FFMG" <sp********@myoddweb.comwrote:
: In my site I have a config table, (MySQL), with about 30 entries;
: the data is loaded on every single page load. This is not the only
: call to the db, (we do a total of about 8 calls to the db).
:
: As with many configurations settings, once the options are set
: the values will not change much.

Use an include file that holds a function that returns the data inside
an array. This works well for textual and numeric data.

<?php
require("data.php")
?>

<data.php>
function GetMyArray() {
return(array("data1", "data2", 1, 2, 3, 4));
}
</data.php>

Good luck!

--
Jim Carlock
Post replies to the group.

[FFMG]

"FFMG" <[e-mail]:: In my site I have a config table, (MySQL), with about 30 entries;

: the data is loaded on every single page load. This is not the only
: call to the db, (we do a total of about 8 calls to the db).
:
: As with many configurations settings, once the options are set
: the values will not change much.

Use an include file that holds a function that returns the data inside
an array. This works well for textual and numeric data.

<?php
require("data.php")
?>

<data.php>
function GetMyArray() {
return(array("data1", "data2", 1, 2, 3, 4));}</data.php>

Sorry I am not sure I follow, what is that code for?

How does it answer my OP?

Simon

[Krustov]

<comp.lang.php>
<FFMG>
<23 Jan 2007 22:51:23 -0800>
<11*********************@s48g2000cws.googlegroups. com>

It all works as expected, but the question is, did I waste my time?
How can I do some load test on my server to see what would be better?
Database call or flat file read.

My personal guess is that the flat file will only help under very heavy
load, (a couple of hundreds/thousand pages a seconds).

What do you think?
Will the flat file help at all?

I've done the script below using flat files .

How its going to perform under a heavy load is anybodys guess - but i
think theres too much worring about heavy loads as most websites will
never have hundreds or thousands of users at the same time .

Most websites arnt google / microsoft / youtube / etc etc .
--
www.phptakeaway.co.uk
(work in progress)

[Norman Peelman]

FFMG wrote:

In my site I have a config table, (MySQL), with about 30 entries; the
data is loaded on every single page load. This is not the only call to
the db, (we do a total of about 8 calls to the db).
As with many configurations settings, once the options are set the
values will not change much.

So I thought it would be a good idea to move the data to a flat file.
I have written a small wrapper class to first look for the file, if it
exists then load the data from that file, if not the data is loaded
from the database and then saved to a file.
That way I can copy the database without worries about the cached data
itself.

It all works as expected, but the question is, did I waste my time?
How can I do some load test on my server to see what would be better?
Database call or flat file read.

My personal guess is that the flat file will only help under very heavy
load, (a couple of hundreds/thousand pages a seconds).

What do you think?
Will the flat file help at all?

Simon

FFMG,

MySQL will cache queries and not go to the db if it doesn't need
to... in this case, if keeping the config data in the db is convienent
for you then leave it there. Make sure you have the following MySQL
system variables set:

query_cache_size #bytes (ei: query_cache_size 5242880, that's 5MB)
query_cache_type ? (ei: query_cache_size 1, 0 = OFF / 1 = ON / 2 = DEMAND)

So really what's happening is either MySQL will cache the result set
or your web server will cache the flatfile. The only way to test would
be to write a small script that timed both the database call and the
flatfile call.

You can get this info from MySQL Administrator or command line, or
set them in my.ini, etc.

Norm

[Jim Carlock]

"FFMG" <[e-mail]:
: In my site I have a config table, (MySQL), with about 30 entries;
: the data is loaded on every single page load. This is not the only
: call to the db, (we do a total of about 8 calls to the db).
:
: As with many configurations settings, once the options are set
: the values will not change much.

Jim Carlock replied...
:: Use an include file that holds a function that returns the data inside
:: an array. This works well for textual and numeric data.
::
:: <?php
:: require("data.php")
:: ?>
::
:: <data.php>
:: function GetMyArray() { return(array("data1", "data2", 1, 2, 3, 4)); }
:: </data.php>
::

"FFMG" asked...
: Sorry I am not sure I follow, what is that code for?
:
: How does it answer my OP?

A data file holds information. An array typically gets created to
hold text and numbers and it loads super fast. You mentioned
30(?) items, and that seems trivial for the most part (unless each item
ends up of massive size). My response presented an alternative
way to look at it the problem. You stated you tried the the "flat file"
alternative.

If it's the programmer that only does the updates, it's rather easy to
update the included file's array contents. You could also write that
array to disk (I'm thinking along the lines of text and numbers as
opposed to binary data... I haven't messed with trying to put binary
contents into an array).

It sounded like you have a common file that gets loaded up by a lot
of pages, that doesn't get updated often.

Under those conditions, using require() or include() seems like a
natural choice (it requires much more thought though if you've
got multiple users that update things concurrently, so in that case
a DBMS is probably the best thing to stick with).

The question you asked, "Flat file, did I waste my time?" comes
across as one of those goofy subjective questions, like "Did I
waste my time getting married?" It's a rather personal question
that only you can answer yourself. I can not argue that one way
or the other, it's 100% your question! And either YOU answer it,
"I wasted my time!", or you state, "I learned something, so while
I spent some time on it, all is well and I'll get over it". Those may
or may not be the only answers (some folks create really goofy
answers to such ponderings) and noone but you can answer that
question.

It's not a good idea to ask anyone else if "YOU" wasted your
time. How can anyone else know without you specifically telling
them that you wasted your time?

--
Jim Carlock
Post replies to the group.

[FFMG]

>

"FFMG" asked...
: Sorry I am not sure I follow, what is that code for?
:
: How does it answer my OP?

A data file holds information. An array typically gets created to
hold text and numbers and it loads super fast. You mentioned
30(?) items, and that seems trivial for the most part (unless each item
ends up of massive size). My response presented an alternative
way to look at it the problem. You stated you tried the the "flat file"
alternative.

Sorry, your reply did not say that.
In fact when you said, "Use an include file that holds a function that
returns the data inside an array. This works well for textual and
numeric data."

I did not know you meant that this was another approach.
Seen that I was not really asking for another approach but to rather
compare my approach with using the database I might be forgiven for
asking what you were referring to.

>
If it's the programmer that only does the updates,

If that was the case then I would use define(...) or even hardcode the
values in the code.
Using an array is not practical in a 'config' situation.
And using the array the way you propose makes it unusable, (unless you
know that item 0 = data1 and so on).

It sounded like you have a common file that gets loaded up by a lot
of pages, that doesn't get updated often.

Yes, that is what I said, but it does get updated.

Under those conditions, using require() or include() seems like a
natural choice

No it is not. You option makes it very hard to update, if at all.
It is not flexible and does not make the code very portable.

In my case, (as mentioned), it does get updated from time to time.
Updating code and then ftp the contents from time to time is just not
practical in most cases.

It means that no values can be updated by anyone else than the
programmer.
It is indeed a solution, but a bit of a drastic one.

>
The question you asked, "Flat file, did I waste my time?" comes
across as one of those goofy subjective questions, like "Did I
waste my time getting married?"

Really? Was my post the same as asking about marriage?
If you had read my OP, (because it is quite clear that you did not read
it until I asked you what you where taking about), you would have
understood that I was asking about comparing reading a simple file and
reading a simple table.

I said that I wrote a wrapper, (BTW that implies that I am familiar
with such things as 'arrays()', 'include()' and 'require()'), my
question invited a discussion on flat files vs table, (both simple).
I even go further, (to make it clear to those who actually read the
post), and ask if reading a flat file is as fast as reading a table.
So, did I waste my time doing that wrapper?

I did not ask you for 3 lines of code and a 'good luck', good luck for
what?

I can not argue that one way
or the other, it's 100% your question! And either YOU answer it,
"I wasted my time!", or you state, "I learned something, so while
I spent some time on it, all is well and I'll get over it". Those may
or may not be the only answers (some folks create really goofy
answers to such ponderings) and noone but you can answer that
question.

No, a programmer would have said, "You wasted your time because reading
a flat file is slower than reading a table" or "you did not waste your
time because reading a flat file will free some resources on your
database".

Come one, you know exactly what I was asking.
You can throw insults around as much as you like, but the fact remains
that you did not read my post properly and now you try and make it as
if you knew all along what you were taking about.
You then tried to increase your post count by adding a totally useless
post.

I mean,

function GetMyArray() {
return(array("data1", "data2", 1, 2, 3, 4));
}

Did you really think that it was:
1) Answering my post?
2) A practical answer, (given my OP)?

>
It's not a good idea to ask anyone else if "YOU" wasted your
time. How can anyone else know without you specifically telling
them that you wasted your time?

It might be better to only answer posts properly rather than wasting
both our times.

Regards,

Oh, and 'good luck'.

Simon

[Norman Peelman]

FFMG wrote:

In my site I have a config table, (MySQL), with about 30 entries; the
data is loaded on every single page load. This is not the only call to
the db, (we do a total of about 8 calls to the db).
As with many configurations settings, once the options are set the
values will not change much.

So I thought it would be a good idea to move the data to a flat file.
I have written a small wrapper class to first look for the file, if it
exists then load the data from that file, if not the data is loaded
from the database and then saved to a file.
That way I can copy the database without worries about the cached data
itself.

It all works as expected, but the question is, did I waste my time?
How can I do some load test on my server to see what would be better?
Database call or flat file read.

My personal guess is that the flat file will only help under very heavy
load, (a couple of hundreds/thousand pages a seconds).

What do you think?
Will the flat file help at all?

Simon

And in addition to my previous post... you can include your config
table in your regular backups and it's one less thing to worry about if
you happen to lose anything.

Norm