Database suggestion

[Mikhail Kovalev]

Hi.

I work with recursive array trees of 10-20 levels in PHP. So far I
have been using serialize() to store the arrays, generating files 5+
MB large, which take 10-15 seconds to unserialize and about tenfold of
RAM! I've been adviced to use MySQL (relational db's), but are there
any other options beside that and var_export/include? Something that
works in a similar way as MySQL when adding a new element without
loading the whole database itself...

Thanks!

[Toby A Inkster]

Mikhail Kovalev wrote:

I've been adviced to use MySQL (relational db's), but are there any
other options beside that and var_export/include? Something that works
in a similar way as MySQL when adding a new element without loading the
whole database itself...

SQLite, PostgreSQL, Oracle, Microsoft SQL Server, ...

--
Toby A Inkster BSc (Hons) ARCS
[Geek of HTML/SQL/Perl/PHP/Python/Apache/Linux]
[OS: Linux 2.6.17.14-mm-desktop-9mdvsmp, up 10 days, 18:31.]

Sharing Music with Apple iTunes
http://tobyinkster.co.uk/blog/2007/1...tunes-sharing/

[Mikhail Kovalev]

I mean, I don't mind using MySQL in itself, it's just that I don't
want use relational db's.
I don't know if it is possible to predefine the structures I'm using
in MySQL, because the number of levels varies etc.
Here is an example recording a single sequence of 6 elements:

How do I put this in a db?

array (
0 =>
array (
0 =2,
1 =>
array (
1 =>
array (
0 =1,
1 =>
array (
),
),
2 =>
array (
0 =1,
1 =>
array (
3 =>
array (
0 =1,
1 =>
array (
4 =>
array (
0 =1,
1 =>
array (
3 =>
array (
0 =1,
1 =>
array (
1 =>
array (
0 =1,
1 =>
array (
),
),
),
),
),
),
),
),
),
),
),
),
1 =>
array (
0 =2,
1 =>
array (
),
),
2 =>
array (
0 =1,
1 =>
array (
3 =>
array (
0 =1,
1 =>
array (
4 =>
array (
0 =1,
1 =>
array (
3 =>
array (
0 =1,
1 =>
array (
1 =>
array (
0 =1,
1 =>
array (
),
),
),
),
),
),
),
),
),
),
3 =>
array (
0 =2,
1 =>
array (
4 =>
array (
0 =1,
1 =>
array (
3 =>
array (
0 =1,
1 =>
array (
1 =>
array (
0 =1,
1 =>
array (
),
),
),
),
),
),
1 =>
array (
0 =1,
1 =>
array (
),
),
),
),
4 =>
array (
0 =1,
1 =>
array (
3 =>
array (
0 =1,
1 =>
array (
1 =>
array (
0 =1,
1 =>
array (
),
),
),
),
),
),
)


On 5 Des, 12:42, Toby A Inkster <usenet200...@t obyinkster.co.u k>
wrote:

Mikhail Kovalev wrote:

I've been adviced to use MySQL (relational db's), but are there any
other options beside that and var_export/include? Something that works
in a similar way as MySQL when adding a new element without loading the
whole database itself...

SQLite, PostgreSQL, Oracle, Microsoft SQL Server, ...

--
Toby A Inkster BSc (Hons) ARCS
[Geek of HTML/SQL/Perl/PHP/Python/Apache/Linux]
[OS: Linux 2.6.17.14-mm-desktop-9mdvsmp, up 10 days, 18:31.]

Sharing Music with Apple iTunes
http://tobyinkster.co.uk/blog/2007/1...tunes-sharing/

[Jerry Stuckle]

Mikhail Kovalev wrote:

Hi.

I work with recursive array trees of 10-20 levels in PHP. So far I
have been using serialize() to store the arrays, generating files 5+
MB large, which take 10-15 seconds to unserialize and about tenfold of
RAM! I've been adviced to use MySQL (relational db's), but are there
any other options beside that and var_export/include? Something that
works in a similar way as MySQL when adding a new element without
loading the whole database itself...

Thanks!

What do you mean by "without loading the whole database itself"?

--
=============== ===
Remove the "x" from my email address
Jerry Stuckle
JDS Computer Training Corp.
js*******@attgl obal.net
=============== ===

[Mikhail Kovalev]

I unserialize an 8MB file, which contains the db array, and put the
new element there (then serialize the whole thing again and store)

On 5 Des, 13:28, Jerry Stuckle <jstuck...@attg lobal.netwrote:

Mikhail Kovalev wrote:

Hi.

I work with recursive array trees of 10-20 levels in PHP. So far I
have been using serialize() to store the arrays, generating files 5+
MB large, which take 10-15 seconds to unserialize and about tenfold of
RAM! I've been adviced to use MySQL (relational db's), but are there
any other options beside that and var_export/include? Something that
works in a similar way as MySQL when adding a new element without
loading the whole database itself...

Thanks!

What do you mean by "without loading the whole database itself"?

--
=============== ===
Remove the "x" from my email address
Jerry Stuckle
JDS Computer Training Corp.
jstuck...@attgl obal.net
=============== ===

[Mikhail Kovalev]

On 5 Des, 14:02, The Natural Philosopher <[e-mail] e:

Mikhail Kovalev wrote:

I unserialize an 8MB file, which contains the db array, and put the
new element there (then serialize the whole thing again and store)

I am far from understanding what you are actually doing, but it sounds
like you are willing to trade disk usage for speed and memory usage.

Yep, you nailed it right on through, my friend

My question is, will I be able to access and store values by
addresses?
Such as here:
$database[33][NEXT][1221][NEXT][12][COUNT] += 1; for example

Each element contains a COUNT value and an array of childs (NEXT),
which can be non-empty, empty or non-existent (when i check if it
exists)
Looks like this at top level:

Array(
433 =Array(
COUNT =2121,
NEXT =Array( //
23 =Array( count&next etc ...)
433 =...
etc...
)
),
1012 =Array( count & next.....)
.....
)

[Marijn]

My question is, will I be able to access and store values by

addresses?
Such as here:
$database[33][NEXT][1221][NEXT][12][COUNT] += 1; for example

Each element contains a COUNT value and an array of childs (NEXT),
which can be non-empty, empty or non-existent (when i check if it
exists)

Maybe you should tell us what it is exactly that you are storing,
cause it seems to me that you are over complicating things. What is it
you are counting and why do you currently group it in multi-level
arrays?

Marijn

[Mikhail Kovalev]

On 5 Des, 14:11, "C. (http://symcbean.blogsp ot.com/)"
<colin.mckin... @gmail.comwrote :

On 5 Dec, 11:13, Mikhail Kovalev <mikhail_kova.. [e-mail]:

Hi.

I work with recursive array trees of 10-20 levels in PHP. So far I
have been using serialize() to store the arrays, generating files 5+
MB large, which take 10-15 seconds to unserialize and about tenfold of
RAM! I've been adviced to use MySQL (relational db's), but are there
any other options beside that and var_export/include? Something that
works in a similar way as MySQL when adding a new element without
loading the whole database itself...

Thanks!

Any relational DB can model a hierarchical data structure - at the
simplest level:

node_id unique identifier, not null
parent_node null
payload whatever

But getting the threaded structure is a bit tricky -

SELECT parent.node_id as Parent_node_id, parent.payload as
parent_payload,
child.node_id as child_node_id, child.payload as child_payload
FROM mytable parent, mytable child
WHERE child.parent_no de=parent.node_ id
AND parent.node_id= $start_here

.. i.e. by explicitly declaring aliases for each level in the tree you
can get the sub-nodes, but when you don't know how deep the tree is,
its not possible to return the whole structure in a single query using
standard SQL. Oracle allows you to compose queries returning
representations of hierarchical data (have a google for 'connect by' &
'start with') but thats rather non-standard.

I don't know how deep the tree is, but I don't need to return the
whole structure in a single query. Alas, I don't know the nature of
MySQL

At this moment everything I do with my structure happens by calling
addresses
See my reply to The Natural Philosopher.
Here's an example with one node at top level and it's childs:

112 (10)
225 (3)
930 (1)
11 (5)

$array[112][COUNT] = 10;
$array[112][NEXT][225][COUNT] = 3;
$array[112][NEXT][225][NEXT][930][COUNT] = 1;
$array[112][NEXT][225][NEXT][930][NEXT] = array();
$array[112][NEXT][11][COUNT] = 5;
$array[112][NEXT][11][NEXT] = array();

An address is an array, f ex array(112, 225, 930)

I then use the following mechanism:

// retrieves a node with its two values as an array(COUNT, NEXT)
// NEXT contain further childs, but i dont use them
// array_1 is the whole structure, and array_2 is the address
function function_7($arr ay_1, $array_2) { return eval('return
$array_1'.funct ion_8($array_2) .';'); }

// array(112, 225, 930) becomes "[112][NEXT][225][NEXT][930]"
function function_8($arr ay_1) { return implode('[NEXT]',
array_map('func tion_9', $array_1)); }

function function_9($int eger_1) { return '['.$integer_1.']'; }

// saves a value array_3 to an address array_2 in structure array_1
function function_10($ar ray_1, $array_2, $array_3)
{
eval('$array_1' .function_8($ar ray_2).' = $array_3;');
return $array_1;
}

[Mikhail Kovalev]

On 5 Des, 14:52, Marijn <marijn.huizend [e-mail] rote:

My question is, will I be able to access and store values by
addresses?
Such as here:
$database[33][NEXT][1221][NEXT][12][COUNT] += 1; for example

Each element contains a COUNT value and an array of childs (NEXT),
which can be non-empty, empty or non-existent (when i check if it
exists)

Maybe you should tell us what it is exactly that you are storing,
cause it seems to me that you are over complicating things. What is it
you are counting and why do you currently group it in multi-level
arrays?

Marijn

I'm storing some statistics on Markov chains of characters in words

[Captain Paralytic]

On 5 Dec, 13:56, Mikhail Kovalev <mikhail_kova.. [e-mail]:

Alas, I don't know the nature of
MySQL

This has nothing to do with MySQL per se and everything to do with
understanding data storage models in general.

Here is some good reading on the subject.
http://del.icio.us/Captain_Paralytic/hierarchical