Postgres clustering?

[warwick.poole]

I am interested in finding out about Enterprise scale Postgres
installations and clustering, especially on Linux.

Essentially I would like to know the possibility that Postgres can store
the database data in a central location (ex: on a SAN fiber array) and
have a cluster of machines sharing processor/RAM/IO bandwidth to do the
application processing. Or perhaps there is another solution similar to
what www.emicnetworks.com have developed for MySQL where the 'cluster'
seems to be really a set of individual MySQL servers that syncronize
their data at a very high speed?

I have found from my investigations that the usual Linux cluster tools,
like Beowulf/Mosix/openMosix/Alinka/Qlusters dont support Postgres and
similar apps because of the shared memory or other issues. The
complexity of RDBMS transactions seems to have limited parallel
processing in this field to the big (read: EXPENSIVE) guys like Oracle
and IBM.

Does anyone know of a technique to centralize the Postgres data storage
and farm out the processor/RAM/IO needs to a scalable resource? It seems
like it should be able to be done, but at the moment its looking to me
like the choice looks like this:

- Buy bigger and bigger hardware to run Postgres.
- Fork out large sums of money and port applications to Oracle or DB2

Anyone have any success with OpenSource tools for this or similar
issues?

Thanks.

---------------------------(end of broadcast)---------------------------
TIP 8: explain analyze is your friend

[Keith Bottner]

There currently is no solution as I have asked this question quite a few
times in the past. The best solution that exists for Postgres is
master/slave asynchronous replication which is not what you want.

I am very interested in a similar solution and believe that I could get some
resources from my company for implementing such a system. Are you interested
in helping to develop the functionality for Postgres?

Is anybody else in the Postgres world even interested in pursuing this
functionality set in more than lip service? Are there any currently active
Postgres projects out there trying to achieve this?

Is there any activity on the replication issues that appear on the TODO
Postgres list at http://developer.postgresql.org/todo.php? If so who is
working on them so I can contact them?

Feedback on these topics would be appreciated.
Keith

-----Original Message-----
From: pg*****************@postgresql.org
[mailto:pg*****************@postgresql.org] On Behalf Of warwick.poole
Sent: Thursday, February 26, 2004 2:41 PM
To: pg***********@postgresql.org
Subject: [GENERAL] Postgres clustering?
I am interested in finding out about Enterprise scale Postgres installations
and clustering, especially on Linux.

Essentially I would like to know the possibility that Postgres can store the
database data in a central location (ex: on a SAN fiber array) and have a
cluster of machines sharing processor/RAM/IO bandwidth to do the application
processing. Or perhaps there is another solution similar to what
www.emicnetworks.com have developed for MySQL where the 'cluster' seems to
be really a set of individual MySQL servers that syncronize their data at a
very high speed?

I have found from my investigations that the usual Linux cluster tools, like
Beowulf/Mosix/openMosix/Alinka/Qlusters dont support Postgres and similar
apps because of the shared memory or other issues. The complexity of RDBMS
transactions seems to have limited parallel processing in this field to the
big (read: EXPENSIVE) guys like Oracle and IBM.

Does anyone know of a technique to centralize the Postgres data storage and
farm out the processor/RAM/IO needs to a scalable resource? It seems like it
should be able to be done, but at the moment its looking to me like the
choice looks like this:

- Buy bigger and bigger hardware to run Postgres.
- Fork out large sums of money and port applications to Oracle or DB2

Anyone have any success with OpenSource tools for this or similar issues?

Thanks.

---------------------------(end of broadcast)---------------------------
TIP 8: explain analyze is your friend
---------------------------(end of broadcast)---------------------------
TIP 4: Don't 'kill -9' the postmaster

[Andrew Sullivan]

On Thu, Feb 26, 2004 at 03:22:02PM -0600, Keith Bottner wrote:

I am very interested in a similar solution and believe that I could get some
resources from my company for implementing such a system. Are you interested
in helping to develop the functionality for Postgres?

Is anybody else in the Postgres world even interested in pursuing this
functionality set in more than lip service? Are there any currently active
Postgres projects out there trying to achieve this?

If you want this, then go and support the folks working on the
Postgres-R project. They have a design for this sort of thing there,
although it does some remarkably tricky things that not everyone
thinks will work. (I happen to be among the somewhat optimistic on
this front, but I haven't been able to bring any money to the
project.)

A

--
Andrew Sullivan

---------------------------(end of broadcast)---------------------------
TIP 2: you can get off all lists at once with the unregister command
(send "unregister YourEmailAddressHere" to ma*******@postgresql.org)

[Keith Bottner]

Thanks Andrew, I will do some digging on the Postgres-R front to see what
their focus is.

Keith

-----Original Message-----
From: pg*****************@postgresql.org
[mailto:pg*****************@postgresql.org] On Behalf Of Andrew Sullivan
Sent: Thursday, February 26, 2004 4:36 PM
To: pg***********@postgresql.org
Subject: Re: [GENERAL] Postgres clustering?
On Thu, Feb 26, 2004 at 03:22:02PM -0600, Keith Bottner wrote:

I am very interested in a similar solution and believe that I could
get some resources from my company for implementing such a system. Are
you interested in helping to develop the functionality for Postgres?

Is anybody else in the Postgres world even interested in pursuing this
functionality set in more than lip service? Are there any currently
active Postgres projects out there trying to achieve this?

If you want this, then go and support the folks working on the Postgres-R
project. They have a design for this sort of thing there, although it does
some remarkably tricky things that not everyone thinks will work. (I happen
to be among the somewhat optimistic on this front, but I haven't been able
to bring any money to the
project.)

A

--
Andrew Sullivan

---------------------------(end of broadcast)---------------------------
TIP 2: you can get off all lists at once with the unregister command
(send "unregister YourEmailAddressHere" to ma*******@postgresql.org)
---------------------------(end of broadcast)---------------------------
TIP 2: you can get off all lists at once with the unregister command
(send "unregister YourEmailAddressHere" to ma*******@postgresql.org)