473,399 Members | 4,192 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,399 software developers and data experts.

Data partitioning?

hi,

I am becoming more and more convinced that in order to achieve the
required performance and scalability I need to split my data amoungst
many backend machines.

Ideally I would start with about 10 machine and have 1/10th of the data
on each. As the data set grows I would then buy additional machines to
spread the data over.

I think this is called data partitioning in oracle?
What does postgresql have to offer in this area?

Does software already exist to query multiple identical schemas each
with different data stored in them, and then correlate the results?

If not what suggestions does anyone have for writing my own?

Thanks,
Mat

--

---------------------------(end of broadcast)---------------------------
TIP 5: Have you checked our extensive FAQ?

http://www.postgresql.org/docs/faqs/FAQ.html

Nov 12 '05 #1
1 1453
On Tue, Oct 07, 2003 at 10:34:18AM +0100, ps*******@freeuk.com wrote:
I am becoming more and more convinced that in order to achieve the
required performance and scalability I need to split my data amoungst
many backend machines.
How much data do you have, and what are you trying to run on? That
sounds like a _lot_. In any case, you can't do this in PostgreSQL
right now. A single back end controls all the data.
I think this is called data partitioning in oracle?
Usually people mean spreading tables across different disks by some
criteria when they talk about that. Completely different machines is
pretty usual.
If not what suggestions does anyone have for writing my own?


Be very careful. Multi-database concurrency problems are easy to
cause and hard to solve.

A

----
Andrew Sullivan 204-4141 Yonge Street
Afilias Canada Toronto, Ontario Canada
<an****@libertyrms.info> M2P 2A8
+1 416 646 3304 x110
---------------------------(end of broadcast)---------------------------
TIP 8: explain analyze is your friend

Nov 12 '05 #2

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

13
by: curwen | last post by:
Hi all, I'd like to have you opinion about the following performance/design issue: a logging system is generating a lot of records every our,(about 500k/day) I need to store them in a huge...
18
by: Jeff Boes | last post by:
I'm sure this is a concept that's been explored here. I have a table (fairly simple, just two columns, one of which is a 32-digit checksum) with several million rows (currently, about 7 million)....
7
by: Jane | last post by:
In Oracle we can partition a table as follows. What is the equivalent in DB2? CREATE TABLE sales_list (salesman_id NUMBER(5), salesman_name VARCHAR2(30), sales_state VARCHAR2(20),...
0
by: Robert Klemme | last post by:
Hi, this is just to verify: I checked the documentation and it seems that SQL 2005 does not support hash partitioning on a number of columns. As far as I can see only range partitioning is...
10
by: shsandeep | last post by:
DB2 V8.2 (not Viper yet and no range partitioning!!) I have created a table T1 (col1, col2) with col1 as the primary key. When I try to create a partitioning key on col2, it gives me error that it...
8
by: mitek | last post by:
Hi, All I have strange situation with table design for DB2 9.1 on Windows I have 3 tables with same structure : 1 table - is MDC 2 table - is partitioned MDC table 3 table - is compressed...
0
by: harrylarenson | last post by:
Hi, Happy New Year. I am trying to insert a query to a partitioned view but the error is : Server: Msg 4436, Level 16, State 12, Line 1 UNION ALL view 'T' is not updatable because a...
0
by: Nate Eaton | last post by:
According to the original whitepaper on UDB range partitioning (http:// www-106.ibm.com/developerworks/db2/library/techarticle/0202zuzarte/ 0202zuzarte.pdf), you can use a range as a criteria,...
10
by: nflacco | last post by:
I'm tinkering around with a data collection system, and have come up with a very hackish way to store my data- for reference, I'm anticipating collecting at least 100 million different dataId...
0
by: Charles Arthur | last post by:
How do i turn on java script on a villaon, callus and itel keypad mobile phone
0
by: emmanuelkatto | last post by:
Hi All, I am Emmanuel katto from Uganda. I want to ask what challenges you've faced while migrating a website to cloud. Please let me know. Thanks! Emmanuel
0
BarryA
by: BarryA | last post by:
What are the essential steps and strategies outlined in the Data Structures and Algorithms (DSA) roadmap for aspiring data scientists? How can individuals effectively utilize this roadmap to progress...
1
by: nemocccc | last post by:
hello, everyone, I want to develop a software for my android phone for daily needs, any suggestions?
1
by: Sonnysonu | last post by:
This is the data of csv file 1 2 3 1 2 3 1 2 3 1 2 3 2 3 2 3 3 the lengths should be different i have to store the data by column-wise with in the specific length. suppose the i have to...
0
Oralloy
by: Oralloy | last post by:
Hello folks, I am unable to find appropriate documentation on the type promotion of bit-fields when using the generalised comparison operator "<=>". The problem is that using the GNU compilers,...
0
jinu1996
by: jinu1996 | last post by:
In today's digital age, having a compelling online presence is paramount for businesses aiming to thrive in a competitive landscape. At the heart of this digital strategy lies an intricately woven...
0
agi2029
by: agi2029 | last post by:
Let's talk about the concept of autonomous AI software engineers and no-code agents. These AIs are designed to manage the entire lifecycle of a software development project—planning, coding, testing,...
0
isladogs
by: isladogs | last post by:
The next Access Europe User Group meeting will be on Wednesday 1 May 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM). In this session, we are pleased to welcome a new...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.