473,396 Members | 1,780 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,396 software developers and data experts.

can postgresql handle these large tables

Hi folks,
I'm busy with analyzing some data and therefore will
have to store 2 big tables containing 50 million obs &
25 million obs. Selecting some interesting obs can
decrease these tables with maybe a factor 10 but
eventually I will have to combine them using a join.
These tables contain almost 20 colums and after
selecting the appropriate ones I will have maybe 10
colums.

Off course I will have to make use of indexes and try
to optimize the settings in postgresql.conf file like:

geqo=true
geqo_treshold=11
geqo_effort= 1
geqo_generations=5

But my main question is whether postgresql will be
able to handle these large volumes of data, or should
I ask whether my hardware will be capable of working
with these large tables.

Answers, Ideas, remarks...everything is welcome folks,
Many thanks in advance, it is MUCH appreciated what is
done in this list!!


__________________________________
Do you Yahoo!?
Yahoo! Mail - Helps protect you from nasty viruses.
http://promotions.yahoo.com/new_mail

---------------------------(end of broadcast)---------------------------
TIP 7: don't forget to increase your free space map settings

Nov 23 '05 #1
1 1975
Dino Vliet wrote:
Hi folks,
I'm busy with analyzing some data and therefore will
have to store 2 big tables containing 50 million obs &
25 million obs. Selecting some interesting obs can
decrease these tables with maybe a factor 10 but
eventually I will have to combine them using a join.
These tables contain almost 20 colums and after
selecting the appropriate ones I will have maybe 10
colums.

Off course I will have to make use of indexes and try
to optimize the settings in postgresql.conf file like:

geqo=true
geqo_treshold=11
geqo_effort= 1
geqo_generations=5


These value will affect only if you join more then "geqo_threshold"
tables, it doesn't depend on the number of column and/or
rows number.
Regards
Gaetano Mendola
Nov 23 '05 #2

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

1
by: Ralph | last post by:
My company is researching the idea of moving from a very large, old and complex UniData DB to something else that is newer and quicker. I was thinking about PostgreSQL, but am not sure PostgreSQL...
125
by: Sarah Tanembaum | last post by:
Beside its an opensource and supported by community, what's the fundamental differences between PostgreSQL and those high-price commercial database (and some are bloated such as Oracle) from...
33
by: Joshua D. Drake | last post by:
Hello, I think the below just about says it all: http://www.commandprompt.com/images/mammoth_versus_dolphin_500.jpg Sincerely, Joshua Drake
74
by: John Wells | last post by:
Yes, I know you've seen the above subject before, so please be gentle with the flamethrowers. I'm preparing to enter a discussion with management at my company regarding going forward as either...
29
by: Paul Ganainm | last post by:
Hi all, Following up on another thread, here is a comparison between FB and PG from an FB'ers POV. BTW, FB is the love-child of Open-Source-Interbase. BTW, I have tried to be as accurate as I...
0
by: Rob Oakley | last post by:
I have an embedded system running FreeBSD (5.1) that does not have any local (rotating) storage (i.e disk drives). PostgreSQL also runs on this box and (at this point) has two tables. It is an...
9
by: Andy B | last post by:
If I bought one of these boxes/OS combos as a postgresql database server, would postgresql be able to make the best use of it with a huge (e.g. 40GB) database? Box: HP ProLiant DL585, with ...
5
by: nd02tsk | last post by:
Hello MySQL has information about several storage engines. MEMORY to handle temporary tables, InnoDB to handle transactions and which also can split its table data over several files/partitions....
2
by: sabbadin12 | last post by:
Hi, I'm going to work on an application that uses a postgreSQL database so that it can uses SQLServer 2005. I think I solved most problems on the programming side, but I still have some doubts...
0
by: ryjfgjl | last post by:
In our work, we often receive Excel tables with data in the same format. If we want to analyze these data, it can be difficult to analyze them because the data is spread across multiple Excel files...
0
by: emmanuelkatto | last post by:
Hi All, I am Emmanuel katto from Uganda. I want to ask what challenges you've faced while migrating a website to cloud. Please let me know. Thanks! Emmanuel
0
BarryA
by: BarryA | last post by:
What are the essential steps and strategies outlined in the Data Structures and Algorithms (DSA) roadmap for aspiring data scientists? How can individuals effectively utilize this roadmap to progress...
1
by: Sonnysonu | last post by:
This is the data of csv file 1 2 3 1 2 3 1 2 3 1 2 3 2 3 2 3 3 the lengths should be different i have to store the data by column-wise with in the specific length. suppose the i have to...
0
by: Hystou | last post by:
There are some requirements for setting up RAID: 1. The motherboard and BIOS support RAID configuration. 2. The motherboard has 2 or more available SATA protocol SSD/HDD slots (including MSATA, M.2...
0
by: Hystou | last post by:
Most computers default to English, but sometimes we require a different language, especially when relocating. Forgot to request a specific language before your computer shipped? No problem! You can...
0
jinu1996
by: jinu1996 | last post by:
In today's digital age, having a compelling online presence is paramount for businesses aiming to thrive in a competitive landscape. At the heart of this digital strategy lies an intricately woven...
0
tracyyun
by: tracyyun | last post by:
Dear forum friends, With the development of smart home technology, a variety of wireless communication protocols have appeared on the market, such as Zigbee, Z-Wave, Wi-Fi, Bluetooth, etc. Each...
0
agi2029
by: agi2029 | last post by:
Let's talk about the concept of autonomous AI software engineers and no-code agents. These AIs are designed to manage the entire lifecycle of a software development project—planning, coding, testing,...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.