473,725 Members | 1,942 Online
Bytes | Software Development & Data Engineering Community
+ Post

Home Posts Topics Members FAQ

special table queries slow until dump/restore

Hi all -

I am experiencing continually degrading performance on queries run
against the special system tables. I notice the slowdown when these
meta-data queries are run implicitly "behind the scenes" such as when
psql with readline support tries to complete a table name or pg_admin
retrieves table column info to populate the GUI headers. Performance
picked up dramatically when I had to dump/restore this weekend. I have
not seen any comparable performance problems which might suggest a more
general issue than just the system tables.

The PostgreSQL database cluster involved consists of a couple of
databases, one of which represents effectively all the data. It is
fairly unremarkable in size (10-15 GB disk space consumed in
/var/lib/pgsql in roughly 120 tables, all in the public schema) and the
only thing which might set it apart would be that several update
processes are constantly contributing new data by way of creating and
dropping temporary tables (and possibly inflating some namespace?) My
configuration is PostgreSQL 7.4.1 on Linux 2.4.22 (Fedora Core 1) i386.

I did not find anything directly relevant in the mailing list archives,
thus this post. Is there some VACUUM command or option I should be
running which operates on the special tables? Short of letting this
database run and turning on the appropriate logging, is there any way to
track this down to specific queries and get better insight? It could
take several weeks to see a pronounced slowdown.

Any other experience with a similar phenomenon? Any info which corrects
my ignorance or at least helps formulate a usable bug report would be
appreciated.

Damon

---------------------------(end of broadcast)---------------------------
TIP 7: don't forget to increase your free space map settings

Nov 23 '05 #1
1 1545
Damon Hart <dh***@sundial. com> writes:
I am experiencing continually degrading performance on queries run
against the special system tables. I notice the slowdown when these
meta-data queries are run implicitly "behind the scenes" such as when
psql with readline support tries to complete a table name or pg_admin
retrieves table column info to populate the GUI headers. Performance
picked up dramatically when I had to dump/restore this weekend. ... several update
processes are constantly contributing new data by way of creating and
dropping temporary tables


It sounds to me like you aren't keeping up housekeeping adequately.
You need to be sure that the system catalogs are vacuumed often enough
and that your FSM settings are large enough to keep track of all the
free space. These are not different considerations from avoiding bloat
in user tables and indexes --- see the archives for plenty of discussion.

regards, tom lane

---------------------------(end of broadcast)---------------------------
TIP 1: subscribe and unsubscribe commands go to ma*******@postg resql.org

Nov 23 '05 #2

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

3
2525
by: Albretch | last post by:
I am trying to insert some textual data belonging to an HTML page into a table column with 'TEXT' as data type mysql's maual _/manual.html#String_types tell you, you may insert up to (2^16 - 1), that is 65535 characters, but I am getting errors when I try to insert a column larger than 236 characters. Initially I thought it might be because I had to escape some characters, but after playing a some scenarios and just inserting a bunch...
2
2954
by: VM | last post by:
When I display data to a Windows datagrid I usually fill the underlying table (in another class) and then, once it contains all the data, I attach it to the grid. But there are some processes that involve large amounts of data so the user won't be able to see anything in the grid until after the table's already filled up (usually 40-60 secs with these large processes). Is it possible (without changing the process of filling the table...
5
8127
by: Mike Nolan | last post by:
I have a 600K row table on my production system (running 7.3.3) that I dump and load on my development system (7.4.1) every night using cron jobs. I would like to be able to restore the table under a different table name without first having to play games with the development copy of the table. If I edit the dump file with 'sed' to change the table name, I get 'invalid command \N' errors trying to reload it.
6
11308
by: Soeren Gerlach | last post by:
Hi, some weeks ago I started to develop an application using Postgresql the first time. I'm running 7.4.3 on a Linux box with a plain 2.6.7 kernel, the storage is handled by 5 SATA disks, managed by a 3ware controller and using a xfs filesystem. The DB server is a 3 Ghz P4 with 4 Gig of Ram, so the machine is quite fast for most purposes I need it; the DB server is a pure DB server, theres no application running on it. I dump and...
5
6107
by: patrick | last post by:
Mysql 4.1.15 on Win2k. Using InnoDB. Using the mysql administrator gui to create a backup, everything goes fine, and restores quickly. Using the command line: mysqldump %dbname% --single-transaction > %dbname%.sql creates a file about 15% smaller than the gui produces, and is
2
3435
by: Himmel | last post by:
Hello! The reference database I currently use runs queries that pull data from hundreds of tables in order to create user-friendly form view. The problem is that these queries can take upwards of 30 seconds to return. The solution that I've devised is to create an admin function to cache all these query results and then dump that data into a new, consolidated table, and then have the users search through those new tables. Since our...
8
2389
by: rshivaraman | last post by:
Hi : I have a TableA with around 10 columns with varchar and numeric datatypes It has 500 million records and its size is 999999999 KB. i believe it is kb i got this data after running sp_spaceused on it. The index_size was also pretty big in 6 digits. On looking at the tableA
5
6562
by: Troels Arvin | last post by:
Hello, Every so often, I'm asked to help people recover data from tables that were either dropped or where to much data was DELETEed. The complications related to restoring data are a problem. The SAS users are laughing because they can (to a certain extend) easily go back to an earlier SAS table by simply starting dsm. Of course, a flat file table is different than a relational table; but still, the trouble related to restoring DB2...
17
7842
by: DeZZar | last post by:
Hi all, I need to regularly backup my database as an Excel file and have been using the File Export option. Problem is I need anyone using the database to be able to do this easily - nopt just me. For all users the database is hidden and only a menu screen is visable. Can I create a button on my menu screen that will export a table >
0
8752
by: Hystou | last post by:
Most computers default to English, but sometimes we require a different language, especially when relocating. Forgot to request a specific language before your computer shipped? No problem! You can effortlessly switch the default language on Windows 10 without reinstalling. I'll walk you through it. First, let's disable language synchronization. With a Microsoft account, language settings sync across devices. To prevent any complications,...
0
9401
Oralloy
by: Oralloy | last post by:
Hello folks, I am unable to find appropriate documentation on the type promotion of bit-fields when using the generalised comparison operator "<=>". The problem is that using the GNU compilers, it seems that the internal comparison operator "<=>" tries to promote arguments from unsigned to signed. This is as boiled down as I can make it. Here is my compilation command: g++-12 -std=c++20 -Wnarrowing bit_field.cpp Here is the code in...
0
9257
jinu1996
by: jinu1996 | last post by:
In today's digital age, having a compelling online presence is paramount for businesses aiming to thrive in a competitive landscape. At the heart of this digital strategy lies an intricately woven tapestry of website design and digital marketing. It's not merely about having a website; it's about crafting an immersive digital experience that captivates audiences and drives business growth. The Art of Business Website Design Your website is...
1
9174
by: Hystou | last post by:
Overview: Windows 11 and 10 have less user interface control over operating system update behaviour than previous versions of Windows. In Windows 11 and 10, there is no way to turn off the Windows Update option using the Control Panel or Settings app; it automatically checks for updates and installs any it finds, whether you like it or not. For most users, this new feature is actually very convenient. If you want to control the update process,...
0
9111
tracyyun
by: tracyyun | last post by:
Dear forum friends, With the development of smart home technology, a variety of wireless communication protocols have appeared on the market, such as Zigbee, Z-Wave, Wi-Fi, Bluetooth, etc. Each protocol has its own unique characteristics and advantages, but as a user who is planning to build a smart home system, I am a bit confused by the choice of these technologies. I'm particularly interested in Zigbee because I've heard it does some...
0
8096
agi2029
by: agi2029 | last post by:
Let's talk about the concept of autonomous AI software engineers and no-code agents. These AIs are designed to manage the entire lifecycle of a software development project—planning, coding, testing, and deployment—without human intervention. Imagine an AI that can take a project description, break it down, write the code, debug it, and then launch it, all on its own.... Now, this would greatly impact the work of software developers. The idea...
1
6702
isladogs
by: isladogs | last post by:
The next Access Europe User Group meeting will be on Wednesday 1 May 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM). In this session, we are pleased to welcome a new presenter, Adolph Dupré who will be discussing some powerful techniques for using class modules. He will explain when you may want to use classes instead of User Defined Types (UDT). For example, to manage the data in unbound forms. Adolph will...
0
4782
by: adsilva | last post by:
A Windows Forms form does not have the event Unload, like VB6. What one acts like?
1
3221
by: 6302768590 | last post by:
Hai team i want code for transfer the data from one system to another through IP address by using C# our system has to for every 5mins then we have to update the data what the data is updated we have to send another system

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.