473,320 Members | 1,838 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,320 software developers and data experts.

Why index used/not used

Hello.

Explain.
I have table "traf_raw" contains field "sip_id" (integer). This field
indexed with "CREATE INDEX traf_raw_sip ON traf_raw (sip_id)".

Question.
When I try to get different rows postgres use index with one "sip_id"
and not use index with another "sip_id". I don't understand why it is
happen, but with more complex queries Seq Scan is so slowly.

Example.
With "sip_id='19'" there many rows in table, with "sip_id='29'" there
is no rows.

cnupm=> ANALYZE traf_raw;
ANALYZE
cnupm=> EXPLAIN ANALYZE SELECT * FROM traf_raw WHERE sip_id='19' LIMIT
10 OFFSET 100000;
QUERY PLAN

---------------------------------------------------------------------------------------------------------------------
Limit (cost=5230.95..5230.99 rows=1 width=56) (actual
time=2505.89..2505.89 rows=0 loops=1)
-> Seq Scan on traf_raw (cost=0.00..5230.99 rows=10808 width=56)
(actual time=0.04..2490.02 rows=10977 loops=1)
Filter: (sip_id = 19)
Total runtime: 2505.95 msec
(4 rows)

cnupm=> EXPLAIN ANALYZE SELECT * FROM traf_raw WHERE sip_id='29' LIMIT
10 OFFSET 100000;
QUERY PLAN

---------------------------------------------------------------------------------------------------------------------------------
Limit (cost=391.39..392.70 rows=1 width=56) (actual time=43.08..43.08
rows=0 loops=1)
-> Index Scan using traf_raw_sip on traf_raw (cost=0.00..392.70
rows=99 width=56) (actual time=43.07..43.07 rows=0 loops=1)
Index Cond: (sip_id = 29)
Total runtime: 43.16 msec
(4 rows)

--
engineer

---------------------------(end of broadcast)---------------------------
TIP 9: the planner will ignore your desire to choose an index scan if your
joining column's datatypes do not match

Nov 23 '05 #1
3 1696
hello,

Try reading through the pgsql-performance mailing list. Generally the database needs to be vacuumed and analyzed to update the stats usually for the planner to make the correct choices.

Mike

On Wed, Jul 21, 2004 at 11:00:06AM +0600, Anton Maksimenkov wrote:
Hello.

Explain.
I have table "traf_raw" contains field "sip_id" (integer). This field
indexed with "CREATE INDEX traf_raw_sip ON traf_raw (sip_id)".

Question.
When I try to get different rows postgres use index with one "sip_id"
and not use index with another "sip_id". I don't understand why it is
happen, but with more complex queries Seq Scan is so slowly.

Example.
With "sip_id='19'" there many rows in table, with "sip_id='29'" there
is no rows.

cnupm=> ANALYZE traf_raw;
ANALYZE
cnupm=> EXPLAIN ANALYZE SELECT * FROM traf_raw WHERE sip_id='19' LIMIT
10 OFFSET 100000;
QUERY PLAN

---------------------------------------------------------------------------------------------------------------------
Limit (cost=5230.95..5230.99 rows=1 width=56) (actual
time=2505.89..2505.89 rows=0 loops=1)
-> Seq Scan on traf_raw (cost=0.00..5230.99 rows=10808 width=56)
(actual time=0.04..2490.02 rows=10977 loops=1)
Filter: (sip_id = 19)
Total runtime: 2505.95 msec
(4 rows)

cnupm=> EXPLAIN ANALYZE SELECT * FROM traf_raw WHERE sip_id='29' LIMIT
10 OFFSET 100000;
QUERY PLAN

---------------------------------------------------------------------------------------------------------------------------------
Limit (cost=391.39..392.70 rows=1 width=56) (actual time=43.08..43.08
rows=0 loops=1)
-> Index Scan using traf_raw_sip on traf_raw (cost=0.00..392.70
rows=99 width=56) (actual time=43.07..43.07 rows=0 loops=1)
Index Cond: (sip_id = 29)
Total runtime: 43.16 msec
(4 rows)

--
engineer

---------------------------(end of broadcast)---------------------------
TIP 9: the planner will ignore your desire to choose an index scan if your
joining column's datatypes do not match


---------------------------(end of broadcast)---------------------------
TIP 5: Have you checked our extensive FAQ?

http://www.postgresql.org/docs/faqs/FAQ.html

Nov 23 '05 #2
On Tue, 2004-07-20 at 23:00, Anton Maksimenkov wrote:
Hello.

Explain.
I have table "traf_raw" contains field "sip_id" (integer). This field
indexed with "CREATE INDEX traf_raw_sip ON traf_raw (sip_id)".

Question.
When I try to get different rows postgres use index with one "sip_id"
and not use index with another "sip_id". I don't understand why it is
happen, but with more complex queries Seq Scan is so slowly.

Example.
With "sip_id='19'" there many rows in table, with "sip_id='29'" there
is no rows.

cnupm=> ANALYZE traf_raw;
ANALYZE
cnupm=> EXPLAIN ANALYZE SELECT * FROM traf_raw WHERE sip_id='19' LIMIT
10 OFFSET 100000;
QUERY PLAN

---------------------------------------------------------------------------------------------------------------------
Limit (cost=5230.95..5230.99 rows=1 width=56) (actual
time=2505.89..2505.89 rows=0 loops=1)
-> Seq Scan on traf_raw (cost=0.00..5230.99 rows=10808 width=56)
(actual time=0.04..2490.02 rows=10977 loops=1)
Filter: (sip_id = 19)
Total runtime: 2505.95 msec
(4 rows)
Note that in this instance, your query is returning >10k rows.
While in this example:
cnupm=> EXPLAIN ANALYZE SELECT * FROM traf_raw WHERE sip_id='29' LIMIT
10 OFFSET 100000;
QUERY PLAN

---------------------------------------------------------------------------------------------------------------------------------
Limit (cost=391.39..392.70 rows=1 width=56) (actual time=43.08..43.08
rows=0 loops=1)
-> Index Scan using traf_raw_sip on traf_raw (cost=0.00..392.70
rows=99 width=56) (actual time=43.07..43.07 rows=0 loops=1)
Index Cond: (sip_id = 29)
Total runtime: 43.16 msec
(4 rows)


you are only returning 0 rows.

PostgreSQL uses a cost based planner. So, in the first instance, it
thought it was returning enough rows to justify grabbing all the rows
first.

There are several settings that tune the planner to tell it when to
switch from an index scan to a seq scan. random_page_cost is the most
commonly adusted one. Try dropping it from the default of 4 to
something around 1.4 to 2.0 or so.

Also, you can force the planner to not use seq scans unless it has to by
setting enable_seqscan to off:

set enable_seqscan = off;

and run the query again. Test several of your queries and find out
where the turning point is, and then adjust the random_page_cost to
cause it to switch at about the right time.
---------------------------(end of broadcast)---------------------------
TIP 7: don't forget to increase your free space map settings

Nov 23 '05 #3
Scott Marlowe wrote:
There are several settings that tune the planner to tell it when to
switch from an index scan to a seq scan. random_page_cost is the most
commonly adusted one. Try dropping it from the default of 4 to
something around 1.4 to 2.0 or so. .... and run the query again. Test several of your queries and find out
where the turning point is, and then adjust the random_page_cost to
cause it to switch at about the right time.


Thank's for explain. With "random_page_cost = 2.0" situation is much
better. I will follow you tips.

EXPLAIN ANALYZE SELECT * FROM traf_raw WHERE sip_id='19' LIMIT 10 OFFSET
100000;
....
Limit (cost=5042.22..5042.56 rows=1 width=56) (actual
time=110.32..110.32 rows=0 loops=1)
-> Index Scan using traf_raw_sip on traf_raw (cost=0.00..5042.56
rows=11020 width=56) (actual time=0.30..95.95 rows=10977 loops=1)
Index Cond: (sip_id = 19)
Total runtime: 110.52 msec

--
engineer

---------------------------(end of broadcast)---------------------------
TIP 8: explain analyze is your friend

Nov 23 '05 #4

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

6
by: Heiko | last post by:
Hello, is there any way (v$-view) to get informaion about how often an index hast been used since of starting the Database? Thanks for help Heiko
2
by: Mansoor Azam | last post by:
I have the following table with indexes CREATE TABLE dbo.Scratch ( ItemID int IDENTITY (1, 1) NOT NULL , Login varchar (12) NOT NULL , StartDate datetime NULL , PayDate datetime NULL ,...
3
by: Phil Latio | last post by:
I am following a book on PHP and MySQL and have come across the below SQL statement. CREATE TABLE users ( user_id MEDIUMINT(8) UNSIGNED NOT NULL AUTO_INCREMENT, username VARCHAR(20) NOT NULL,...
14
by: Sean C. | last post by:
Helpful folks, Most of my previous experience with DB2 was on s390 mainframe systems and the optimizer on this platform always seemed very predictable and consistent. Since moving to a WinNT/UDB...
8
by: Mike | last post by:
Hello, I have a few rather urgent questions that I hope someone can help with (I need to figure this out prior to a meeting tomorrow.) First, a bit of background: The company I work for is...
29
by: shmartonak | last post by:
For maximum portability what should the type of an array index be? Can any integer type be used safely? Or should I only use an unsigned type? Or what? If I'm using pointers to access array...
5
by: Bas Scheffers | last post by:
Hi, I have a table with about 100K rows, on which I have created a btree index of the type table_name(int, int, int, timestamp). At first postgres was using it for my AND query on all four...
2
by: Hervé Piedvache | last post by:
Hi, I have may be a stupid question, but I'm a little surprised with some explains I have, using date fields ... I would like to understand exactly when index are used ... I'm using...
15
by: rAinDeEr | last post by:
Suppose i have a table which holds thousands of records with the following structure CREATE TABLE "test "."T_CNTRY" ( "CNTRY_CDE" CHAR(2) NOT NULL , "CNTRY_NAME" VARCHAR(50) ) and i have...
6
by: Henry J. | last post by:
I have a composite index on two columns in a table. However, the index is not used in a query that restricts the 2nd column to a constant. If both columns are linked with columns in other join...
0
by: DolphinDB | last post by:
The formulas of 101 quantitative trading alphas used by WorldQuant were presented in the paper 101 Formulaic Alphas. However, some formulas are complex, leading to challenges in calculation. Take...
0
by: ryjfgjl | last post by:
ExcelToDatabase: batch import excel into database automatically...
0
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 6 Mar 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:15 (7.15PM). In this month's session, we are pleased to welcome back...
0
by: Vimpel783 | last post by:
Hello! Guys, I found this code on the Internet, but I need to modify it a little. It works well, the problem is this: Data is sent from only one cell, in this case B5, but it is necessary that data...
0
by: jfyes | last post by:
As a hardware engineer, after seeing that CEIWEI recently released a new tool for Modbus RTU Over TCP/UDP filtering and monitoring, I actively went to its official website to take a look. It turned...
0
by: ArrayDB | last post by:
The error message I've encountered is; ERROR:root:Error generating model response: exception: access violation writing 0x0000000000005140, which seems to be indicative of an access violation...
1
by: PapaRatzi | last post by:
Hello, I am teaching myself MS Access forms design and Visual Basic. I've created a table to capture a list of Top 30 singles and forms to capture new entries. The final step is a form (unbound)...
0
by: CloudSolutions | last post by:
Introduction: For many beginners and individual users, requiring a credit card and email registration may pose a barrier when starting to use cloud servers. However, some cloud server providers now...
0
by: Faith0G | last post by:
I am starting a new it consulting business and it's been a while since I setup a new website. Is wordpress still the best web based software for hosting a 5 page website? The webpages will be...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.