473,669 Members | 2,415 Online
Bytes | Software Development & Data Engineering Community
+ Post

Home Posts Topics Members FAQ

tsearch2 queries faster than expected

I have been using tsearch2 for quite a while with a fair amount of
success.

The other day I was playiing around with a query, and randomly changed
a few things. I noticed a 10 times speedup and didn't know why. Both
queries return identical results.

The idea was to do a proximity search, where one word appears within 10
minutes of the other.

I'm not sure if this is a bug or something weird. Using postgres
7.4.2, and I think the 7.4.2 version of tsearch2.

The two queries:
Fast:
explain analyze select m1.message_date , m1.message_id from messages m1,
messages m2, to_tsquery('hal ey') q1, to_tsquery('bir thday') q2 where
m2.message_date between m1.message_date - '5 minutes'::inter val and
m1.message_date + '5 minutes'::inter val and m1.idxfti @@ q1 and
m2.idxfti @@ q2 and m1.message_id <> m2.message_id;

QUERY PLAN
------------------------------------------------------------------------
------------------------------------------------------------------------
---------------------------------------------------------------------
Nested Loop (cost=485403.85 ..549229077.83 rows=1651000057 width=12)
(actual time=190.952..2 21.859 rows=4 loops=1)
Join Filter: (("outer".messa ge_date >= ("inner".messag e_date -
'00:05:00'::int erval)) AND ("outer".messag e_date <=
("inner".messag e_date + '00:05:00'::int erval)) AND ("inner".messag e_id
<> "outer".message _id))
-> Nested Loop (cost=0.00..484 867.85 rows=121898 width=12) (actual
time=0.879..33. 273 rows=86 loops=1)
-> Function Scan on q2 (cost=0.00..12. 50 rows=1000 width=32)
(actual time=0.034..0.0 37 rows=1 loops=1)
-> Index Scan using fti_idx on messages m2
(cost=0.00..483 .33 rows=122 width=44) (actual time=0.831..32. 828
rows=86 loops=1)
Index Cond: (m2.idxfti @@ "outer".q2)
Filter: (m2.idxfti @@ "outer".q2)
-> Materialize (cost=485403.85 ..487158.83 rows=121898 width=12)
(actual time=0.189..1.4 77 rows=160 loops=86)
-> Nested Loop (cost=0.00..484 867.85 rows=121898 width=12)
(actual time=16.132..11 0.991 rows=160 loops=1)
-> Function Scan on q1 (cost=0.00..12. 50 rows=1000
width=32) (actual time=0.061..0.0 65 rows=1 loops=1)
-> Index Scan using fti_idx on messages m1
(cost=0.00..483 .33 rows=122 width=44) (actual time=16.048..99 .997
rows=160 loops=1)
Index Cond: (m1.idxfti @@ "outer".q1)
Filter: (m1.idxfti @@ "outer".q1)
Total runtime: 223.481 ms
(14 rows)

Slow:

explain analyze select m1.message_date , m1.message, m2.message_date
from messages m1, messages m2 where m2.message_date BETWEEN
m1.message_date - '5 minutes'::inter val and m1.message_date + '5
minutes'::inter val and m1.idxfti @@ to_tsquery('hal ey') and m2.idxfti
@@ to_tsquery('bir thday') and m1.message_id <> m2.message_id;

QUERY PLAN
------------------------------------------------------------------------
------------------------------------------------------------------------
---------------------------------------------------------------------
Nested Loop (cost=0.00..597 84.68 rows=1654 width=56) (actual
time=746.830..3 132.006 rows=4 loops=1)
Join Filter: (("inner".messa ge_date >= ("outer".messag e_date -
'00:05:00'::int erval)) AND ("inner".messag e_date <=
("outer".messag e_date + '00:05:00'::int erval)) AND ("outer".messag e_id
<> "inner".message _id))
-> Index Scan using fti_idx on messages m1 (cost=0.00..483 .33
rows=122 width=52) (actual time=8.770..69. 013 rows=160 loops=1)
Index Cond: (idxfti @@ '\'haley\''::ts query)
Filter: (idxfti @@ '\'haley\''::ts query)
-> Index Scan using fti_idx on messages m2 (cost=0.00..483 .33
rows=122 width=12) (actual time=0.112..18. 899 rows=86 loops=160)
Index Cond: (idxfti @@ '\'birthday\'': :tsquery)
Filter: (idxfti @@ '\'birthday\'': :tsquery)
Total runtime: 3132.665 ms
(9 rows)
---------------------------(end of broadcast)---------------------------
TIP 6: Have you searched our list archives?

http://archives.postgresql.org

Nov 23 '05 #1
2 1571
Jeffrey Melloy <jm*****@visual distortion.org> writes:
The other day I was playiing around with a query, and randomly changed
a few things. I noticed a 10 times speedup and didn't know why. Both
queries return identical results.


It looks like the planner's overestimate of the number of rows returned
by the function causes it to insert a Materialize step in the inside of
the nestloop join, so as to avoid recomputing the inner scan multiple
times. Which is a good idea. I wonder why it didn't do it in both
cases?

regards, tom lane

---------------------------(end of broadcast)---------------------------
TIP 7: don't forget to increase your free space map settings

Nov 23 '05 #2
Jeffrey Melloy <jm*****@visual distortion.org> writes:
The other day I was playiing around with a query, and randomly changed
a few things. I noticed a 10 times speedup and didn't know why. Both
queries return identical results.


It looks like the planner's overestimate of the number of rows returned
by the function causes it to insert a Materialize step in the inside of
the nestloop join, so as to avoid recomputing the inner scan multiple
times. Which is a good idea. I wonder why it didn't do it in both
cases?

regards, tom lane

---------------------------(end of broadcast)---------------------------
TIP 7: don't forget to increase your free space map settings

Nov 23 '05 #3

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

13
2862
by: Nigel J. Andrews | last post by:
This will be a little vague, it was last night and I can't now do the test in that db (see below) so can't give the exact wording. I seem to remember a report a little while ago about tsearch v2 causing unexpected backend exit messages with 7.3.4 and now I'm getting similar messages unpredictably and I can't find the thread in the archives either. What I did was install tsearch2 using share/contrib/tsearch2.sql, which placed everything...
1
1757
by: psql-mail | last post by:
I have applied the recent tsearch2 patch and recompiled the tsearch2 module but I am still experiencing the same backend crashes as I previously described. Thanks for any help, Mat GDB output:
3
2044
by: Diogo Biazus | last post by:
Hi, Is there any performance diference between the following SQL commands: SELECT * FROM documents WHERE content_ix @@ to_tsquery('word1&word2|word3'); SELECT * FROM documents WHERE content_ix @@ to_tsquery('word1') AND content_ix @@ to_tsquery('word2') OR content_ix @@ to_tsquery('word3'); I'm having to do this on some complex querys to put LIKEs between some
9
1995
by: Pavel Stehule | last post by:
Hello I try tsearch2 within czech environment. It is works fine, but I have two questions. 1. I have words "se", "ve" in my czech stop words. But I get this words in result. Why? Have I problem with my configuration? tsearch2=# select * from ts_debug('jmenuji se Pavel Stěhule a bydlím ve Skalici.');
2
3684
by: Fischer Ulrich | last post by:
Hi I have a problem with the restoring of a database which uses tsearch2. I made a backup as discribed in 'tsearch-v2-intro' on the tsearch2 page. Now i'm trying to restore it into a testdatabase. The problem is, that the entries of the relations 'pg_ts_cfg', 'pg_ts_cfgmap', 'pg_ts_dict' and 'pg_ts_parser' are not correctly restored. I get errors like: psql:restoretest.out:28223: ERROR: copy: line 1, Cannot insert a duplicate key...
0
336
by: Jeffrey Melloy | last post by:
I have been using tsearch2 for quite a while with a fair amount of success. The other day I was playiing around with a query, and randomly changed a few things. I noticed a 10 times speedup and didn't know why. Both queries return identical results. The idea was to do a proximity search, where one word appears within 10 minutes of the other.
0
2096
by: Markus Wollny | last post by:
Hi! Sorry to bother you, but I just don't know how to get tsearch2 configured correctly for my setup. I've got a 7.4.3 database-cluster initdb'ed with de_DE@euro as locale, the database is with Unicode encoding. I made and installed contrib/tsearch2 after installing the dump/reload-patch http://www.sai.msu.su/~megera/postgres/gist/tsearch/V2/regprocedure_7.4.patch.gz as advised by the docs. So far everything is looking good, I have...
3
4590
by: Marcel Boscher | last post by:
Hello everybody, i tried to "J.U.S.T" install the FullTextSearchTool tsearch2 under the guidiance of : http://www.sai.msu.su/~megera/postgres/gist/tsearch/V2/ <http://www.sai.msu.su/%7Emegera/postgres/gist/tsearch/V2/> and i'm almost losing my mind... <runningagainstwallheadsfirst> after i installed postgre which works fine i change to the folder /contrib/tsearch2/
2
3322
by: Net Virtual Mailing Lists | last post by:
Hello, If I have a rule like this: CREATE OR REPLACE RULE sometable_update AS ON UPDATE TO table2 DO UPDATE cache SET updated_dt=NULL WHERE tablename='sometable'; CREATE OR REPLACE RULE sometable_insert AS ON INSERT TO table2 DO UPDATE cache SET updated_dt=NULL WHERE tablename='sometable'; CREATE OR REPLACE RULE sometable_delete AS ON DELETE TO table2 DO UPDATE
0
8383
by: Hystou | last post by:
Most computers default to English, but sometimes we require a different language, especially when relocating. Forgot to request a specific language before your computer shipped? No problem! You can effortlessly switch the default language on Windows 10 without reinstalling. I'll walk you through it. First, let's disable language synchronization. With a Microsoft account, language settings sync across devices. To prevent any complications,...
0
8894
Oralloy
by: Oralloy | last post by:
Hello folks, I am unable to find appropriate documentation on the type promotion of bit-fields when using the generalised comparison operator "<=>". The problem is that using the GNU compilers, it seems that the internal comparison operator "<=>" tries to promote arguments from unsigned to signed. This is as boiled down as I can make it. Here is my compilation command: g++-12 -std=c++20 -Wnarrowing bit_field.cpp Here is the code in...
0
8803
jinu1996
by: jinu1996 | last post by:
In today's digital age, having a compelling online presence is paramount for businesses aiming to thrive in a competitive landscape. At the heart of this digital strategy lies an intricately woven tapestry of website design and digital marketing. It's not merely about having a website; it's about crafting an immersive digital experience that captivates audiences and drives business growth. The Art of Business Website Design Your website is...
1
8587
by: Hystou | last post by:
Overview: Windows 11 and 10 have less user interface control over operating system update behaviour than previous versions of Windows. In Windows 11 and 10, there is no way to turn off the Windows Update option using the Control Panel or Settings app; it automatically checks for updates and installs any it finds, whether you like it or not. For most users, this new feature is actually very convenient. If you want to control the update process,...
0
7407
agi2029
by: agi2029 | last post by:
Let's talk about the concept of autonomous AI software engineers and no-code agents. These AIs are designed to manage the entire lifecycle of a software development project—planning, coding, testing, and deployment—without human intervention. Imagine an AI that can take a project description, break it down, write the code, debug it, and then launch it, all on its own.... Now, this would greatly impact the work of software developers. The idea...
1
6210
isladogs
by: isladogs | last post by:
The next Access Europe User Group meeting will be on Wednesday 1 May 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM). In this session, we are pleased to welcome a new presenter, Adolph Dupré who will be discussing some powerful techniques for using class modules. He will explain when you may want to use classes instead of User Defined Types (UDT). For example, to manage the data in unbound forms. Adolph will...
0
4384
by: adsilva | last post by:
A Windows Forms form does not have the event Unload, like VB6. What one acts like?
1
2792
by: 6302768590 | last post by:
Hai team i want code for transfer the data from one system to another through IP address by using C# our system has to for every 5mins then we have to update the data what the data is updated we have to send another system
2
1787
bsmnconsultancy
by: bsmnconsultancy | last post by:
In today's digital era, a well-designed website is crucial for businesses looking to succeed. Whether you're a small business owner or a large corporation in Toronto, having a strong online presence can significantly impact your brand's success. BSMN Consultancy, a leader in Website Development in Toronto offers valuable insights into creating effective websites that not only look great but also perform exceptionally well. In this comprehensive...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.