473,883 Members | 1,652 Online
Bytes | Software Development & Data Engineering Community
+ Post

Home Posts Topics Members FAQ

'like' refuses to use an index???

Hi, everybody!

I just ran into a weird problem on 7.3.4.
Here is a simple testcase:

rapidb=# create table nametab (name text);
CREATE TABLE
rapidb=# create index name_idx on nametab(name);
CREATE INDEX
rapidb=# set enable_seqscan= false;
SET
rapidb=# set enable_sort=fal se;
SET
rapidb=# explain select * from nametab where name like 'blah%';
QUERY PLAN
------------------------------------------------------------------------
Seq Scan on nametab (cost=100000000 .00..100000022. 50 rows=5 width=32)
Filter: (name ~~ 'blah%'::text)
(2 rows)

rapidb=# explain select * from nametab where name like 'blah%' order by name;
QUERY PLAN
--------------------------------------------------------------------------
Index Scan using name_idx on nametab (cost=0.00..54. 50 rows=5 width=32)
Filter: (name ~~ 'blah%'::text)
(2 rows)
See - the first query wants to use seqscan, even though I am explicitly
telling it not to.
The second query does use the index for sorting (good), but still not
for the condition.

Does anyone have any idea what could be wrong here?
I'd greatly appreciate that...

Thanks a lot!

Dima.

P.S. I don't think this has anything to do with the table being empty -
first of all this is just a simple testcase, my real table has about 120
million rows (and I just analyzed it a few minutes ago).... also the
problem seems to only be with 'like' - if you replace 'like' with '=' in
the above query then it *will* use the index, even though the table is
still empty
---------------------------(end of broadcast)---------------------------
TIP 2: you can get off all lists at once with the unregister command
(send "unregister YourEmailAddres sHere" to ma*******@postg resql.org)

Nov 12 '05 #1
17 6205
Dima Tkach <dm****@openrat ings.com> writes:
Does anyone have any idea what could be wrong here?


You didn't initdb in C locale ...

regards, tom lane

---------------------------(end of broadcast)---------------------------
TIP 1: subscribe and unsubscribe commands go to ma*******@postg resql.org

Nov 12 '05 #2
Tom Lane wrote:
Dima Tkach <dm****@openrat ings.com> writes:

Does anyone have any idea what could be wrong here?


You didn't initdb in C locale ...

regards, tom lane

Ouch!
Is there any way to fix that without recreating the database?
Also, are you sure about this? Because the text comparison operators do
seem to work fine...

name like 'blah%' does not work, but name >= 'blah' and name < 'blai'
*does*... aren't these locale-dependent too?

Thanks a lot for your help!

Dima

---------------------------(end of broadcast)---------------------------
TIP 1: subscribe and unsubscribe commands go to ma*******@postg resql.org

Nov 12 '05 #3
Quoting Dima Tkach <dm****@openrat ings.com>:
Tom Lane wrote:
Dima Tkach <dm****@openrat ings.com> writes:

Does anyone have any idea what could be wrong here?


You didn't initdb in C locale ...

regards, tom lane

Ouch!
Is there any way to fix that without recreating the database?
Also, are you sure about this? Because the text comparison operators do
seem to work fine...

name like 'blah%' does not work, but name >= 'blah' and name < 'blai'
*does*... aren't these locale-dependent too?

Thanks a lot for your help!

Dima

---------------------------(end of broadcast)---------------------------
TIP 1: subscribe and unsubscribe commands go to ma*******@postg resql.org


I wanted to know this too because I notice that using like with wildcards
appears to be similar to a regular expression in that the index is not used.
This is what I have...

ethernet=# select version();
version
----------------------------------------------------------------------
PostgreSQL 7.4 on i686-pc-linux-gnu, compiled by GCC gcc (GCC) 3.2.2
(1 row)

ethernet=# \d vendors
Table "public.vendors "
Column | Type | Modifiers
---------+-----------------------+-----------
header | character(6) |
company | character varying(80) |
Indexes:
"vender_id_ idx" btree (header)
ethernet=# explain select * from vendors where header like '000423';
QUERY PLAN
---------------------------------------------------------------------------------
Index Scan using vender_id_idx on vendors (cost=0.00..113 .26 rows=36 width=68)
Index Cond: (header = '000423'::bpcha r)
Filter: (header ~~ '000423'::text)
(3 rows)
Ok, that made sense-

ethernet=# explain select * from vendors where header like '%000423%';
QUERY PLAN
----------------------------------------------------------
Seq Scan on vendors (cost=0.00..151 .15 rows=3 width=68)
Filter: (header ~~ '%000423%'::tex t)
(2 rows)

This didn't make sense until I did...

ethernet=# explain select * from vendors where header like '0004%';
QUERY PLAN
---------------------------------------------------------------------------------
Index Scan using vender_id_idx on vendors (cost=0.00..113 .26 rows=36 width=68)
Index Cond: ((header >= '0004'::bpchar) AND (header < '0005'::bpchar) )
Filter: (header ~~ '0004%'::text)
(3 rows)

which again made sense because of the header's size but both-

ethernet=# explain select * from vendors where header ~* '0004';
QUERY PLAN
-----------------------------------------------------------
Seq Scan on vendors (cost=0.00..151 .15 rows=58 width=68)
Filter: (header ~* '0004'::text)
(2 rows)

ethernet=# explain select * from vendors where header ~* '000423';
QUERY PLAN
----------------------------------------------------------
Seq Scan on vendors (cost=0.00..151 .15 rows=3 width=68)
Filter: (header ~* '000423'::text)
(2 rows)

are sequentially scanned which means that regex's do not use indexes. Is that
right also?

--
Keith C. Perry, MS E.E.
Director of Networks & Applications
VCSN, Inc.
http://vcsn.com

_______________ _______________ ______
This email account is being host by:
VCSN, Inc : http://vcsn.com

---------------------------(end of broadcast)---------------------------
TIP 7: don't forget to increase your free space map settings

Nov 12 '05 #4
Nah...
This is a different story - for teh index to be useful, the *beginning*
of your search string must be known.
So "like '00423%" and "~ '^00423'" should both work, but "like '%423'"
and "~ '00423'" both won't - it's like searching a telephone book for
somebody, whose last name ends with "erry" (as opposed to begins with
"Perr").

Dima
Keith C. Perry wrote:
I wanted to know this too because I notice that using like with wildcards
appears to be similar to a regular expression in that the index is not used.
This is what I have...

ethernet=# select version();
version
----------------------------------------------------------------------
PostgreSQL 7.4 on i686-pc-linux-gnu, compiled by GCC gcc (GCC) 3.2.2
(1 row)

ethernet=# \d vendors
Table "public.vendors "
Column | Type | Modifiers
---------+-----------------------+-----------
header | character(6) |
company | character varying(80) |
Indexes:
"vender_id_ idx" btree (header)
ethernet=# explain select * from vendors where header like '000423';
QUERY PLAN
---------------------------------------------------------------------------------
Index Scan using vender_id_idx on vendors (cost=0.00..113 .26 rows=36 width=68)
Index Cond: (header = '000423'::bpcha r)
Filter: (header ~~ '000423'::text)
(3 rows)
Ok, that made sense-

ethernet=# explain select * from vendors where header like '%000423%';
QUERY PLAN
----------------------------------------------------------
Seq Scan on vendors (cost=0.00..151 .15 rows=3 width=68)
Filter: (header ~~ '%000423%'::tex t)
(2 rows)

This didn't make sense until I did...

ethernet=# explain select * from vendors where header like '0004%';
QUERY PLAN
---------------------------------------------------------------------------------
Index Scan using vender_id_idx on vendors (cost=0.00..113 .26 rows=36 width=68)
Index Cond: ((header >= '0004'::bpchar) AND (header < '0005'::bpchar) )
Filter: (header ~~ '0004%'::text)
(3 rows)

which again made sense because of the header's size but both-

ethernet=# explain select * from vendors where header ~* '0004';
QUERY PLAN
-----------------------------------------------------------
Seq Scan on vendors (cost=0.00..151 .15 rows=58 width=68)
Filter: (header ~* '0004'::text)
(2 rows)

ethernet=# explain select * from vendors where header ~* '000423';
QUERY PLAN
----------------------------------------------------------
Seq Scan on vendors (cost=0.00..151 .15 rows=3 width=68)
Filter: (header ~* '000423'::text)
(2 rows)

are sequentially scanned which means that regex's do not use indexes. Is that
right also?


---------------------------(end of broadcast)---------------------------
TIP 6: Have you searched our list archives?

http://archives.postgresql.org

Nov 12 '05 #5
> You didn't initdb in C locale ...

The documentation section on localization could use some enhancements
and maybe some more examples.

The 'c' locale isn't very well defined in the docs, except to say that it
is 'special' and is the default if no other locale is defined. That
section doesn't mention that you need that locale to get 'like' to
use an index, for example.

However, I think RH always sets the LANG environmental variable, so
that's going to be picked up by initdb, which means that the C locale
will NOT be used unless specifically asked for. Other OS packages may
also force the choice of a default LANG value.

Dumb question of the hour: How does one find out what locale a
DB is initialized in?
--
Mike Nolan

---------------------------(end of broadcast)---------------------------
TIP 4: Don't 'kill -9' the postmaster

Nov 12 '05 #6
Mike Nolan <no***@gw.tssi. com> writes:
However, I think RH always sets the LANG environmental variable, so
that's going to be picked up by initdb, which means that the C locale
will NOT be used unless specifically asked for. Other OS packages may
also force the choice of a default LANG value.
Yeah. There have been some discussions on pgsql-hackers about
defaulting to C locale instead of honoring LANG, but we haven't done
anything.
Dumb question of the hour: How does one find out what locale a
DB is initialized in?


In 7.4, you can just "show lc_collate". In prior versions you need
to use pg_controldata to see what's stored in pg_control.

BTW, 7.4 also has a specialized index opclass that can be used to create
LIKE-compatible indexes even if you are using a non-C locale.

regards, tom lane

---------------------------(end of broadcast)---------------------------
TIP 5: Have you checked our extensive FAQ?

http://www.postgresql.org/docs/faqs/FAQ.html

Nov 12 '05 #7
Quoting Dima Tkach <dm****@openrat ings.com>:
Nah...
This is a different story - for teh index to be useful, the *beginning*
of your search string must be known.
So "like '00423%" and "~ '^00423'" should both work, but "like '%423'"
and "~ '00423'" both won't - it's like searching a telephone book for
somebody, whose last name ends with "erry" (as opposed to begins with
"Perr").

Dima
Keith C. Perry wrote:
I wanted to know this too because I notice that using like with wildcards
appears to be similar to a regular expression in that the index is not used.

This is what I have...

ethernet=# select version();
version
----------------------------------------------------------------------
PostgreSQL 7.4 on i686-pc-linux-gnu, compiled by GCC gcc (GCC) 3.2.2
(1 row)

ethernet=# \d vendors
Table "public.vendors "
Column | Type | Modifiers
---------+-----------------------+-----------
header | character(6) |
company | character varying(80) |
Indexes:
"vender_id_ idx" btree (header)
ethernet=# explain select * from vendors where header like '000423';
QUERY PLAN

---------------------------------------------------------------------------------
Index Scan using vender_id_idx on vendors (cost=0.00..113 .26 rows=36

width=68)
Index Cond: (header = '000423'::bpcha r)
Filter: (header ~~ '000423'::text)
(3 rows)
Ok, that made sense-

ethernet=# explain select * from vendors where header like '%000423%';
QUERY PLAN
----------------------------------------------------------
Seq Scan on vendors (cost=0.00..151 .15 rows=3 width=68)
Filter: (header ~~ '%000423%'::tex t)
(2 rows)

This didn't make sense until I did...

ethernet=# explain select * from vendors where header like '0004%';
QUERY PLAN

---------------------------------------------------------------------------------
Index Scan using vender_id_idx on vendors (cost=0.00..113 .26 rows=36

width=68)
Index Cond: ((header >= '0004'::bpchar) AND (header < '0005'::bpchar) )
Filter: (header ~~ '0004%'::text)
(3 rows)

which again made sense because of the header's size but both-

ethernet=# explain select * from vendors where header ~* '0004';
QUERY PLAN
-----------------------------------------------------------
Seq Scan on vendors (cost=0.00..151 .15 rows=58 width=68)
Filter: (header ~* '0004'::text)
(2 rows)

ethernet=# explain select * from vendors where header ~* '000423';
QUERY PLAN
----------------------------------------------------------
Seq Scan on vendors (cost=0.00..151 .15 rows=3 width=68)
Filter: (header ~* '000423'::text)
(2 rows)

are sequentially scanned which means that regex's do not use indexes. Is

that
right also?



Ahhh, so it is!! So let me ask you this. In order to build an index that would
be able to handle something like "lastname like '%erry'", would you need that
full text search patch in contrib (tsearch?) or could you do it with an index on
a function?

--
Keith C. Perry, MS E.E.
Director of Networks & Applications
VCSN, Inc.
http://vcsn.com

_______________ _______________ ______
This email account is being host by:
VCSN, Inc : http://vcsn.com

---------------------------(end of broadcast)---------------------------
TIP 9: the planner will ignore your desire to choose an index scan if your
joining column's datatypes do not match

Nov 12 '05 #8
Quoting Mike Nolan <no***@gw.tssi. com>:
You didn't initdb in C locale ...


The documentation section on localization could use some enhancements
and maybe some more examples.

The 'c' locale isn't very well defined in the docs, except to say that it
is 'special' and is the default if no other locale is defined. That
section doesn't mention that you need that locale to get 'like' to
use an index, for example.

However, I think RH always sets the LANG environmental variable, so
that's going to be picked up by initdb, which means that the C locale
will NOT be used unless specifically asked for. Other OS packages may
also force the choice of a default LANG value.

Dumb question of the hour: How does one find out what locale a
DB is initialized in?
--
Mike Nolan

---------------------------(end of broadcast)---------------------------
TIP 4: Don't 'kill -9' the postmaster


According to the doc its the pg_controldata utility but when I run it, I get an
error:

"could not open file "-D/global/pg_control" for reading: No such file or directory"

I wonder if that is because I didn't use a locale when I initialized the
database. (My locale -a return "C")

--
Keith C. Perry, MS E.E.
Director of Networks & Applications
VCSN, Inc.
http://vcsn.com

_______________ _______________ ______
This email account is being host by:
VCSN, Inc : http://vcsn.com

---------------------------(end of broadcast)---------------------------
TIP 5: Have you checked our extensive FAQ?

http://www.postgresql.org/docs/faqs/FAQ.html

Nov 12 '05 #9
Keith C. Perry wrote:
Ahhh, so it is!! So let me ask you this. In order to build an index that would
be able to handle something like "lastname like '%erry'", would you need that
full text search patch in contrib (tsearch?) or could you do it with an index on
a function?

I suppose, if this is all you need, you could just write a function,
that returns it input reversed, and then index by that function...

---------------------------(end of broadcast)---------------------------
TIP 1: subscribe and unsubscribe commands go to ma*******@postg resql.org

Nov 12 '05 #10

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

4
2457
by: kurtk | last post by:
I have install Apache and php5. I can execute php with no problem. However, I would like index.php to be recognized and executed the way that index.html is. That is, I want to be able to type www.mysite.com in the browser and not have to always type www.mysite.com/index.php. Is that an .htaccess change of some sort? I not an Apache expert.
1
1517
by: python | last post by:
Is there an Index server available in Python? For example: I have large intranet with several servers and I would like to index documents like search engines do. Users then can search for a domument in ALL intranet servers like I do on Google. Thanks for answers L.A.
15
4050
by: Steve M | last post by:
Hello, I'm trying to figure out the index position of a tuple member. I know the member name, but I need to know the members index position. I know that if I use the statement print tuple that it will print the contents of that location. What I don't understand is if I know that foo is a member of tuple, how do I get foo's index position. Thanks-in-Advance Steve
10
16531
by: deko | last post by:
I understand it's possible to make a composite Primary Key by holding down the control key and selecting multiple fields, then right-clicking and selecting Primary Key. But I've heard that's not a good idea. What I want to do is use table constraints to ensure only unique transactions - defined by a combination of amount, date, account and customer - are entered in the table while leaving the AutoNumber Transaction_ID the sole field in...
24
2965
by: Henrik Steffen | last post by:
hello all, on my master-db-server i'm running postgres 7.4.1, and I have got two slave-servers running postgres 7.4.2 running the following query on the master-server (7.4.1) delivers: explain analyze select * from foo where bar like '0101%' and foobar like 'top%';
3
2054
by: Kevin Macdonald | last post by:
I expected Postgresql to use an indexed access method, but in certain cases it is using a sequential scan. Details are below: Table: P1_NRN_ROAD ( sobjid int8 primary key, v int8 not null, ord int2 not null)
1
1891
by: rloef | last post by:
I have a multi-million row table with three indexes in MySQL-5.0.15. These indexes have the following number of distinct values: date 415 block 100000 scan 45 If I'm doing a query on this table in what order should I hit these
2
1891
by: Martin v. Löwis | last post by:
I've been working on PEP 353 for some time now. Please comment, in particular if you are using 64-bit systems. Regards, Martin PEP: 353 Title: Using ssize_t as the index type Version: $Revision: 42333 $
6
2190
by: xkenneth | last post by:
Looking to do something similair. I'm working with alot of timestamps and if they're within a couple seconds I need them to be indexed and removed from a list. Is there any possible way to index with a custom cmp() function? I assume it would be something like... list.index(something,mycmp) Thanks!
0
9932
marktang
by: marktang | last post by:
ONU (Optical Network Unit) is one of the key components for providing high-speed Internet services. Its primary function is to act as an endpoint device located at the user's premises. However, people are often confused as to whether an ONU can Work As a Router. In this blog post, we’ll explore What is ONU, What Is Router, ONU & Router’s main usage, and What is the difference between ONU and Router. Let’s take a closer look ! Part I. Meaning of...
0
9777
by: Hystou | last post by:
Most computers default to English, but sometimes we require a different language, especially when relocating. Forgot to request a specific language before your computer shipped? No problem! You can effortlessly switch the default language on Windows 10 without reinstalling. I'll walk you through it. First, let's disable language synchronization. With a Microsoft account, language settings sync across devices. To prevent any complications,...
0
11109
Oralloy
by: Oralloy | last post by:
Hello folks, I am unable to find appropriate documentation on the type promotion of bit-fields when using the generalised comparison operator "<=>". The problem is that using the GNU compilers, it seems that the internal comparison operator "<=>" tries to promote arguments from unsigned to signed. This is as boiled down as I can make it. Here is my compilation command: g++-12 -std=c++20 -Wnarrowing bit_field.cpp Here is the code in...
0
10726
jinu1996
by: jinu1996 | last post by:
In today's digital age, having a compelling online presence is paramount for businesses aiming to thrive in a competitive landscape. At the heart of this digital strategy lies an intricately woven tapestry of website design and digital marketing. It's not merely about having a website; it's about crafting an immersive digital experience that captivates audiences and drives business growth. The Art of Business Website Design Your website is...
1
10833
by: Hystou | last post by:
Overview: Windows 11 and 10 have less user interface control over operating system update behaviour than previous versions of Windows. In Windows 11 and 10, there is no way to turn off the Windows Update option using the Control Panel or Settings app; it automatically checks for updates and installs any it finds, whether you like it or not. For most users, this new feature is actually very convenient. If you want to control the update process,...
0
9558
agi2029
by: agi2029 | last post by:
Let's talk about the concept of autonomous AI software engineers and no-code agents. These AIs are designed to manage the entire lifecycle of a software development project—planning, coding, testing, and deployment—without human intervention. Imagine an AI that can take a project description, break it down, write the code, debug it, and then launch it, all on its own.... Now, this would greatly impact the work of software developers. The idea...
0
5782
by: TSSRALBI | last post by:
Hello I'm a network technician in training and I need your help. I am currently learning how to create and manage the different types of VPNs and I have a question about LAN-to-LAN VPNs. The last exercise I practiced was to create a LAN-to-LAN VPN between two Pfsense firewalls, by using IPSEC protocols. I succeeded, with both firewalls in the same network. But I'm wondering if it's possible to do the same thing, with 2 Pfsense firewalls...
1
4602
by: 6302768590 | last post by:
Hai team i want code for transfer the data from one system to another through IP address by using C# our system has to for every 5mins then we have to update the data what the data is updated we have to send another system
2
4198
muto222
by: muto222 | last post by:
How can i add a mobile payment intergratation into php mysql website.

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.