473,574 Members | 17,716 Online
Bytes | Software Development & Data Engineering Community
+ Post

Home Posts Topics Members FAQ

Please Help Obi-Wan: Efficient Join/Cursor/Something/Anything?

Hi all

I have a bit of a dilema that I am hoping some of you smart dudes
might be able to help me with.

1. I have a table with about 50 million records in it and quite a few
columns. [Table A]

2. I have another table with just over 300 records in it and a single
column (besides the id). [Table B]

3. I want to:

Select all of those records from Table A where [table A].description
does NOT contain any of (select color from [table B])

4. An example

Table A
id ... [other columns] ... description
1 the green hornet
2 a red ball
3 a green dog
4 the yellow submarine
5 the pink panther
Table B
id color
55 blue
56 gold
57 green
58 purple
59 pink
60 white

So I want to select all those rows in Table A where none of the words
from Table B.color appear in the description field in Table A.
I.E: The query would return the following from Table A:

2 a red ball
4 the yellow submarine
The real life problem has more variables and is a little more
complicated than this but this should suffice to give me the right
idea.
Due to the number of rows involved I need this to be relevantly
efficient. Can someone suggest the most efficient way to proceed.
PS. Please excuse my ignorance.
Cheers
Sean
Jul 20 '05 #1
3 2202
On 7 Nov 2003 09:36:26 -0800, sf*****@efinanc ialnews.com (Sean) wrote:
Due to the number of rows involved I need this to be relevantly
efficient. Can someone suggest the most efficient way to proceed.


Well, I don't know exactly about efficient. :)

I came up with this method, using like instead of not like (then
reversing the results) because like is supposedly faster, and not in
on integers isn't a big deal. Maybe reversed is faster (not like on
the cursor, in on the integers), but I was doubting it. Couldn't think
a way around a big honkin' SQL statement built by a cursor, executed
in nicely limited 8000 character chunks. At least this cuts down the
number of executions (versus 300 seperate ones...)

Test bed:

create table TableA (id int, description varchar(50))
insert into TableA values (1, 'the green hornet')
insert into TableA values (2, 'a red ball')
insert into TableA values (3, 'a green dog')
insert into TableA values (4, 'the yellow submarine')
insert into TableA values (5, 'the pink panther')
create table TableB (id int, color varchar(50))
insert into TableB values (55, 'blue')
insert into TableB values (56, 'gold')
insert into TableB values (57, 'green')
insert into TableB values (58, 'purple')
insert into TableB values (59, 'pink')
insert into TableB values (60, 'white')

SQL to get values:

declare @SQLstr varchar(8000)
declare @Color varchar(50)

create table #resultstable (id int)

declare color_cursor CURSOR for select distinct color from TableB
open color_cursor
fetch next from color_cursor into @Color
select @SQLStr = 'select id from TableA where ((1=0) '
while @@fetch_status = 0
begin
if (len(@SQLStr) > 7000)
begin
select @SQLStr = @SQLStr + ')'
insert into #resultstable(i d) execute @SQLStr
select @SQLStr = 'select id from TableA where ((1=0) '
end
else
begin
select @SQLStr = @SQLStr + ' or description like ''%'
+ @Color + '%'' '
end
fetch next from color_cursor into @Color
end

select @SQLStr = @SQLStr + ')'
insert into #resultstable(i d) execute(@SQLStr )
close color_cursor
deallocate color_cursor

select * from TableA where id not in (select distinct id from
#resultstable)
drop table #resultstable
___________
To replay by email, chop off the head!
Jul 20 '05 #2
sf*****@efinanc ialnews.com (Sean) wrote in message news:<da******* *************** ****@posting.go ogle.com>...
Hi all
1. I have a table with about 50 million records [sic] in it and quite a few
columns. [Table A] <<

Rows are not records -- big difference.
I want to select all of those records [sic] from Table A where TableA.descript ion does NOT contain any of (SELECT color FROM TableB)
<<

Please post DDL, so that people do not have to guess what the keys,
constraints, Declarative Referential Integrity, datatypes, etc. in
your schema are. I think that description is free text, Latin
alphabet and lowercased, but who knows?
So I want to select all those rows in Table A where none of the words from TableB.color appear in the description field [sic] in
TableA. <<

SELECT *
FROM TableA AS A1
WHERE NOT EXISTS
(SELECT *
FROM TableB AS B1
WHERE A1.description LIKE '%' + B1.color +'%');
Due to the number of rows involved I need this to be relevantly

efficient. Can someone suggest the most efficient way to proceed. <<

Don't use SQL for text searches; there are better products for that.
This is simply going to be a slow table scan.
Jul 20 '05 #3
Hi Sean,

You can use ... A left outer join B ... where B is null. -- Louis

create table #A (X varchar(100))
insert into #A values ('the green hornet')
insert into #A values ('a red ball')
insert into #A values ('a green dog')
insert into #A values ('the yellow submarine')
insert into #A values ('the pink panther')
create Table #B (X varchar(100))
insert into #B values ('green')
insert into #B values ('pink')

select a.*
from #A as a
left outer join #B as b
on a.x like '%'+b.x+'%'
where b.x is null

returns:
x
---------
a red ball
the yellow submarine
Jul 20 '05 #4

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

1
1981
by: HolaGoogle | last post by:
Hi all, Please help me with the following..it's realy urgent and i tried everything i could and i can't get it work properly!! Thanks in advance. Here's what i'm trying to accomplish: in my form i have a table with 3 rows (3 input text), in which user can enter values and then clik save. When he clicks the Save button, i'd like to be...
3
1593
by: mahsa | last post by:
hi I have this code MailMessage objEmail = new MailMessage(); objEmail.To = "ma_em@yahoo.com"; objEmail.From = "INFO@FLNE.COM"; objEmail.Cc = "msa_em@yahoo.com"; objEmail.Subject = "Test Email";
0
1981
by: s_erez | last post by:
Hi, This is a realy tricky one. I have an ASP.NET application where some pages are reading data from a DB and presenting reports. In order for the user to wait while the page is reading data from the DB I am using a DIV with a please wait message which is removed once the page is loaded. In addition I am using a global error handling using...
7
3588
by: x muzuo | last post by:
Hi guys, I have got a prob of javascript form validation which just doesnt work with my ASP code. Can any one help me out please. Here is the code: {////<<head> <title>IIBO Submit Page</title> </head> <style type="text/css">
4
3509
by: pshindle | last post by:
DB2 Team - I just downloaded and unzipped the new Fixpack 9 for DB2 ESE V8 for Windows (FP9_WR21350_ESE.exe). I then burned the unzipped Fixpack files to a CD. I proceded to install this Fixpack on a test machine running ESE but encountered a serious error during the install. It started 'copying new files' but then stopped cold and...
13
4312
by: Joner | last post by:
Hello, I'm having trouble with a little programme of mine where I connect to an access database. It seems to connect fine, and disconnect fine, but then after it won't reconnect, I get the error "operation is not allowed when object is open" so I take out the line of code: BookDetails.Connection1.Open and it comes up with the error...
5
2578
by: settyv | last post by:
Hi, Below is the Javascript function that am trying to call from asp:Button control. <script language="javascript"> function ValidateDate(fromDate,toDate) { var fromDate=new Date();
0
3039
by: 2Barter.net | last post by:
newsmail@reuters.uk.ed10.net Fwd: Money for New Orleans, AL & GA Inbox Reply Reply to all Forward Print Add 2Barter.net to Contacts list Delete this message Report phishing Show original
4
2329
by: fatboySudsy | last post by:
Hi, I have constructed a client program that has given me some error codes that i just cannot see. I was wondering if a different set of eyes with much more experience than me could help me out. Here are the error codes and underneath i have listed the program. Thanks in advance for looking. client.c: In function `main': client.c:118:...
0
7820
marktang
by: marktang | last post by:
ONU (Optical Network Unit) is one of the key components for providing high-speed Internet services. Its primary function is to act as an endpoint device located at the user's premises. However, people are often confused as to whether an ONU can Work As a Router. In this blog post, we’ll explore What is ONU, What Is Router, ONU & Router’s main...
0
8081
Oralloy
by: Oralloy | last post by:
Hello folks, I am unable to find appropriate documentation on the type promotion of bit-fields when using the generalised comparison operator "<=>". The problem is that using the GNU compilers, it seems that the internal comparison operator "<=>" tries to promote arguments from unsigned to signed. This is as boiled down as I can make it. ...
1
7835
by: Hystou | last post by:
Overview: Windows 11 and 10 have less user interface control over operating system update behaviour than previous versions of Windows. In Windows 11 and 10, there is no way to turn off the Windows Update option using the Control Panel or Settings app; it automatically checks for updates and installs any it finds, whether you like it or not. For...
0
8121
tracyyun
by: tracyyun | last post by:
Dear forum friends, With the development of smart home technology, a variety of wireless communication protocols have appeared on the market, such as Zigbee, Z-Wave, Wi-Fi, Bluetooth, etc. Each protocol has its own unique characteristics and advantages, but as a user who is planning to build a smart home system, I am a bit confused by the...
1
5635
isladogs
by: isladogs | last post by:
The next Access Europe User Group meeting will be on Wednesday 1 May 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM). In this session, we are pleased to welcome a new presenter, Adolph Dupré who will be discussing some powerful techniques for using class modules. He will explain when you may want to use classes...
0
3759
by: TSSRALBI | last post by:
Hello I'm a network technician in training and I need your help. I am currently learning how to create and manage the different types of VPNs and I have a question about LAN-to-LAN VPNs. The last exercise I practiced was to create a LAN-to-LAN VPN between two Pfsense firewalls, by using IPSEC protocols. I succeeded, with both firewalls in...
0
3777
by: adsilva | last post by:
A Windows Forms form does not have the event Unload, like VB6. What one acts like?
1
1360
muto222
by: muto222 | last post by:
How can i add a mobile payment intergratation into php mysql website.
0
1084
bsmnconsultancy
by: bsmnconsultancy | last post by:
In today's digital era, a well-designed website is crucial for businesses looking to succeed. Whether you're a small business owner or a large corporation in Toronto, having a strong online presence can significantly impact your brand's success. BSMN Consultancy, a leader in Website Development in Toronto offers valuable insights into creating...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.