473,387 Members | 1,532 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,387 software developers and data experts.

Optimizing a simple UNION query

As usual, it is 2:00am, and I'm pulling my hair out, finally resorting to
posting in the newsgroups for help. :)

Simple problem, in theory.

Given table "map":

CREATE TABLE map (
entry_id int(10) unsigned NOT NULL auto_increment,
piece_id int(10) unsigned NOT NULL default '0',
hval mediumint(3) unsigned NOT NULL default '0',
pct decimal(4,2) NOT NULL default '0.00',
PRIMARY KEY (entry_id),
KEY piece_id (piece_id),
KEY hue (hval),
KEY pct (pct),
KEY idxcombo (hval,pct,piece_id)
) TYPE=MyISAM;

Pardon all of the indexes, but we've all been there. Trying anything at
this point. Basically, there are multiple rows for each "piece_id" (foreign
key into another table, but not unique in this table). Each of the rows
also contains a value for "hval" and "pct". No big deal. So, something like
this:

piece_id hval pct
1 20 1.55
1 124 2.67
1 25 14.02
1 20 1.12
2 192 7.89
2 22 2.00
2 21 4.31
2 25 62.90

What I am trying to do sounds simple, but ... I need to run a query to find
how many unique items (corresponding to piece_id) contain "hval" values
across multiple ranges, and whose SUM(pct) for those hval's is over a
certain threshhold. For example:

PSEUDO QUERY: Give me all unique piece_id's that have an hval between "1
and 20" and "200-275". Along the way, sum up the pct (percentage) values
for each hval/piece, because at the end, we want the total percentages of
each case, and that total percentage must be over our threshhold value.

A simple query for one range is, of course, elementary (note the threshhold
value of 25):

---
SELECT SUM( pct ) AS total, piece_id
FROM map
WHERE ( hval >=217 AND hval <=267)
GROUP BY piece_id
HAVING total >=25
ORDER BY total DESC
LIMIT 0,30
---

This says "give me all piece_id's and the total percentage (SUMmed) for all
hvals between a range of 217 and 267, if that total percentage is over 25".

Easy enough. But adding a new range in the form of another set of (hval
=xxx AND hval <=yyy) won't work for me, as I need to find all piece_id's

that have records in BOTH ranges, not either/or. For instance:

---
SELECT SUM( pct ) AS total, piece_id
FROM map
WHERE ( hval >=217 AND hval <=267) AND ( hval >=20 AND hval <=75)
GROUP BY piece_id
HAVING total >=25
ORDER BY total DESC
LIMIT 0,30
---

This won't work, for obvious reasons. Any given row will contain only one
hval, and in most cases, it will never fall into both ranges, although it
may (rarely). I need to handle both situations, I guess. So, I'm trying a
UNION:

---
(SELECT SUM( pct ) AS total, piece_id
FROM map
WHERE ( hval >=217 AND hval <=267)
GROUP BY piece_id
HAVING total >=25)

UNION

(SELECT SUM( pct ) AS total, piece_id
FROM map
WHERE ( hval >=20 AND hval <=75)
GROUP BY piece_id
HAVING total >=25)

ORDER BY total DESC
LIMIT 0, 30
---

I'm trying to find which "piece_id"s have rows that fall into both ranges. I
suppose this would work, but ... the EXPLAIN against this reveals:

table type possible_keys key key_len ref rows Extra
map range hval,idxcombo idxcombo 3 NULL 26704 Using where; Using
index; Using temporary; Using f...
map range hval,idxcombo idxcombo 3 NULL 25111 Using where; Using
index; Using temporary; Using f...
map index NULL hval 3 NULL 281716 Using index
Nice. I was OK with the first two. But damn ... that third one will kill
me, especially when this table ends up with millions of rows. :(

Any ideas? I am running MySQL 4.0, but not 4.1, so I don't have subselect
capability handy.

Seems so simple. I probably just can't see the forest for the trees right
now. Any help would be greatly appreciated!

Peace.
Jul 20 '05 #1
0 1980

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

3
by: Haider Kazmi | last post by:
I need help trying to optimize a SQL query. I am using Oracle 8i. I have a table with about 1.2 million records, lets call it T1. I am doing a join from another table, lets say T2 which has a...
0
by: unixman | last post by:
As usual, it is 2:00am, and I'm pulling my hair out, finally resorting to posting in the newsgroups for help. :) Simple problem, in theory. Given table "map": CREATE TABLE map ( entry_id...
3
by: Paradigm | last post by:
I am using Access 2K as a front end to a MYSQL database. I am trying to run a Union query on the MYSQL database. The query is (much simplified) SELECT as ID from faxdata UNION SELECT as ID ...
20
by: Brian Tkatch | last post by:
An ORDER BY a simple-integer inside a FUNCTION, results in SQL0440N, unless the FUNCTION expects an INTEGER as its parameter. For example: DECLARE GLOBAL TEMPORARY TABLE A(A CHAR(1)) INSERT INTO...
3
by: mikes | last post by:
I have 2 separate queries, which effectively are the same except they draw data from separate tables. Both tables are (design-wise) identical, only the data is different. for each query, there are...
6
by: das | last post by:
Hello all, I have a table with thousands of rows and is in this format: id col1 col2 col3 col4 --- ------ ----- ------ ------ 1 nm 78 xyz pir 2 ...
0
by: rashmigaikwad | last post by:
Hi All, I need help in optimizing the query mentioned below: SELECT SUM(CASE WHEN PROD_TYP='HBRMC' AND INC_MULTIPLE < 2.50 AND LTV <= 0.75 OR LTV IS NULL THEN ADV_IN_QTR ELSE 0 END) ...
9
by: Emin | last post by:
Dear Experts, I have a fairly simple query in which adding a where clause slows things down by at least a factor of 100. The following is the slow version of the query ...
5
by: Merennulli | last post by:
To start with, I'll give a simplified overview of my data. BaseRecord (4mil rows, 25k in each Region) ID | Name | Region | etc OtherData (7.5mil rows, 1 or 2 per ID) ID | Type(1/2) | Data ...
0
by: Charles Arthur | last post by:
How do i turn on java script on a villaon, callus and itel keypad mobile phone
0
by: ryjfgjl | last post by:
If we have dozens or hundreds of excel to import into the database, if we use the excel import function provided by database editors such as navicat, it will be extremely tedious and time-consuming...
0
by: ryjfgjl | last post by:
In our work, we often receive Excel tables with data in the same format. If we want to analyze these data, it can be difficult to analyze them because the data is spread across multiple Excel files...
0
BarryA
by: BarryA | last post by:
What are the essential steps and strategies outlined in the Data Structures and Algorithms (DSA) roadmap for aspiring data scientists? How can individuals effectively utilize this roadmap to progress...
1
by: nemocccc | last post by:
hello, everyone, I want to develop a software for my android phone for daily needs, any suggestions?
1
by: Sonnysonu | last post by:
This is the data of csv file 1 2 3 1 2 3 1 2 3 1 2 3 2 3 2 3 3 the lengths should be different i have to store the data by column-wise with in the specific length. suppose the i have to...
0
marktang
by: marktang | last post by:
ONU (Optical Network Unit) is one of the key components for providing high-speed Internet services. Its primary function is to act as an endpoint device located at the user's premises. However,...
0
by: Hystou | last post by:
Most computers default to English, but sometimes we require a different language, especially when relocating. Forgot to request a specific language before your computer shipped? No problem! You can...
0
Oralloy
by: Oralloy | last post by:
Hello folks, I am unable to find appropriate documentation on the type promotion of bit-fields when using the generalised comparison operator "<=>". The problem is that using the GNU compilers,...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.