473,394 Members | 1,845 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,394 software developers and data experts.

Optimizing a simple UNION query

As usual, it is 2:00am, and I'm pulling my hair out, finally resorting to
posting in the newsgroups for help. :)

Simple problem, in theory.

Given table "map":

CREATE TABLE map (
entry_id int(10) unsigned NOT NULL auto_increment,
piece_id int(10) unsigned NOT NULL default '0',
hval mediumint(3) unsigned NOT NULL default '0',
pct decimal(4,2) NOT NULL default '0.00',
PRIMARY KEY (entry_id),
KEY piece_id (piece_id),
KEY hue (hval),
KEY pct (pct),
KEY idxcombo (hval,pct,piece_id)
) TYPE=MyISAM;

Pardon all of the indexes, but we've all been there. Trying anything at
this point. Basically, there are multiple rows for each "piece_id" (foreign
key into another table, but not unique in this table). Each of the rows
also contains a value for "hval" and "pct". No big deal. So, something like
this:

piece_id hval pct
1 20 1.55
1 124 2.67
1 25 14.02
1 20 1.12
2 192 7.89
2 22 2.00
2 21 4.31
2 25 62.90

What I am trying to do sounds simple, but ... I need to run a query to find
how many unique items (corresponding to piece_id) contain "hval" values
across multiple ranges, and whose SUM(pct) for those hval's is over a
certain threshhold. For example:

PSEUDO QUERY: Give me all unique piece_id's that have an hval between "1
and 20" and "200-275". Along the way, sum up the pct (percentage) values
for each hval/piece, because at the end, we want the total percentages of
each case, and that total percentage must be over our threshhold value.

A simple query for one range is, of course, elementary (note the threshhold
value of 25):

---
SELECT SUM( pct ) AS total, piece_id
FROM map
WHERE ( hval >=217 AND hval <=267)
GROUP BY piece_id
HAVING total >=25
ORDER BY total DESC
LIMIT 0,30
---

This says "give me all piece_id's and the total percentage (SUMmed) for all
hvals between a range of 217 and 267, if that total percentage is over 25".

Easy enough. But adding a new range in the form of another set of (hval
=xxx AND hval <=yyy) won't work for me, as I need to find all piece_id's

that have records in BOTH ranges, not either/or. For instance:

---
SELECT SUM( pct ) AS total, piece_id
FROM map
WHERE ( hval >=217 AND hval <=267) AND ( hval >=20 AND hval <=75)
GROUP BY piece_id
HAVING total >=25
ORDER BY total DESC
LIMIT 0,30
---

This won't work, for obvious reasons. Any given row will contain only one
hval, and in most cases, it will never fall into both ranges, although it
may (rarely). I need to handle both situations, I guess. So, I'm trying a
UNION:

---
(SELECT SUM( pct ) AS total, piece_id
FROM map
WHERE ( hval >=217 AND hval <=267)
GROUP BY piece_id
HAVING total >=25)

UNION

(SELECT SUM( pct ) AS total, piece_id
FROM map
WHERE ( hval >=20 AND hval <=75)
GROUP BY piece_id
HAVING total >=25)

ORDER BY total DESC
LIMIT 0, 30
---

I'm trying to find which "piece_id"s have rows that fall into both ranges. I
suppose this would work, but ... the EXPLAIN against this reveals:

table type possible_keys key key_len ref rows Extra
map range hval,idxcombo idxcombo 3 NULL 26704 Using where; Using
index; Using temporary; Using f...
map range hval,idxcombo idxcombo 3 NULL 25111 Using where; Using
index; Using temporary; Using f...
map index NULL hval 3 NULL 281716 Using index
Nice. I was OK with the first two. But damn ... that third one will kill
me, especially when this table ends up with millions of rows. :(

Any ideas? I am running MySQL 4.0, but not 4.1, so I don't have subselect
capability handy.

Seems so simple. I probably just can't see the forest for the trees right
now. Any help would be greatly appreciated!

Peace.
Jul 20 '05 #1
0 1981

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

3
by: Haider Kazmi | last post by:
I need help trying to optimize a SQL query. I am using Oracle 8i. I have a table with about 1.2 million records, lets call it T1. I am doing a join from another table, lets say T2 which has a...
0
by: unixman | last post by:
As usual, it is 2:00am, and I'm pulling my hair out, finally resorting to posting in the newsgroups for help. :) Simple problem, in theory. Given table "map": CREATE TABLE map ( entry_id...
3
by: Paradigm | last post by:
I am using Access 2K as a front end to a MYSQL database. I am trying to run a Union query on the MYSQL database. The query is (much simplified) SELECT as ID from faxdata UNION SELECT as ID ...
20
by: Brian Tkatch | last post by:
An ORDER BY a simple-integer inside a FUNCTION, results in SQL0440N, unless the FUNCTION expects an INTEGER as its parameter. For example: DECLARE GLOBAL TEMPORARY TABLE A(A CHAR(1)) INSERT INTO...
3
by: mikes | last post by:
I have 2 separate queries, which effectively are the same except they draw data from separate tables. Both tables are (design-wise) identical, only the data is different. for each query, there are...
6
by: das | last post by:
Hello all, I have a table with thousands of rows and is in this format: id col1 col2 col3 col4 --- ------ ----- ------ ------ 1 nm 78 xyz pir 2 ...
0
by: rashmigaikwad | last post by:
Hi All, I need help in optimizing the query mentioned below: SELECT SUM(CASE WHEN PROD_TYP='HBRMC' AND INC_MULTIPLE < 2.50 AND LTV <= 0.75 OR LTV IS NULL THEN ADV_IN_QTR ELSE 0 END) ...
9
by: Emin | last post by:
Dear Experts, I have a fairly simple query in which adding a where clause slows things down by at least a factor of 100. The following is the slow version of the query ...
5
by: Merennulli | last post by:
To start with, I'll give a simplified overview of my data. BaseRecord (4mil rows, 25k in each Region) ID | Name | Region | etc OtherData (7.5mil rows, 1 or 2 per ID) ID | Type(1/2) | Data ...
0
by: Charles Arthur | last post by:
How do i turn on java script on a villaon, callus and itel keypad mobile phone
0
by: emmanuelkatto | last post by:
Hi All, I am Emmanuel katto from Uganda. I want to ask what challenges you've faced while migrating a website to cloud. Please let me know. Thanks! Emmanuel
0
BarryA
by: BarryA | last post by:
What are the essential steps and strategies outlined in the Data Structures and Algorithms (DSA) roadmap for aspiring data scientists? How can individuals effectively utilize this roadmap to progress...
1
by: nemocccc | last post by:
hello, everyone, I want to develop a software for my android phone for daily needs, any suggestions?
1
by: Sonnysonu | last post by:
This is the data of csv file 1 2 3 1 2 3 1 2 3 1 2 3 2 3 2 3 3 the lengths should be different i have to store the data by column-wise with in the specific length. suppose the i have to...
0
by: Hystou | last post by:
There are some requirements for setting up RAID: 1. The motherboard and BIOS support RAID configuration. 2. The motherboard has 2 or more available SATA protocol SSD/HDD slots (including MSATA, M.2...
0
marktang
by: marktang | last post by:
ONU (Optical Network Unit) is one of the key components for providing high-speed Internet services. Its primary function is to act as an endpoint device located at the user's premises. However,...
0
by: Hystou | last post by:
Most computers default to English, but sometimes we require a different language, especially when relocating. Forgot to request a specific language before your computer shipped? No problem! You can...
0
tracyyun
by: tracyyun | last post by:
Dear forum friends, With the development of smart home technology, a variety of wireless communication protocols have appeared on the market, such as Zigbee, Z-Wave, Wi-Fi, Bluetooth, etc. Each...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.