dlite922 1,584
Recognized Expert Top Contributor
I'm building a dynamic reporting system. The report can of course query multiple tables.
The query declares which tables it will access, the fields that it needs for display, and the dependent fieds or fields not in the report, but in the "filter" part.
The problem is when a "filter" is not used, the field will not show up in the dynamically built SQL.
For example, the user doesn't choose to see a report for a particular product, but for all products, so therefore I don't constraint by the product field, but the product table is still included in FROM and joined.
When I run the query manually at the prompt, it takes a very long time to run (over 2min for 50 records) and when I take out the tables that are not used the query takes 0.03 seconds to run.
Is there any way to change this MySQL behavior? to complete ignore the tables that have no fields related to them, but are used in joins.
I'm trying to prevent coding so as to put the table names based on the criteria used.
Sorry can't give actuall query example because data is sensitive, but I can try to write out a similar query: -
-
/** SLOW **/
-
SELECT c.number, c.name FROM customer AS c, invoice AS i, ordreLine AS o WHERE c.id = i.customerID AND i.id = o.invoiceID;
-
-
/** FAST **/
-
SELECT c.number, c.name FROM customer AS c, invoice AS i, ordreLine AS o WHERE c.id = i.customerID AND i.id = o.invoiceID AND (i.date < 2008-02-14 AND o.qty > 5);
-
-
You can see, same tables, but when I use fields in those tables in the WHERE clause, it executes fast.
I hope i've made my problem clear, if not let me know.
Dan
3 2232
It seems that you have an index on i.date or o.qty, but you have no index on either i.id or o.invoiceID !
Just make sure that you have an index on all c.id, i.customerID, i.id and o.invoiceID and run again, ist should be much faster.
If you only have a one-to-one or one-to zero match for all tables, then you don't need to build the slow cartesian product, a fast left-join would be enough.
That means if you have only one customer record that you want to enrich with data from the invoice table if it is there or not as what I understand from your description (that means if c.id and i.customerID are unique values in each table), then instead of writing "customer AS c, invoice AS i WHERE c.id = i.customerID " you can use "customer AS c, invoice AS i left join on (c.id = i.customerID )" -
-
/** SLOW **/
-
SELECT c.number, c.name FROM customer AS c, invoice AS i, ordreLine AS o WHERE c.id = i.customerID AND i.id = o.invoiceID;
-
-
/** FAST **/
-
SELECT c.number, c.name FROM customer AS c, invoice AS i, ordreLine AS o WHERE c.id = i.customerID AND i.id = o.invoiceID AND (i.date < 2008-02-14 AND o.qty > 5);
-
-
You can see, same tables, but when I use fields in those tables in the WHERE clause, it executes fast.
Dan
dlite922 1,584
Recognized Expert Top Contributor
thanks for that clarification,
Since none of my queries are really this trivial, would a Left Join work this fast when i'm querying three to six tables?
I assume so, but I have to look at the manual on how to do multiple left joins.
Thanks again,
Dan
It seems that you have an index on i.date or o.qty, but you have no index on either i.id or o.invoiceID !
Just make sure that you have an index on all c.id, i.customerID, i.id and o.invoiceID and run again, ist should be much faster.
If you only have a one-to-one or one-to zero match for all tables, then you don't need to build the slow cartesian product, a fast left-join would be enough.
That means if you have only one customer record that you want to enrich with data from the invoice table if it is there or not as what I understand from your description (that means if c.id and i.customerID are unique values in each table), then instead of writing "customer AS c, invoice AS i WHERE c.id = i.customerID " you can use "customer AS c, invoice AS i left join on (c.id = i.customerID )"
thanks for that clarification,
Since none of my queries are really this trivial, would a Left Join work this fast when i'm querying three to six tables?
I assume so, but I have to look at the manual on how to do multiple left joins.
Thanks again,
Dan
Joining many tables is no performance problem in general if you made sure that the joined fields are all indexed. Also the order of the join is important. For example:
Table A has thousand records, table B and C one million. If you join B and C first , you might end up with ten-thousand records that match the join and would be looked up before matching with A, which results in 10 records at the end. Also finding all matches in the large index tables of B and C is very slow. This is very inefficient. If you would join A with B first, you would end up with 100 records only. The index table of A is small, so it is a fast match. Now only these 100 records needs to be matched with C afterwards which leads to 10 resulting records and is very efficient.
Sometimes the database can determine the best matching order automatically, but you can not rely on that. The best way is to write your query in a way which is already optimized.
Sign in to post your reply or Sign up for a free account.
Similar topics |
by: Tim Fountain |
last post by:
We've recently enabled slow query logging on a server and it's proving
interesting seeing which queries are bogging things down. This one is
puzzling me a little:
SELECT articleid, type, authorid, authorname, text, posted FROM
comments WHERE status = 'normal' ORDER BY posted DESC LIMIT 5;
The purpose of this query is to list the five most recent
(non-deleted) comments. Here is the table structure:
|
by: Jesse Sheidlower |
last post by:
I'm struggling with speed issues on some queries that
I would have expected to be relatively fast. Perhaps
even more frustratingly, when I've tried to break
these down into their components, they still execute
very slowly. I've looked over all the relevant suggestions
for optimization and so forth, and there's nothing I can
tell that I'm missing.
An example of a query is to get all the words (the cg.cw
field) in a particular...
|
by: DBNovice |
last post by:
I have a database that keeps records on the issue and failure of an item.
Currently, the database is poorly desisned; therefore I'm performing queries
to break the data into normalized tables and attempting to perform a "left
join" query to build a cross-reference table.
The left join query is currently taking nearly 2 hours for MySQL to process,
using Navicat as a front-end. My system specs are 1.4Mhz Pentium Processor
with 1GB of RAM...
|
by: Jason |
last post by:
The following stored procedure is taking too long (in my opinion). The
problem seems to be the SUM line. When commented out the query takes a
second or two. When included the response time climbs to minute and a
half.
Is my code that inefficient or is SUM and ABS calls just that slow?
Any suggestions to spead this up?
Thanks,
- Jason
|
by: Raghuraman |
last post by:
Hai
1. Is it possiable to delete a record from the parent table.It is
even ok to me , if it leads to the deletion of all the child tables .
2.I 've come across a situiation where the name of the table is to be
supplied by the variable in my sp ,like
| |
by: Rolan |
last post by:
There must be a way to enhance the performance of a query, or find a
plausible workaround, but I seem to be hitting a wall. I have tried a
few tweaks, however, there has been no improvement.
Simply, I'm including one calcualtion from a separate table/query. It
sums the total units sold to date by ProductID number and is used in
other select queries to perform various calculations. Perhaps there is
an advantage in working with a maximum...
|
by: mattytee123 |
last post by:
I have about 20 tables, of which I would like to do a union query and
count of how many of each different code there is?
The simplified verson of the table is structured like this.
Code Count
1234 1
2468 1
1234 1
2468 1
|
by: Rory Campbell-Lange |
last post by:
The following query on some small datasets takes over a second to run.
I'd be grateful for some help in understanding the explain output, and
to remake the code.
Looks like the sort is using up quite a bit of resources. I made an
index on boards using columns "b.n_type, b.n_id, b.t_name" but the index
was not recorded in explain analyze. (see "testindex" below).
I am using PostgreSQL 7.4.2 on i386-pc-linux-gnu (Debian). The query is...
|
by: rAinDeEr |
last post by:
Suppose i have a table which holds thousands of records with the
following structure
CREATE TABLE "test "."T_CNTRY" (
"CNTRY_CDE" CHAR(2) NOT NULL ,
"CNTRY_NAME" VARCHAR(50) )
and i have Created an index like below ::
|
by: marktang |
last post by:
ONU (Optical Network Unit) is one of the key components for providing high-speed Internet services. Its primary function is to act as an endpoint device located at the user's premises. However, people are often confused as to whether an ONU can Work As a Router. In this blog post, we’ll explore What is ONU, What Is Router, ONU & Router’s main usage, and What is the difference between ONU and Router. Let’s take a closer look !
Part I. Meaning of...
|
by: Hystou |
last post by:
Most computers default to English, but sometimes we require a different language, especially when relocating. Forgot to request a specific language before your computer shipped? No problem! You can effortlessly switch the default language on Windows 10 without reinstalling. I'll walk you through it.
First, let's disable language synchronization. With a Microsoft account, language settings sync across devices. To prevent any complications,...
| |
by: jinu1996 |
last post by:
In today's digital age, having a compelling online presence is paramount for businesses aiming to thrive in a competitive landscape. At the heart of this digital strategy lies an intricately woven tapestry of website design and digital marketing. It's not merely about having a website; it's about crafting an immersive digital experience that captivates audiences and drives business growth.
The Art of Business Website Design
Your website is...
|
by: Hystou |
last post by:
Overview:
Windows 11 and 10 have less user interface control over operating system update behaviour than previous versions of Windows. In Windows 11 and 10, there is no way to turn off the Windows Update option using the Control Panel or Settings app; it automatically checks for updates and installs any it finds, whether you like it or not. For most users, this new feature is actually very convenient. If you want to control the update process,...
|
by: tracyyun |
last post by:
Dear forum friends,
With the development of smart home technology, a variety of wireless communication protocols have appeared on the market, such as Zigbee, Z-Wave, Wi-Fi, Bluetooth, etc. Each protocol has its own unique characteristics and advantages, but as a user who is planning to build a smart home system, I am a bit confused by the choice of these technologies. I'm particularly interested in Zigbee because I've heard it does some...
|
by: agi2029 |
last post by:
Let's talk about the concept of autonomous AI software engineers and no-code agents. These AIs are designed to manage the entire lifecycle of a software development project—planning, coding, testing, and deployment—without human intervention. Imagine an AI that can take a project description, break it down, write the code, debug it, and then launch it, all on its own....
Now, this would greatly impact the work of software developers. The idea...
|
by: isladogs |
last post by:
The next Access Europe User Group meeting will be on Wednesday 1 May 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM).
In this session, we are pleased to welcome a new presenter, Adolph Dupré who will be discussing some powerful techniques for using class modules.
He will explain when you may want to use classes instead of User Defined Types (UDT). For example, to manage the data in unbound forms.
Adolph will...
|
by: TSSRALBI |
last post by:
Hello
I'm a network technician in training and I need your help.
I am currently learning how to create and manage the different types of VPNs and I have a question about LAN-to-LAN VPNs.
The last exercise I practiced was to create a LAN-to-LAN VPN between two Pfsense firewalls, by using IPSEC protocols.
I succeeded, with both firewalls in the same network. But I'm wondering if it's possible to do the same thing, with 2 Pfsense firewalls...
| |
by: 6302768590 |
last post by:
Hai team
i want code for transfer the data from one system to another through IP address by using C# our system has to for every 5mins then we have to update the data what the data is updated we have to send another system
| |