I have only 1m records in my database running on a laptop of speed
1.6GHz, memory 512MB ram, and Toshiba MK8032GAX hard disk.
I use 'LIMIT x,10' for the query to utilise record paging.
When the value of x is nearer to 0, the query speed is fast.
Presently, 'LIMIT 0,10', 'LIMIT 50000,10' and 'LIMIT 100000,10' takes
about 0sec, 11secs and 4mins respectively.
1) Is there anything I should do, in terms of sql statement or database
design, to obtain the same speed for cases where x is in the middle or
nearer to the end?
2) How fast is a simple select statement (not select count(*)) on a
system like mine?
3) Suppose I left my original select .. limit query as is, what is the
minimum hardware that I need to improve the speed?
A million thanks. 5 4743
On Thu, 19 Oct 2006 17:55:25 +0800, in mailing.databas e.mysql -
<no****@home.co m>
<45********@new s.starhub.net.s gwrote:
>| I have only 1m records in my database running on a laptop of speed | 1.6GHz, memory 512MB ram, and Toshiba MK8032GAX hard disk. | | I use 'LIMIT x,10' for the query to utilise record paging. | When the value of x is nearer to 0, the query speed is fast. | Presently, 'LIMIT 0,10', 'LIMIT 50000,10' and 'LIMIT 100000,10' takes | about 0sec, 11secs and 4mins respectively. | | 1) Is there anything I should do, in terms of sql statement or database | design, to obtain the same speed for cases where x is in the middle or | nearer to the end? | | 2) How fast is a simple select statement (not select count(*)) on a | system like mine? | | 3) Suppose I left my original select .. limit query as is, what is the | minimum hardware that I need to improve the speed? | | A million thanks.
Use Explain [your query goes here]
then look at the results. Which table(s) are accessed most frequently
during the queries execution.
Can some of the fields be indexed.
Are the joins (if any) optimised.
How much data are you bringing through (specifying 20 fields but only
displaying 2).
--------------------------------------------------------------- jn******@yourpa ntsyahoo.com.au : Remove your pants to reply
---------------------------------------------------------------
Use Explain [your query goes here]
then look at the results. Which table(s) are accessed most frequently
during the queries execution.
Can some of the fields be indexed.
Are the joins (if any) optimised.
How much data are you bringing through (specifying 20 fields but only
displaying 2).
--------------------------------------------------------------- jn******@yourpa ntsyahoo.com.au : Remove your pants to reply
---------------------------------------------------------------
My query is a very simple one.
SELECT id FROM foobar WHERE name = 'bla bla' LIMIT X,Y;
Where X,Y are constants.
There are only two fields, a primary key and an indexed field.
On Thu, 19 Oct 2006 23:19:35 +0800, in mailing.databas e.mysql -
<no****@home.co m>
<45********@new s.starhub.net.s gwrote:
>| | Use Explain [your query goes here] | then look at the results. Which table(s) are accessed most frequently | during the queries execution. | Can some of the fields be indexed. | Are the joins (if any) optimised. | How much data are you bringing through (specifying 20 fields but only | displaying 2). | --------------------------------------------------------------- | jn******@yourpa ntsyahoo.com.au : Remove your pants to reply | --------------------------------------------------------------- | | | My query is a very simple one. | | SELECT id FROM foobar WHERE name = 'bla bla' LIMIT X,Y; | | Where X,Y are constants. | | There are only two fields, a primary key and an indexed field.
Well you've got me.
A couple of things though.
You don't say what OS you're using. Shutdown your mySQL server and
defrag your system. Maybe the files are all over the place and mySQL
has to do extra work to retrieve the data.
Run your query again - any noticable difference
What table type are you using myISAM or INNODB?
You could try create a copy of your table (minus the PK and index)
then import the data from the orginal table to the duplicate. Then add
your PK and index. I'm thinking that the indices maybe in need of a
freshen up. Try your query on the duplicate table, is there any speed
difference? Also my adding the PK and index fields later this will
give you an indication of the time required for mySQL to rebuild the
data - handy for when you drop pk and drop index/create pk/index.
What else is running on your system? Is there a lot of disk thrashing
when you are doing the slow query. Maybe the system is running out of
resources and needs to do a lot of memory swapping.
--------------------------------------------------------------- jn******@yourpa ntsyahoo.com.au : Remove your pants to reply
---------------------------------------------------------------
Jeff North wrote:
>
Well you've got me.
A couple of things though.
You don't say what OS you're using. Shutdown your mySQL server and
defrag your system. Maybe the files are all over the place and mySQL
has to do extra work to retrieve the data.
Run your query again - any noticable difference
What table type are you using myISAM or INNODB?
You could try create a copy of your table (minus the PK and index)
then import the data from the orginal table to the duplicate. Then add
your PK and index. I'm thinking that the indices maybe in need of a
freshen up. Try your query on the duplicate table, is there any speed
difference? Also my adding the PK and index fields later this will
give you an indication of the time required for mySQL to rebuild the
data - handy for when you drop pk and drop index/create pk/index.
What else is running on your system? Is there a lot of disk thrashing
when you are doing the slow query. Maybe the system is running out of
resources and needs to do a lot of memory swapping.
Suppose everything is in order, how long will such query on a two-field
table of only 1m records takes on windows xp, 1.6Ghz, 512mb?
SELECT id FROM foobar WHERE name = 'bla bla' LIMIT X,Y;
where X and Y are constants.
Thank you, Jeff.
On Sat, 21 Oct 2006 09:01:38 +0800, in mailing.databas e.mysql -
<no****@home.co m>
<45******@news. starhub.net.sgw rote:
>| Jeff North wrote: | > | Well you've got me. | A couple of things though. | You don't say what OS you're using. Shutdown your mySQL server and | defrag your system. Maybe the files are all over the place and mySQL | has to do extra work to retrieve the data. | Run your query again - any noticable difference | > | What table type are you using myISAM or INNODB? | You could try create a copy of your table (minus the PK and index) | then import the data from the orginal table to the duplicate. Then add | your PK and index. I'm thinking that the indices maybe in need of a | freshen up. Try your query on the duplicate table, is there any speed | difference? Also my adding the PK and index fields later this will | give you an indication of the time required for mySQL to rebuild the | data - handy for when you drop pk and drop index/create pk/index. | > | What else is running on your system? Is there a lot of disk thrashing | when you are doing the slow query. Maybe the system is running out of | resources and needs to do a lot of memory swapping. | | Suppose everything is in order, how long will such query on a two-field | table of only 1m records takes on windows xp, 1.6Ghz, 512mb? | | SELECT id FROM foobar WHERE name = 'bla bla' LIMIT X,Y; | where X and Y are constants. | | Thank you, Jeff.
Shouldn't take upto 4 minutes.
Have your tried:
EXPLAIN SELECT id FROM foobar WHERE name = 'bla bla' LIMIT X,Y;
or
DESCRIBE SELECT id FROM foobar WHERE name = 'bla bla' LIMIT X,Y;
they might be able to show you the bottleneck.
--------------------------------------------------------------- jn******@yourpa ntsyahoo.com.au : Remove your pants to reply
--------------------------------------------------------------- This thread has been closed and replies have been disabled. Please start a new discussion. Similar topics |
by: Steve |
last post by:
I've run in to a problem with a query I'm trying to write. I have attached a
sample SQL script at the end of this post to show an overview of what I'm
working with.
I want to be able to use LIMIT to control how many rows from one table are
returned, independent of how many rows there are in a second table that is
joined to the first.
When I execute the following SQL script:
|
by: Mike |
last post by:
New to PHP and MySQL.
Using PHP5 and MySQL 4.1
Windows XP Pro
IIS 5.1
I'm trying to page a recordset, and am using a LIMIT clause to fetch a
defined range of records from my db. However, the returned dataset is not
limited to the range I have in the SQL clause.
Here's the code:
|
by: emily_g107 |
last post by:
Hi,
I need to limit results in the following query type:
http://www.somewhere.com/php/sql-a.php3?server=1&db=mydatabase&table=mytable&sql_query=SELECT+Field_1%2CField_2%2CField_3%2Cidno+from+mytable+where+1+and+field_1+like+%22string%22+&sql_order=&pos=1
I found a reference that says I should be able to use LIMIT x, but
I don't know where/exactly how to add that to the string. Once I know
what it's supposed to look like, and can...
|
by: D. Dante Lorenso |
last post by:
I need to know that original number of rows that WOULD have been returned
by a SELECT statement if the LIMIT / OFFSET where not present in the
statement.
Is there a way to get this data from PG ?
SELECT
... ;
----> returns 100,000 rows
|
by: elein |
last post by:
Yes, I vacuumed. Reproduced on both 7.3.2 and 7.5.
Brain dead java beans want order by clauses in views
that they use.
my view is:
select ....
from bigtable b left join lookuptable l
order by bigkey desc;
| |
by: Larry Neylon |
last post by:
Hi,
I'm working on a VBScript application on IIS6 and I'm looking for some
advice about the best way of replacing or improving session variable usage.
The application is in a secure extranet environment. Currently the
application has a search customers page with 10 search fields which list the
results below the search fields. The requirement for this screen was that
the user could return to this result page at any point from any page...
|
by: lawrence k |
last post by:
Want to replace the limit clause in a query, but can't get it right.
What's wrong with this:
$pattern = "(.*)limit (.*)";
$replacement = '$1';
$replacement .= "LIMIT $limit";
$replacement .= '$2';
$query = preg_replace ($pattern, $replacement, $query);
|
by: supsupmo |
last post by:
Let's say I have a table, data_tbl, with the following columns:
object_id of type integer
mytimestamp of type timestamp with time zone
data_1 of type integer
data_2 of type integer
And each object within this table gets a new record every second. So if there are 4 different object_ids, then 4 new records are inserted each second.
Now, my problem is bad performance on a particular query. Given an object_id, this query retrieves x...
|
by: burtonfigg |
last post by:
I've read this article:
http://www.oracle.com/technology/pub/articles/oracle_php_cookbook/ullman-ajax.html
And from reading it thought it'd be interesting to try a bit of Ajax on my site.
What I'd really like to do is to set up a little ajax widget on my site, so that if people want to view more of the ecard thumbnails, they can click little next/prev icons to view the next / previous 8 thumbnail ecard images. I suppose in a way I want...
|
by: marktang |
last post by:
ONU (Optical Network Unit) is one of the key components for providing high-speed Internet services. Its primary function is to act as an endpoint device located at the user's premises. However, people are often confused as to whether an ONU can Work As a Router. In this blog post, we’ll explore What is ONU, What Is Router, ONU & Router’s main usage, and What is the difference between ONU and Router. Let’s take a closer look !
Part I. Meaning of...
|
by: Hystou |
last post by:
Most computers default to English, but sometimes we require a different language, especially when relocating. Forgot to request a specific language before your computer shipped? No problem! You can effortlessly switch the default language on Windows 10 without reinstalling. I'll walk you through it.
First, let's disable language synchronization. With a Microsoft account, language settings sync across devices. To prevent any complications,...
| |
by: tracyyun |
last post by:
Dear forum friends,
With the development of smart home technology, a variety of wireless communication protocols have appeared on the market, such as Zigbee, Z-Wave, Wi-Fi, Bluetooth, etc. Each protocol has its own unique characteristics and advantages, but as a user who is planning to build a smart home system, I am a bit confused by the choice of these technologies. I'm particularly interested in Zigbee because I've heard it does some...
|
by: agi2029 |
last post by:
Let's talk about the concept of autonomous AI software engineers and no-code agents. These AIs are designed to manage the entire lifecycle of a software development project—planning, coding, testing, and deployment—without human intervention. Imagine an AI that can take a project description, break it down, write the code, debug it, and then launch it, all on its own....
Now, this would greatly impact the work of software developers. The idea...
|
by: isladogs |
last post by:
The next Access Europe User Group meeting will be on Wednesday 1 May 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM).
In this session, we are pleased to welcome a new presenter, Adolph Dupré who will be discussing some powerful techniques for using class modules.
He will explain when you may want to use classes instead of User Defined Types (UDT). For example, to manage the data in unbound forms.
Adolph will...
|
by: conductexam |
last post by:
I have .net C# application in which I am extracting data from word file and save it in database particularly. To store word all data as it is I am converting the whole word file firstly in HTML and then checking html paragraph one by one.
At the time of converting from word file to html my equations which are in the word document file was convert into image.
Globals.ThisAddIn.Application.ActiveDocument.Select();...
|
by: TSSRALBI |
last post by:
Hello
I'm a network technician in training and I need your help.
I am currently learning how to create and manage the different types of VPNs and I have a question about LAN-to-LAN VPNs.
The last exercise I practiced was to create a LAN-to-LAN VPN between two Pfsense firewalls, by using IPSEC protocols.
I succeeded, with both firewalls in the same network. But I'm wondering if it's possible to do the same thing, with 2 Pfsense firewalls...
|
by: adsilva |
last post by:
A Windows Forms form does not have the event Unload, like VB6. What one acts like?
| |
by: 6302768590 |
last post by:
Hai team
i want code for transfer the data from one system to another through IP address by using C# our system has to for every 5mins then we have to update the data what the data is updated we have to send another system
| | |