Hi,
I have a table that contains log data, usually around a million records. The
table has about 10 columns with various attributes of the logged data,
nothing special. We're using SQL Server 2000.
Some of the columns (for example "category") have duplicate values
throughout the records. We have a web page that queries the table to show
all the unique columns, for example:
select distinct CATEGORY from table TEST
Obviously the server has to scan all rows in order to get all unique columns
which takes quite a while, especially since that web page contains several
of these types of queries. We also have a MAX(DATE) and MIN(DATE) query that
also add to the load.
I already created indexes on the CATEGORY (actually on all categories)
column which might help a little but I'm pretty sure that there has got to
be a better way.
I also create a view (select distinct CATEGORY from table TEST) and tried to
index it, but it won't let me index a query that contains a DISTINCT
statement.
Isn't there a way to create an index that contains only the distinct values?
Is there another way to speed this up?
Thanks for any hints! 4 6105
"Florian" <RE************ **********@gmx. net> wrote in message
news:_c******** **********@news read2.news.pas. earthlink.net.. . Hi,
I have a table that contains log data, usually around a million records.
The table has about 10 columns with various attributes of the logged data, nothing special. We're using SQL Server 2000.
Some of the columns (for example "category") have duplicate values throughout the records. We have a web page that queries the table to show all the unique columns, for example:
select distinct CATEGORY from table TEST
Obviously the server has to scan all rows in order to get all unique
columns which takes quite a while, especially since that web page contains several of these types of queries. We also have a MAX(DATE) and MIN(DATE) query
that also add to the load.
I already created indexes on the CATEGORY (actually on all categories) column which might help a little but I'm pretty sure that there has got to be a better way.
I also create a view (select distinct CATEGORY from table TEST) and tried
to index it, but it won't let me index a query that contains a DISTINCT statement.
Isn't there a way to create an index that contains only the distinct
values? Is there another way to speed this up?
Thanks for any hints!
If only you load/update the data relatively infrequently, then you could
create a 'lookup' table for each attribute, and populate them from the main
table after loading it (rather like the dimensions in a star schema):
insert into dbo.Categories (Category)
select distinct Category
from dbo.Test
Your client code could then query the lookup tables instead of the log
table. If you want to use indexed views, then you could create a view like
this:
create view dbo.Categories
with schemabinding
as
select Category, count_big(*) as 'Occurrences'
from dbo.Test
group by Category
That should be indexable, although there are quite a few other restrictions,
so you would need to check them out. But having multiple indexed views on
the table would make data modifications much slower, so if the data changes
frequently you might have to use some sort of lookup table approach anyway.
Simon
> select distinct CATEGORY from table TEST
Try experimenting with group by instead of distinct. Also in query
analyzer, enable view execution plan, to get an idea what mssql is
doing. "select category from mytable group by category"
"Simon Hayes" <sq*@hayes.ch > wrote in message
news:40******** **@news.bluewin .ch... "Florian" <RE************ **********@gmx. net> wrote in message news:_c******** **********@news read2.news.pas. earthlink.net.. . Hi,
I have a table that contains log data, usually around a million records. The table has about 10 columns with various attributes of the logged data, nothing special. We're using SQL Server 2000.
Some of the columns (for example "category") have duplicate values throughout the records. We have a web page that queries the table to
show all the unique columns, for example:
select distinct CATEGORY from table TEST
Obviously the server has to scan all rows in order to get all unique columns which takes quite a while, especially since that web page contains
several of these types of queries. We also have a MAX(DATE) and MIN(DATE) query that also add to the load.
I already created indexes on the CATEGORY (actually on all categories) column which might help a little but I'm pretty sure that there has got
to be a better way.
I also create a view (select distinct CATEGORY from table TEST) and
tried to index it, but it won't let me index a query that contains a DISTINCT statement.
Isn't there a way to create an index that contains only the distinct values? Is there another way to speed this up?
Thanks for any hints!
If only you load/update the data relatively infrequently, then you could create a 'lookup' table for each attribute, and populate them from the
main table after loading it (rather like the dimensions in a star schema):
insert into dbo.Categories (Category) select distinct Category from dbo.Test
Your client code could then query the lookup tables instead of the log table. If you want to use indexed views, then you could create a view like this:
create view dbo.Categories with schemabinding as select Category, count_big(*) as 'Occurrences' from dbo.Test group by Category
That should be indexable, although there are quite a few other
restrictions, so you would need to check them out. But having multiple indexed views on the table would make data modifications much slower, so if the data
changes frequently you might have to use some sort of lookup table approach
anyway.
Thanks, I tried the indexed view and that seems to work OK now, pretty
fast - can't complain. Data shouldn't be updated that often so that should
be OK. Otherwise I might have to go with a lookup table - but it's not a
real good solution for our scenario for reasons I'm not going to bore
anybody with :)
Thanks!
"louis nguyen" <lo************ @hotmail.com> wrote in message
news:b0******** *************** **@posting.goog le.com... select distinct CATEGORY from table TEST Try experimenting with group by instead of distinct. Also in query analyzer, enable view execution plan, to get an idea what mssql is doing. "select category from mytable group by category"
Yes, the group thing worked great - I also analyzed the execution plan and
now it's only returning the actual number of rows I'm getting - not the
whole table anymore.
Thanks. This thread has been closed and replies have been disabled. Please start a new discussion. Similar topics |
by: Abby Lee |
last post by:
1st sorry about leangth...couldn't really cut anymore.
I want the output to be
Organization 320000
Fund 100004
Program 777777
Account1 7234.55
Account2 -347.99
Account3 823.55
|
by: Kelvin |
last post by:
Okay so this is baking my noodle. I want to select all the
attritbutes/fields from a table but then to excluded any row in which
a single attributes data has been duplicated.
I.E. Here's my table:-
ID Ref Name DATE
1 AAA Joe 1/2
2 BBB Ken 1/2
3 AAA Len 6/3
|
by: keith |
last post by:
I am trying to get a exact count of different distinct entries in an
Access column. At first, I was trying to work with three columns, but
I've narrowed it down to one to simplify it. I've searched Google
Groups for Distinct Count and Count, the Microsoft Help file (which
apparently has bad links in Office 2003), and looked at other files,
but I can't find the answer. The closest I've been able to get is to
create a query to find the...
|
by: mathilda |
last post by:
My boss has been adamant that SELECT DISTINCT is a faster query than
SELECT all other factors being equal. I disagree. We are linking an
Access front end to a SQL Server back end and normally are only
returning one record. She states that with disctinct the query stops
as soon as it finds a matching record. Both of us are relative novices
in database theory (obviously).
Can someone help settle this?
|
by: Johnathan Doe |
last post by:
I can google search to find the range of values that can be represented
in a float by reading up on the IEEE std, but is that the same as how
many distinct values that can go in a float type?
For instance, floats can distinguish 0.000001 and 0.000002. If I
started with 0.000001 and kept adding 0.000001 until I hit some maximum
value (FLT_MAX?) and kept a counter of how many times I added 0.000001,
would I have a count of how many...
| |
by: @sh |
last post by:
This may be a really simple question, but I always have problems with
Distinct queries.
In this instance, I have a table of Delivery addresses, some will be exactly
the same EXCEPT for the DeliveryID field, but eitherway I need the
DeliveryID so that I can pass it onto the next form.
So, I need to run a query that will find only addresses that are unique
whilst retaining the respective DID for the rows...
|
by: andrewrubie |
last post by:
Hi All,
I have built a search page(asp) in dreamweaver for a friend with a used
records store and website. The results page lists all recordings their
database(ms access 2002) holds with(or similar to) the desired title
entered by the user. The database uses a many to many relationship
between tblArtists, tblLINKArtist_Recording and tblRecordings to allow
every artist to have many recordings associated with them and every
recording to...
|
by: osmarjunior |
last post by:
Hi. I made a method to execute queries in my database (Firebird
embedded v.1.5.3).
The Database class contains methods to create the database specific
objects.
public static DbDataReader ExecuteReader(String query)
{
// Creates the DbCommand
DbCommand command = Database.GetCommand();
|
by: Bill |
last post by:
I'm trying to write a query that will select a distinct count of more
than one field. I have records that display user productivity. Each of
the records have a time associated with it and I want to be able to
tell the distinct count of the products numbers and of the pallets ID
that they worked with. Do need to work out some sort of sub-query to do
this?
|
by: marktang |
last post by:
ONU (Optical Network Unit) is one of the key components for providing high-speed Internet services. Its primary function is to act as an endpoint device located at the user's premises. However, people are often confused as to whether an ONU can Work As a Router. In this blog post, we’ll explore What is ONU, What Is Router, ONU & Router’s main usage, and What is the difference between ONU and Router. Let’s take a closer look !
Part I. Meaning of...
|
by: Hystou |
last post by:
Most computers default to English, but sometimes we require a different language, especially when relocating. Forgot to request a specific language before your computer shipped? No problem! You can effortlessly switch the default language on Windows 10 without reinstalling. I'll walk you through it.
First, let's disable language synchronization. With a Microsoft account, language settings sync across devices. To prevent any complications,...
| |
by: Hystou |
last post by:
Overview:
Windows 11 and 10 have less user interface control over operating system update behaviour than previous versions of Windows. In Windows 11 and 10, there is no way to turn off the Windows Update option using the Control Panel or Settings app; it automatically checks for updates and installs any it finds, whether you like it or not. For most users, this new feature is actually very convenient. If you want to control the update process,...
|
by: agi2029 |
last post by:
Let's talk about the concept of autonomous AI software engineers and no-code agents. These AIs are designed to manage the entire lifecycle of a software development project—planning, coding, testing, and deployment—without human intervention. Imagine an AI that can take a project description, break it down, write the code, debug it, and then launch it, all on its own....
Now, this would greatly impact the work of software developers. The idea...
|
by: conductexam |
last post by:
I have .net C# application in which I am extracting data from word file and save it in database particularly. To store word all data as it is I am converting the whole word file firstly in HTML and then checking html paragraph one by one.
At the time of converting from word file to html my equations which are in the word document file was convert into image.
Globals.ThisAddIn.Application.ActiveDocument.Select();...
|
by: TSSRALBI |
last post by:
Hello
I'm a network technician in training and I need your help.
I am currently learning how to create and manage the different types of VPNs and I have a question about LAN-to-LAN VPNs.
The last exercise I practiced was to create a LAN-to-LAN VPN between two Pfsense firewalls, by using IPSEC protocols.
I succeeded, with both firewalls in the same network. But I'm wondering if it's possible to do the same thing, with 2 Pfsense firewalls...
|
by: adsilva |
last post by:
A Windows Forms form does not have the event Unload, like VB6. What one acts like?
|
by: 6302768590 |
last post by:
Hai team
i want code for transfer the data from one system to another through IP address by using C# our system has to for every 5mins then we have to update the data what the data is updated we have to send another system
| |
by: bsmnconsultancy |
last post by:
In today's digital era, a well-designed website is crucial for businesses looking to succeed. Whether you're a small business owner or a large corporation in Toronto, having a strong online presence can significantly impact your brand's success. BSMN Consultancy, a leader in Website Development in Toronto offers valuable insights into creating effective websites that not only look great but also perform exceptionally well. In this comprehensive...
| |