By using this site, you agree to our updated Privacy Policy and our Terms of Use. Manage your Cookies Settings.
435,346 Members | 2,331 Online
Bytes IT Community
+ Ask a Question
Need help? Post your question and get tips & solutions from a community of 435,346 IT Pros & Developers. It's quick & easy.

OLAP and/or data mining?

P: n/a
Hello,

If I wrote the next ebay (yes I know, yawn-snore) and I had a database
with 5 million auction items in it, what would be a really good
strategy to get a search done very quickly? Would it involve
something called OLAP and/or "data mining"? The only technology I am
familiar with is simply SQL Server databases with stored procedures.
I think I'd be guessing correctly and say that this technology simply
wouldn't be fast enough *on it's own* to do super fast queries against
massive amounts of data.

Any insights would be of great interest. Thanks.

-Frameworker.
Jul 20 '05 #1
Share this Question
Share on Google+
5 Replies


P: n/a

"Framework fan" <te**************@hotmail.com> wrote in message
news:f1**************************@posting.google.c om...
Hello,

If I wrote the next ebay (yes I know, yawn-snore) and I had a database
with 5 million auction items in it, what would be a really good
strategy to get a search done very quickly? Would it involve
something called OLAP and/or "data mining"? The only technology I am
familiar with is simply SQL Server databases with stored procedures.
I think I'd be guessing correctly and say that this technology simply
wouldn't be fast enough *on it's own* to do super fast queries against
massive amounts of data.
OLAP is exactly what you'd want.

Data mining may be useful post-auction to see who bought what, what things
need more promotion etc.

I don't know why you don't think SQL Server wouldn't be fast enough for a
small setup like 5 million auctions.

I have a database with 14 million rows that's probably as complex as an
auction database and most queries are a couple of seconds or less. (some
are longer because of some fairly complex floating point math that has to be
done along the way.)

I have another database that handles at least that many inserts per day w/o
a major problem. And this is all on 3+ y.o. equipment.

Give me a quad Xeon MP box and the right disk subsystem and this thing could
scream.


Any insights would be of great interest. Thanks.

-Frameworker.

Jul 20 '05 #2

P: n/a
"Greg D. Moore (Strider)" <mo*****@greenms.com> wrote in message
news:fC******************@twister.nyroc.rr.com...

"Framework fan" <te**************@hotmail.com> wrote in message
news:f1**************************@posting.google.c om...
Hello,

If I wrote the next ebay (yes I know, yawn-snore) and I had a database
with 5 million auction items in it, what would be a really good
strategy to get a search done very quickly? Would it involve
something called OLAP and/or "data mining"? The only technology I am
familiar with is simply SQL Server databases with stored procedures.

Using stored procedures as component building blocks
you can create all forms of data mining and analytical
processing applications.

I think I'd be guessing correctly and say that this technology simply
wouldn't be fast enough *on it's own* to do super fast queries against
massive amounts of data.
OLAP is exactly what you'd want.

Data mining may be useful post-auction to see who bought what, what things
need more promotion etc.

I don't know why you don't think SQL Server wouldn't be fast enough for a
small setup like 5 million auctions.

Seconded.

If you write your procs in the right manner, and maintain
the data integrity in your database then you should have
null performance problems.
I have a database with 14 million rows that's probably as complex as an
auction database and most queries are a couple of seconds or less. (some
are longer because of some fairly complex floating point math that has to be done along the way.)

I have another database that handles at least that many inserts per day w/o a major problem. And this is all on 3+ y.o. equipment.

Give me a quad Xeon MP box and the right disk subsystem and this thing could scream.


Any insights would be of great interest. Thanks.

-Frameworker.


Jul 20 '05 #3

P: n/a
Hi Greg,

I have one follow up question please:

I'd like to incorporate OLAP in to my design. Do I go ahead and
create a sensibly normalised relational database design, with many
dozens of stored procedures to query the tables in "the usual manner",
*then* add on OLAP as a kind of "bolt on", or does OLAP technology
require a totally different database/stored procedures design strategy
right from the word go?

Thanks!

- Frameworker.
"Greg D. Moore \(Strider\)" <mo*****@greenms.com> wrote in message news:<fC******************@twister.nyroc.rr.com>.. .
"Framework fan" <te**************@hotmail.com> wrote in message
news:f1**************************@posting.google.c om...
Hello,

If I wrote the next ebay (yes I know, yawn-snore) and I had a database
with 5 million auction items in it, what would be a really good
strategy to get a search done very quickly? Would it involve
something called OLAP and/or "data mining"? The only technology I am
familiar with is simply SQL Server databases with stored procedures.
I think I'd be guessing correctly and say that this technology simply
wouldn't be fast enough *on it's own* to do super fast queries against
massive amounts of data.


OLAP is exactly what you'd want.

Data mining may be useful post-auction to see who bought what, what things
need more promotion etc.

I don't know why you don't think SQL Server wouldn't be fast enough for a
small setup like 5 million auctions.

I have a database with 14 million rows that's probably as complex as an
auction database and most queries are a couple of seconds or less. (some
are longer because of some fairly complex floating point math that has to be
done along the way.)

I have another database that handles at least that many inserts per day w/o
a major problem. And this is all on 3+ y.o. equipment.

Give me a quad Xeon MP box and the right disk subsystem and this thing could
scream.


Any insights would be of great interest. Thanks.

-Frameworker.

Jul 20 '05 #4

P: n/a
I'm a speed freak! If I can get a query to run .25 seconds faster
using OLAP technology, I'm all for it. :o)
"mountain man" <hobbit@southern_seaweed.com.op> wrote in message news:<f3*****************@news-server.bigpond.net.au>...
"Greg D. Moore (Strider)" <mo*****@greenms.com> wrote in message
news:fC******************@twister.nyroc.rr.com...

"Framework fan" <te**************@hotmail.com> wrote in message
news:f1**************************@posting.google.c om...
Hello,

If I wrote the next ebay (yes I know, yawn-snore) and I had a database
with 5 million auction items in it, what would be a really good
strategy to get a search done very quickly? Would it involve
something called OLAP and/or "data mining"? The only technology I am
familiar with is simply SQL Server databases with stored procedures.

Using stored procedures as component building blocks
you can create all forms of data mining and analytical
processing applications.

I think I'd be guessing correctly and say that this technology simply
wouldn't be fast enough *on it's own* to do super fast queries against
massive amounts of data.


OLAP is exactly what you'd want.

Data mining may be useful post-auction to see who bought what, what things
need more promotion etc.

I don't know why you don't think SQL Server wouldn't be fast enough for a
small setup like 5 million auctions.

Seconded.

If you write your procs in the right manner, and maintain
the data integrity in your database then you should have
null performance problems.
I have a database with 14 million rows that's probably as complex as an
auction database and most queries are a couple of seconds or less. (some
are longer because of some fairly complex floating point math that has to

be
done along the way.)

I have another database that handles at least that many inserts per day

w/o
a major problem. And this is all on 3+ y.o. equipment.

Give me a quad Xeon MP box and the right disk subsystem and this thing

could
scream.


Any insights would be of great interest. Thanks.

-Frameworker.

Jul 20 '05 #5

P: n/a

"Framework fan" <te**************@hotmail.com> wrote in message
news:f1**************************@posting.google.c om...
I'm a speed freak! If I can get a query to run .25 seconds faster
using OLAP technology, I'm all for it. :o)
I think you completely misunderstand what OLAP is or does. (And I mispoke
below (forgive me, I was tired). You want an OLTP system, not an OLAP.

OLAP and getting better performance are orthogonal to each other. i.e.
using OLAP is not a way to increase performance.


"mountain man" <hobbit@southern_seaweed.com.op> wrote in message

news:<f3*****************@news-server.bigpond.net.au>...
"Greg D. Moore (Strider)" <mo*****@greenms.com> wrote in message
news:fC******************@twister.nyroc.rr.com...

"Framework fan" <te**************@hotmail.com> wrote in message
news:f1**************************@posting.google.c om...
> Hello,
>
> If I wrote the next ebay (yes I know, yawn-snore) and I had a database > with 5 million auction items in it, what would be a really good
> strategy to get a search done very quickly? Would it involve
> something called OLAP and/or "data mining"? The only technology I am > familiar with is simply SQL Server databases with stored procedures.

Using stored procedures as component building blocks
you can create all forms of data mining and analytical
processing applications.

> I think I'd be guessing correctly and say that this technology simply > wouldn't be fast enough *on it's own* to do super fast queries against > massive amounts of data.

OLAP is exactly what you'd want.

Data mining may be useful post-auction to see who bought what, what things need more promotion etc.

I don't know why you don't think SQL Server wouldn't be fast enough for a small setup like 5 million auctions.

Seconded.

If you write your procs in the right manner, and maintain
the data integrity in your database then you should have
null performance problems.
I have a database with 14 million rows that's probably as complex as an auction database and most queries are a couple of seconds or less. (some are longer because of some fairly complex floating point math that has
to be
done along the way.)

I have another database that handles at least that many inserts per
day w/o
a major problem. And this is all on 3+ y.o. equipment.

Give me a quad Xeon MP box and the right disk subsystem and this thing

could
scream.
>
> Any insights would be of great interest. Thanks.
>
> -Frameworker.

Jul 20 '05 #6

This discussion thread is closed

Replies have been disabled for this discussion.