handling large volume of data

[Osama Karim]

Hi There,

I have 600 million records to be processes using string opearion. Can ORACLE handle this if yes what would be the response time.

Regards,
Osama K.

[amitpatel66]

Yes Oracle will handle the process, either STRING operation or any other. Well the Performance can Vary based on many factors like:

1. The way Query is written
2. S/W,H/W,Network issues
3. U need to check the indexes that u have for the table and use the correct one

So probably what u do is write a Query and try executing it. Please do post back if u have any performance issue with the Query.

We would be able to guide u in better way!!

[Osama Karim]

Yes Oracle will handle the process, either STRING operation or any other. Well the Performance can Vary based on many factors like:

1. The way Query is written
2. S/W,H/W,Network issues
3. U need to check the indexes that u have for the table and use the correct one

So probably what u do is write a Query and try executing it. Please do post back if u have any performance issue with the Query.

We would be able to guide u in better way!!

------------------------------------------------------------------------------------------------------------------

Thanks for the reply. let me give you a better picture. The records are basically url which needs to be categorized using lookup which has 33000 records.

For e.g. The URL http://www.aol.com/ belongs to the "HomePage" category, while http://webmail1.webmail.aol.com/25698/aol/en-us/Ma belongs to the "Email" Category

However, this assignment of URLs to category is based on a "Dictionary" which is having 33000 records. Note that the URL matching is NOT one-to-one, but is based on a pattern match. For eg. the pattern for "EMail" category is "%webmail.aol.com%". Because of this patterns, the lookup has to be done using a "like" operator, and NOT a "=" operator.

Please suggest.

Regards,
Osama K.

[amitpatel66]

------------------------------------------------------------------------------------------------------------------

Thanks for the reply. let me give you a better picture. The records are basically url which needs to be categorized using lookup which has 33000 records.

For e.g. The URL http://www.aol.com/ belongs to the "HomePage" category, while http://webmail1.webmail.aol.com/25698/aol/en-us/Ma belongs to the "Email" Category

However, this assignment of URLs to category is based on a "Dictionary" which is having 33000 records. Note that the URL matching is NOT one-to-one, but is based on a pattern match. For eg. the pattern for "EMail" category is "%webmail.aol.com%". Because of this patterns, the lookup has to be done using a "like" operator, and NOT a "=" operator.

Please suggest.

Regards,
Osama K.

Well in case if the data is huge, then the Query execution time will be a problem. Probably Instead of using LIKE operator, try using INSTR and check if the execution time decreases.

SELECT 1 from dual where INSTR('http://webmail1.webmail.aol.com/25698/aol/en-us/Ma ','webmail.aol.com') > 0

And yes, you can also create an Index on the table and use that which would fasten the operation for sure. Check whether FUNCTION INDEX would be helpful.

[Osama Karim]

Well in case if the data is huge, then the Query execution time will be a problem. Probably Instead of using LIKE operator, try using INSTR and check if the execution time decreases.

SELECT 1 from dual where INSTR('http://webmail1.webmail.aol.com/25698/aol/en-us/Ma ','webmail.aol.com') > 0

And yes, you can also create an Index on the table and use that which would fasten the operation for sure. Check whether FUNCTION INDEX would be helpful.

Thanks Amit Will try something like that. Will get back to you if needed.

Bye

[Osama Karim]

Thanks Amit Will try something like that. Will get back to you if needed.

Bye

Hi,

tried this but without luck. Would request if you have any other idea on the same.

Thanx