473,763 Members | 9,275 Online
Bytes | Software Development & Data Engineering Community
+ Post

Home Posts Topics Members FAQ

reduce time for search query

Hi,

I have a task at hand to reduce the time taken for search query to
execute. The query fetches records which will have to sorted by
degrees away from the logged in user. I have a function which
calculates the degrees, but using this in the search query slows the
execution and takes about 10 secs to complete which is unacceptable.

Please advice. Your help is much appreciated

For more details plz see:

http://www.sqlteam.com/forums/topic.asp?TOPIC_ID=97021
Thanks
Isfaar

Feb 26 '08 #1
6 3286
On Tue, 26 Feb 2008 04:41:24 -0800 (PST), pa*******@googl email.com
wrote:
Hi,

I have a task at hand to reduce the time taken for search query to
execute. The query fetches records which will have to sorted by
degrees away from the logged in user. I have a function which
calculates the degrees, but using this in the search query slows the
execution and takes about 10 secs to complete which is unacceptable.

Please advice. Your help is much appreciated

For more details plz see:

http://www.sqlteam.com/forums/topic.asp?TOPIC_ID=97021
Hi Isfaar,

I see lots of useful (and some less useful) idea posted to that topic
over at sqlteam.com. Did you already try them? Did they help?

--
Hugo Kornelis, SQL Server MVP
My SQL Server blog: http://sqlblog.com/blogs/hugo_kornelis
Feb 26 '08 #2
On Feb 27, 3:22 am, Hugo Kornelis
<h...@perFact.R EMOVETHIS.info. INVALIDwrote:
On Tue, 26 Feb 2008 04:41:24 -0800 (PST), paankh...@googl email.com
wrote:
Hi,
I have a task at hand to reduce the time taken for search query to
execute. The query fetches records which will have to sorted by
degrees away from the logged in user. I have a function which
calculates the degrees, but using this in the search query slows the
execution and takes about 10 secs to complete which is unacceptable.
Please advice. Your help is much appreciated
For more details plz see:
http://www.sqlteam.com/forums/topic.asp?TOPIC_ID=97021

Hi Isfaar,

I see lots of useful (and some less useful) idea posted to that topic
over at sqlteam.com. Did you already try them? Did they help?

--
Hugo Kornelis, SQL Server MVP
My SQL Server blog:http://sqlblog.com/blogs/hugo_kornelis
Yes we have tried most of them, however, we did not get the required
results.

We are at present trying out to have a separate table to store all
relationships between the members, then query this table to get
results.

Will update our findings.

--
Feb 27 '08 #3
On Feb 26, 7:41*am, paankh...@googl email.com wrote:
*Hi,

I have a task at hand to reduce the time taken for search query to
execute. The query fetches records which will have to sorted by
degrees away from the logged in user. I have a function which
calculates the degrees, but using this in the search query slows the
execution and takes about 10 secs to complete which is unacceptable.

Please advice. Your help is much appreciated

For more details plz see:

http://www.sqlteam.com/forums/topic.asp?TOPIC_ID=97021

Thanks
Isfaar
Hi Isfaar,
Would it be possible for you to post your query here along with udf
definition. When a scalar UDF is invoked for each row in a query, it
can degrade the performance i.e. for each row you incur UDF invocation
cost. An inline expression might be faster than UDF in this case.

Depending on your SQL Server version, you might have to adopt
different approach. If you are on SQL Server 2005, then cross apply
operator in conjuction with inline table function will do the trick
for you. Inline table function behaves differently than scalar
function.

If you are on SQL Server 2000, a precisely written subquery might do
the trick.

HTH,

Najm
Feb 27 '08 #4
On Wed, 27 Feb 2008 02:11:16 -0800 (PST), ja*******@gmail .com wrote:
>On Feb 27, 3:22 am, Hugo Kornelis
<h...@perFact. REMOVETHIS.info .INVALIDwrote:
>I see lots of useful (and some less useful) idea posted to that topic
over at sqlteam.com. Did you already try them? Did they help?
(...)
>Yes we have tried most of them, however, we did not get the required
results.
Hi jan.afzal,

There is at least one you did not try yet. Posted by Peso:

"Yes. Why don't you give us the present code for review?
Or do you want us to guess?"

Najm already posted a similar request here. I really can't add anything
to what is suggested in that thread without knowing the actual code.

--
Hugo Kornelis, SQL Server MVP
My SQL Server blog: http://sqlblog.com/blogs/hugo_kornelis
Feb 27 '08 #5
(ja*******@gmai l.com) writes:
Sorry for not having to replied earlier.

here is the code.

this is the query that gets fired;
Most likely what is killing you is all the calls to
fnCommonFriends Step. (But if the vw_search view is complex, the reason
may be found there.)

I see that Plamen has offered a rewrite where he
has turned the function into a table-valued function, but will I have
to admit that I am skeptical that this will have any particular effect.

If you are lucky that these conditions:

WHERE U.UserID IN (SELECT DISTINCT UserID FROM vw_search)
AND UP.PropertyDefi nitionID = 29

filter away a major share of the rows, say 90%, it may be suffcient
to do:

INSERT #temp(UserID, ....)
SELECT U.UserID, U.FirstName + ' ' + U.LastName as Name,
UP.PropertyValu e as Location,
pI.headline as Headline,
i.industryName as Industry,
pI.summary as Summary,
pI.interests
FROM Users U
JOIN UserProfile ON U.UserID = UP.UserID
JOIN professionalInf o pI ON U.UserID = pI.memberId
JOIN industries i ON pI.primaryIndus try = i.industryId
WHERE U.UserID IN (SELECT DISTINCT UserID FROM vw_search)
AND UP.PropertyDefi nitionID = 29

SELECT UserID, Name, Location, Headline, Industry, Summary, interests,
dbo.GetConnecti onsCount(U.User ID) AS Connections,
dbo.GetRecommen dations(U.UserI D) AS Recommendations ,
dbo.fnCommonFri endsStep(U.User ID, 36) AS Degree
FROM #temp
WHERE dbo.fnCommonFri endsStep(U.User ID, 36) >= 0

But if the call to dbo.fnCommonFri endsStep is the major filter, the above
is useless.

It is possible that you could replace the function with a recursive CTE.
No, I am not go to give you a sample, because I don't know your tables,
I don't know your business rules, and I don't have any sample data to
test with. And there are some unfortuate restrictions with recursive
CTEs which makes me uncertain that they can actually do the job.

If that does not help, the only remaining option is to materialise the
result of fnCommonFriends Step to a table with the columns (User1, User2,
Degrees). How to maintain that table when a row is added, deleted or
update in the network table would be a new headache.

In summary, while we are some people out here that knows SQL Server well,
our expertise in the product as such is not sufficient to solve a
performance problem like this. We also need specific problem about
the problem at hand:

o What is the purpose of this query? More generally what is the context
for it?
o How often does a query of this type run?
o How common are updates? Partiularly, how common are updates to the
network table?
o CREATE VIEW for all views involved and CREATE TABLE and CREATE INDEX
for the tables involved, including those referred to by views and
function.
o Rowcounts for all involved tables.
o The query-plan for the query.
o Sample data to test solutions for correctness. (To test for performance
we would need more data that is practical to include a news post.)

Yes, it would take you some effort to compile this information, but
you are asking us to make a community to help you. If you are not
prepared to make that effort, should you really expect us to make any
effort?
--
Erland Sommarskog, SQL Server MVP, es****@sommarsk og.se

Books Online for SQL Server 2005 at
http://www.microsoft.com/technet/pro...ads/books.mspx
Books Online for SQL Server 2000 at
http://www.microsoft.com/sql/prodinf...ons/books.mspx
Mar 8 '08 #6
http://www.cogitoinc.com/

Use the right tool for the job.

Mar 9 '08 #7

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

226
12663
by: Stephen C. Waterbury | last post by:
This seems like it ought to work, according to the description of reduce(), but it doesn't. Is this a bug, or am I missing something? Python 2.3.2 (#1, Oct 20 2003, 01:04:35) on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> d1 = {'a':1} >>> d2 = {'b':2} >>> d3 = {'c':3}
181
8890
by: Tom Anderson | last post by:
Comrades, During our current discussion of the fate of functional constructs in python, someone brought up Guido's bull on the matter: http://www.artima.com/weblogs/viewpost.jsp?thread=98196 He says he's going to dispose of map, filter, reduce and lambda. He's going to give us product, any and all, though, which is nice of him.
0
1786
by: SimonC | last post by:
I'm looking to do something similar to a feature found on Ticketmaster.com, where you select your seats at a venue, and then you have two minutes in which to take or leave them. QUESTION 1a. Inside (or just after) the same query that searches for available seats, I need to SIMULTANEOUSLY mark those seats as "on hold". I've only read about, but not yet used MySQL transactions, and wonder if this simultaneous "search-and-hold"...
2
1091
by: theintrepidfox | last post by:
Dear Group Can anyone provide a sample query for the following scenario? Let's assume I want to search for an order someone placed which might be an individual or company. An individuals first name is stored in column FirstName And the individuls last name in column LastName of the contact table and the company name is stored in column CompanyName of the company table.
13
2200
by: MLH | last post by:
I have a RDBMS app consisting of 3 primary mdb's... 1) a front-end with a few STATIC tables and the other menagerie of objects 2) a back-end with most of my DYNAMIC tables. I'll call it my main backend. 3) another back-end = zip.mdb with about 43000 zips/cities/states The app has been operating stably (is that a word?) for some years. No probs. The main backend is 63.3 megs now and contains tens of thousands of letters - legal...
7
4241
by: Jerome | last post by:
Hallo, I know a lot has already been told about date/time fields in a database but still confuses me, specif when dealing with SQLserver(Express). It seems that sqlserver only accepts the date in a "yyyyMMdd" format? (difference between Express and MSDE2000A ?) What is the one and only true way to deal with this problem in VB2005: Local settings are Dutch (Belgium) ; thus date is in "dd/MM/yy" (or perhaps dd/MM/yyyy) and time in...
4
2881
by: subash | last post by:
Please check this query $query = "SELECT a.intNEHID AS id,a.vchNEHTitle AS title" . "\n FROM tbNews AS a, tbNews AS b,tbMinistry AS c" . "\n WHERE (a.vchNEHDesc LIKE '%".$searchWords."%' OR a.vchNEHTitle LIKE " . "\n'%".$searchWords."%' OR a.vchNEHAbstract LIKE '%".$searchWords."%')" . "\n AND a.vchNEHType IN 'News' " . "\n AND a.chrNEHStatus != 'Delete' " . "\n AND a.intNEHID=b.intNEHID...
6
2842
by: Simon | last post by:
Dear reader, I have an Access application which works as back-end and front-end. In case it's running on a local PC it works perfect. If I install it on a server the response time is increasing drastically.
3
4614
by: sbettadpur | last post by:
hello friends, I need some solution for reducing the query execution time. Let me explain briefly: I have 5000 records in mysql database. actually i am using browse buttons to see the records. i,e forward, backward and fastforward etc.. if i press forward button it will show one record in their respective text field. so my problem is if i submit forward button it's taking more time to load values in the text box, and one more i...
0
10002
jinu1996
by: jinu1996 | last post by:
In today's digital age, having a compelling online presence is paramount for businesses aiming to thrive in a competitive landscape. At the heart of this digital strategy lies an intricately woven tapestry of website design and digital marketing. It's not merely about having a website; it's about crafting an immersive digital experience that captivates audiences and drives business growth. The Art of Business Website Design Your website is...
1
9938
by: Hystou | last post by:
Overview: Windows 11 and 10 have less user interface control over operating system update behaviour than previous versions of Windows. In Windows 11 and 10, there is no way to turn off the Windows Update option using the Control Panel or Settings app; it automatically checks for updates and installs any it finds, whether you like it or not. For most users, this new feature is actually very convenient. If you want to control the update process,...
0
9823
tracyyun
by: tracyyun | last post by:
Dear forum friends, With the development of smart home technology, a variety of wireless communication protocols have appeared on the market, such as Zigbee, Z-Wave, Wi-Fi, Bluetooth, etc. Each protocol has its own unique characteristics and advantages, but as a user who is planning to build a smart home system, I am a bit confused by the choice of these technologies. I'm particularly interested in Zigbee because I've heard it does some...
0
8822
agi2029
by: agi2029 | last post by:
Let's talk about the concept of autonomous AI software engineers and no-code agents. These AIs are designed to manage the entire lifecycle of a software development project—planning, coding, testing, and deployment—without human intervention. Imagine an AI that can take a project description, break it down, write the code, debug it, and then launch it, all on its own.... Now, this would greatly impact the work of software developers. The idea...
1
7368
isladogs
by: isladogs | last post by:
The next Access Europe User Group meeting will be on Wednesday 1 May 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM). In this session, we are pleased to welcome a new presenter, Adolph Dupré who will be discussing some powerful techniques for using class modules. He will explain when you may want to use classes instead of User Defined Types (UDT). For example, to manage the data in unbound forms. Adolph will...
0
6643
by: conductexam | last post by:
I have .net C# application in which I am extracting data from word file and save it in database particularly. To store word all data as it is I am converting the whole word file firstly in HTML and then checking html paragraph one by one. At the time of converting from word file to html my equations which are in the word document file was convert into image. Globals.ThisAddIn.Application.ActiveDocument.Select();...
0
5406
by: adsilva | last post by:
A Windows Forms form does not have the event Unload, like VB6. What one acts like?
1
3917
by: 6302768590 | last post by:
Hai team i want code for transfer the data from one system to another through IP address by using C# our system has to for every 5mins then we have to update the data what the data is updated we have to send another system
3
2794
bsmnconsultancy
by: bsmnconsultancy | last post by:
In today's digital era, a well-designed website is crucial for businesses looking to succeed. Whether you're a small business owner or a large corporation in Toronto, having a strong online presence can significantly impact your brand's success. BSMN Consultancy, a leader in Website Development in Toronto offers valuable insights into creating effective websites that not only look great but also perform exceptionally well. In this comprehensive...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.