What are good ways for keeping your script faster then 47.01 seconds?

[Ajm113]

Ok, as some of you may know I am making a search engine and right now it takes atleast 47 or 40 seconds to load up 5 or more results here is what it is doing;

1.Load logo, textbox, what the user is searching for.

2.Get user info and connect to mysql

3.Select and search the mysql tables.

4.Do a while statment and pull up the websites and look at the description meta tag and pull it up to display.

5.If they don't just use the description in the database.

6.Load the image and text on each result.

7.Echo the number of results and "top" button when done.

What are some tips and tricks on keeping load time faster? My results page praticly has no special effects or very advanced CSS Designs in it.

[Purple]

Hi Ajm113,

The first thing you need to do is work out where the bulk of the 47 second is being taken. I suggest you write timed log event through your code and apply pareto's law

Purple

[Ajm113]

I think its the meta checking area where it checks if the site has a description meta tag that is making it slower. Isn't there shortcuts to this?

[PHP]while($row=mysql_fetch_array($query)) { // Starts spitting out the data

$url=$row["url"];
$title=$row["title"];

$fp = fopen( $url, 'r' );

$content = "";


while( !feof( $fp ) ) {

$buffer = trim( fgets( $fp, 4096 ) );
$content .= $buffer;

}

$metatagarray = get_meta_tags( $url );
$description = $metatagarray[ "description" ];
$description = substr($description,0,240);

if($description == '')
{
$description=$row["description"];
}
//echo out results
}
[/PHP]

[Purple]

Hi Ajm113,

I thnik I agree with you - might be worth writing a logging function to confirm your suspicions....

But back to the question - cant you write a process which caches the meta data and saves it in your database so you dont need to pull it back for every query ?

My thoughts - everyone else welcome to join in :)

Purple

[Ajm113]

I thought about that, but then if someone was supposed to change the look of their site or had a metatag that updated everyday then thats when I made that. So if it finds the description tag it would desplay it and if it didn't. To save some time for the user. The script would pull the description from the database.

[Purple]

Hi,

That is what I expect would be most efficient, just have you background process running on an appropriate interval updating the tags etc on the database - obviously as your database grows it will take longer to update the tag detail..

Let us know how you go on.

Good luck and regards

Purple

[pbmods]

Is your MySQL database indexed properly? Check your slow_query_log.

Try EXPLAINing your queries and make sure you're not doing full scans where you don't have to.

[Motoma]

Hi Ajm113,

I thnik I agree with you - might be worth writing a logging function to confirm your suspicions....

But back to the question - cant you write a process which caches the meta data and saves it in your database so you dont need to pull it back for every query ?

My thoughts - everyone else welcome to join in :)

Purple

Yes, the process of collecting meta tags is likely your issue, as you have to synchronously perform a read of other URLs.
Your best bet would be to cache the meta tags, and update them on a regular basis.