can php program clog the rental web server?

[mich dobelman]

I am developing a web crawler which collects info from more than 50,0000
pages.
but i am afraid this is going to clog the rental web server if I run it
every single day.

Is it possible to stop the web hosting server because of my program?

MD

[Jerry Stuckle]

mich dobelman wrote:

I am developing a web crawler which collects info from more than 50,0000
pages.
but i am afraid this is going to clog the rental web server if I run it
every single day.

Is it possible to stop the web hosting server because of my program?

MD

Of course you can. Anyone on the server can, with a poorly written
script. For instance,

<?php
for ($i = 0; $i < 10; $i--) {
// Do something here not requiring I/O
}

Will tie up the server for a while (I know from experience :-) ).

Because of this, most shared hosts keep the max execution time of PHP
scripts rather short, like 30 seconds. That way a single script can't
screw things up for too long.
--
=============== ===
Remove the "x" from my email address
Jerry Stuckle
JDS Computer Training Corp.
js*******@attgl obal.net
=============== ===

[MD]

How about putting sleep() when visiting each page ?

Is this going to reduce the risk of stopping the server?

MD

"Jerry Stuckle" <js*******@attg lobal.netwrote in message
news:oO******** *************** *******@comcast .com...

mich dobelman wrote:

I am developing a web crawler which collects info from more than 50,0000
pages.
but i am afraid this is going to clog the rental web server if I run it
every single day.

Is it possible to stop the web hosting server because of my program?

MD

Of course you can. Anyone on the server can, with a poorly written
script. For instance,

<?php
for ($i = 0; $i < 10; $i--) {
// Do something here not requiring I/O
}

Will tie up the server for a while (I know from experience :-) ).

Because of this, most shared hosts keep the max execution time of PHP
scripts rather short, like 30 seconds. That way a single script can't
screw things up for too long.
--
=============== ===
Remove the "x" from my email address
Jerry Stuckle
JDS Computer Training Corp.
js*******@attgl obal.net
=============== ===

[Jerry Stuckle]

MD wrote:

"Jerry Stuckle" <js*******@attg lobal.netwrote in message
news:oO******** *************** *******@comcast .com...

>>mich dobelman wrote:

>>>I am developing a web crawler which collects info from more than 50,0000
pages.
but i am afraid this is going to clog the rental web server if I run it
every single day.

Is it possible to stop the web hosting server because of my program?

MD

Of course you can. Anyone on the server can, with a poorly written
script. For instance,

<?php
for ($i = 0; $i < 10; $i--) {
// Do something here not requiring I/O
}

Will tie up the server for a while (I know from experience :-) ).

Because of this, most shared hosts keep the max execution time of PHP
scripts rather short, like 30 seconds. That way a single script can't
screw things up for too long.

How about putting sleep() when visiting each page ?

Is this going to reduce the risk of stopping the server?

MD

(Top posting fixed)

Yes, but why would you want to? You'll run up against the timeout value
your host sets for PHP.

What are you doing that will hog all of those resources?
--
=============== ===
Remove the "x" from my email address
Jerry Stuckle
JDS Computer Training Corp.
js*******@attgl obal.net
=============== ===

[MD]

(Top posting fixed)

>
Yes, but why would you want to? You'll run up against the timeout value
your host sets for PHP.

What are you doing that will hog all of those resources?
--
=============== ===
Remove the "x" from my email address
Jerry Stuckle
JDS Computer Training Corp.
js*******@attgl obal.net
=============== ===

>You'll run up against the timeout value your host sets for PHP.

I can always change the value from php.

[Jerry Stuckle]

MD wrote:

>>(Top posting fixed)

Yes, but why would you want to? You'll run up against the timeout value
your host sets for PHP.

What are you doing that will hog all of those resources?
--
============= =====
Remove the "x" from my email address
Jerry Stuckle
JDS Computer Training Corp.
js*******@att global.net
============= =====

>>You'll run up against the timeout value your host sets for PHP.

I can always change the value from php.

Not necessarily. A shared host often restricts changing the timeout and
certain other values.
--
=============== ===
Remove the "x" from my email address
Jerry Stuckle
JDS Computer Training Corp.
js*******@attgl obal.net
=============== ===