By using this site, you agree to our updated Privacy Policy and our Terms of Use. Manage your Cookies Settings.
431,663 Members | 762 Online
Bytes IT Community
+ Ask a Question
Need help? Post your question and get tips & solutions from a community of 431,663 IT Pros & Developers. It's quick & easy.

script execution time too long

P: n/a


I have a script that does a huge database update, which takes a long
time. However, the host I am running on (and I suspect most hosts) are
running in "safe mode." This means I cannot change the maximum execution
time of scripts. So, the script stops right in the middle of doing the
update.

On top of that, I am trying to make code that I can distribute to
others. As of now, the only way that this script can be run is with sys
admin intervention. I would like to make it run without any help from
the system administrator.

Does any one have a good solution for this problem?

-d
Jul 17 '05 #1
Share this Question
Share on Google+
10 Replies


P: n/a
i would redesign the database to make it faster and make it so that no admin
intervention would be needed.

"Doug" <do*****@XXXXremovetheXearthlink.net> wrote in message
news:Aw*****************@newsread3.news.atl.earthl ink.net...


I have a script that does a huge database update, which takes a long time.
However, the host I am running on (and I suspect most hosts) are running
in "safe mode." This means I cannot change the maximum execution time of
scripts. So, the script stops right in the middle of doing the update.

On top of that, I am trying to make code that I can distribute to others.
As of now, the only way that this script can be run is with sys admin
intervention. I would like to make it run without any help from the
system administrator.

Does any one have a good solution for this problem?

-d

Jul 17 '05 #2

P: n/a

Thanks.... That is an option, but, there is a very large amount of data
involved. Plus the amount of indexes on the data slow down the entry
into the DB.

-d

konsu wrote:
i would redesign the database to make it faster and make it so that no admin
intervention would be needed.

"Doug" <do*****@XXXXremovetheXearthlink.net> wrote in message
news:Aw*****************@newsread3.news.atl.earthl ink.net...

I have a script that does a huge database update, which takes a long time.
However, the host I am running on (and I suspect most hosts) are running
in "safe mode." This means I cannot change the maximum execution time of
scripts. So, the script stops right in the middle of doing the update.

On top of that, I am trying to make code that I can distribute to others.
As of now, the only way that this script can be run is with sys admin
intervention. I would like to make it run without any help from the
system administrator.

Does any one have a good solution for this problem?

-d


Jul 17 '05 #3

P: n/a
well, you are talking about a web server processing a request with some html
form data and not being able to finish the prcessing because the database
operation takes too long. the reason why it takes too long is not because
there is a lot of data in the form, otherwise it would choke with all that
data much earlier, maybe even on the client side, so it is not the form, it
is the design of the database. did you try to test it on an empty database?
does it work? if it does then one thing to look at is to segment the
database into pieces. you have to do something with the database because
from what you are saying it looks like it is unmanageable and hence is not
practical to use anymore. it is useless in other words. ;-)

konstantin
"Doug" <do*****@XXXXremovetheXearthlink.net> wrote in message
news:EE*****************@newsread3.news.atl.earthl ink.net...

Thanks.... That is an option, but, there is a very large amount of data
involved. Plus the amount of indexes on the data slow down the entry into
the DB.

-d

konsu wrote:
i would redesign the database to make it faster and make it so that no
admin intervention would be needed.

Jul 17 '05 #4

P: n/a
"Doug48" wrote:
I have a script that does a huge database update, which takes
a long
time. However, the host I am running on (and I suspect most
hosts) are
running in "safe mode." This means I cannot change the maximum
execution
time of scripts. So, the script stops right in the middle of
doing the
update.

On top of that, I am trying to make code that I can distribute
to
others. As of now, the only way that this script can be run
is with sys
admin intervention. I would like to make it run without any
help from
the system administrator.

Does any one have a good solution for this problem?

-d


Hi Doug,

More info please. Are you running from CLI, or via browser. If
running from browser, then you get around the timeout problem by
having one browser window call another after say, 20 secons, and keep
going until the process is done.

If running from CLI, then I may suggest going with a VPS account which
will get you the flexibility to run the bath job uninterrupted. VPS
can be had these days for $30/month or even less.

You mentioned distributing the script to others? Can you clarify.
Obviously, most users would not be able to run CLI. Then they would
run this process via browser, and the multiple window solution would
do the job (although not reliably, as browser is NOT a good way of
running a long batch job, since it can break easily).

--
http://www.dbForumz.com/ This article was posted by author's request
Articles individually checked for conformance to usenet standards
Topic URL: http://www.dbForumz.com/PHP-script-e...ict178903.html
Visit Topic URL to contact author (reg. req'd). Report abuse: http://www.dbForumz.com/eform.php?p=602510
Jul 17 '05 #5

P: n/a

Let me explain, there is a zipped text file of about 72000 lines (with
zip code data), that must be added to the DB. Once it gets to the
server, it is unzipped, and the zip codes are updated and/or added.

-d

konsu wrote:
well, you are talking about a web server processing a request with some html
form data and not being able to finish the prcessing because the database
operation takes too long. the reason why it takes too long is not because
there is a lot of data in the form, otherwise it would choke with all that
data much earlier, maybe even on the client side, so it is not the form, it
is the design of the database. did you try to test it on an empty database?
does it work? if it does then one thing to look at is to segment the
database into pieces. you have to do something with the database because
from what you are saying it looks like it is unmanageable and hence is not
practical to use anymore. it is useless in other words. ;-)

konstantin
"Doug" <do*****@XXXXremovetheXearthlink.net> wrote in message
news:EE*****************@newsread3.news.atl.earthl ink.net...
Thanks.... That is an option, but, there is a very large amount of data
involved. Plus the amount of indexes on the data slow down the entry into
the DB.

-d

konsu wrote:
i would redesign the database to make it faster and make it so that no
admin intervention would be needed.


Jul 17 '05 #6

P: n/a

What's VPS and multiple window solution?

there is a zipped text file of about 72000 lines (with US, us
territories zip code data), that must be added to the DB. Once it gets
to the server, it is unzipped, and the zip codes are updated and/or added.

steve wrote:
"Doug48" wrote:
> I have a script that does a huge database update, which takes
> a long
> time. However, the host I am running on (and I suspect most
> hosts) are
> running in "safe mode." This means I cannot change the maximum
> execution
> time of scripts. So, the script stops right in the middle of
> doing the
> update.
>
> On top of that, I am trying to make code that I can distribute
> to
> others. As of now, the only way that this script can be run
> is with sys
> admin intervention. I would like to make it run without any
> help from
> the system administrator.
>
> Does any one have a good solution for this problem?
>
> -d


Hi Doug,

More info please. Are you running from CLI, or via browser. If
running from browser, then you get around the timeout problem by
having one browser window call another after say, 20 secons, and keep
going until the process is done.

If running from CLI, then I may suggest going with a VPS account which
will get you the flexibility to run the bath job uninterrupted. VPS
can be had these days for $30/month or even less.

You mentioned distributing the script to others? Can you clarify.
Obviously, most users would not be able to run CLI. Then they would
run this process via browser, and the multiple window solution would
do the job (although not reliably, as browser is NOT a good way of
running a long batch job, since it can break easily).

Jul 17 '05 #7

P: n/a
i see ;-)

would you explain why you are processing the file as part of a web request?
(i am still assuming you have a web request ;-)) is there a user who might
want to submit such a file from a web page?

can you create a cron job (assuming you are running unix) or something
similar that would use awk or what else to read the file and update the
database during some night time hours when everyone is sleeping and not
watching? ;-)

konstantin

"Doug" <do*****@XXXXremovetheXearthlink.net> wrote in message
news:78****************@newsread2.news.atl.earthli nk.net...

Let me explain, there is a zipped text file of about 72000 lines (with zip
code data), that must be added to the DB. Once it gets to the server, it
is unzipped, and the zip codes are updated and/or added.

-d

Jul 17 '05 #8

P: n/a

Zip codes are updated and changed all the time.

So, PHP scripts run by cron have no time limit?

-d

konsu wrote:
i see ;-)

would you explain why you are processing the file as part of a web request?
(i am still assuming you have a web request ;-)) is there a user who might
want to submit such a file from a web page?

can you create a cron job (assuming you are running unix) or something
similar that would use awk or what else to read the file and update the
database during some night time hours when everyone is sleeping and not
watching? ;-)

konstantin

"Doug" <do*****@XXXXremovetheXearthlink.net> wrote in message
news:78****************@newsread2.news.atl.earthli nk.net...
Let me explain, there is a zipped text file of about 72000 lines (with zip
code data), that must be added to the DB. Once it gets to the server, it
is unzipped, and the zip codes are updated and/or added.

-d


Jul 17 '05 #9

P: n/a
> So, PHP scripts run by cron have no time limit?

i do not know about that but i think you have a better control over it. and
if you do not you can always use a non-PHP solution.
Jul 17 '05 #10

P: n/a
"Doug" <do*****@XXXXremovetheXearthlink.net> wrote in message
news:Aw*****************@newsread3.news.atl.earthl ink.net...


I have a script that does a huge database update, which takes a long
time. However, the host I am running on (and I suspect most hosts) are
running in "safe mode." This means I cannot change the maximum execution
time of scripts. So, the script stops right in the middle of doing the
update.

On top of that, I am trying to make code that I can distribute to
others. As of now, the only way that this script can be run is with sys
admin intervention. I would like to make it run without any help from
the system administrator.

Does any one have a good solution for this problem?

-d


Well, break up the process into multiple pieces. The initial request would
process the first X number of lines, then tell the client to refresh
(Javascript). The second request then process the next X number of lines and
so on.

It's a good way to indicate to the user that something is happening too.
Jul 17 '05 #11

This discussion thread is closed

Replies have been disabled for this discussion.