473,554 Members | 2,950 Online
Bytes | Software Development & Data Engineering Community
+ Post

Home Posts Topics Members FAQ

script execution time too long



I have a script that does a huge database update, which takes a long
time. However, the host I am running on (and I suspect most hosts) are
running in "safe mode." This means I cannot change the maximum execution
time of scripts. So, the script stops right in the middle of doing the
update.

On top of that, I am trying to make code that I can distribute to
others. As of now, the only way that this script can be run is with sys
admin intervention. I would like to make it run without any help from
the system administrator.

Does any one have a good solution for this problem?

-d
Jul 17 '05 #1
10 5384
i would redesign the database to make it faster and make it so that no admin
intervention would be needed.

"Doug" <do*****@XXXXre movetheXearthli nk.net> wrote in message
news:Aw******** *********@newsr ead3.news.atl.e arthlink.net...


I have a script that does a huge database update, which takes a long time.
However, the host I am running on (and I suspect most hosts) are running
in "safe mode." This means I cannot change the maximum execution time of
scripts. So, the script stops right in the middle of doing the update.

On top of that, I am trying to make code that I can distribute to others.
As of now, the only way that this script can be run is with sys admin
intervention. I would like to make it run without any help from the
system administrator.

Does any one have a good solution for this problem?

-d

Jul 17 '05 #2

Thanks.... That is an option, but, there is a very large amount of data
involved. Plus the amount of indexes on the data slow down the entry
into the DB.

-d

konsu wrote:
i would redesign the database to make it faster and make it so that no admin
intervention would be needed.

"Doug" <do*****@XXXXre movetheXearthli nk.net> wrote in message
news:Aw******** *********@newsr ead3.news.atl.e arthlink.net...

I have a script that does a huge database update, which takes a long time.
However, the host I am running on (and I suspect most hosts) are running
in "safe mode." This means I cannot change the maximum execution time of
scripts. So, the script stops right in the middle of doing the update.

On top of that, I am trying to make code that I can distribute to others.
As of now, the only way that this script can be run is with sys admin
interventio n. I would like to make it run without any help from the
system administrator.

Does any one have a good solution for this problem?

-d


Jul 17 '05 #3
well, you are talking about a web server processing a request with some html
form data and not being able to finish the prcessing because the database
operation takes too long. the reason why it takes too long is not because
there is a lot of data in the form, otherwise it would choke with all that
data much earlier, maybe even on the client side, so it is not the form, it
is the design of the database. did you try to test it on an empty database?
does it work? if it does then one thing to look at is to segment the
database into pieces. you have to do something with the database because
from what you are saying it looks like it is unmanageable and hence is not
practical to use anymore. it is useless in other words. ;-)

konstantin
"Doug" <do*****@XXXXre movetheXearthli nk.net> wrote in message
news:EE******** *********@newsr ead3.news.atl.e arthlink.net...

Thanks.... That is an option, but, there is a very large amount of data
involved. Plus the amount of indexes on the data slow down the entry into
the DB.

-d

konsu wrote:
i would redesign the database to make it faster and make it so that no
admin intervention would be needed.

Jul 17 '05 #4
"Doug48" wrote:
I have a script that does a huge database update, which takes
a long
time. However, the host I am running on (and I suspect most
hosts) are
running in "safe mode." This means I cannot change the maximum
execution
time of scripts. So, the script stops right in the middle of
doing the
update.

On top of that, I am trying to make code that I can distribute
to
others. As of now, the only way that this script can be run
is with sys
admin intervention. I would like to make it run without any
help from
the system administrator.

Does any one have a good solution for this problem?

-d


Hi Doug,

More info please. Are you running from CLI, or via browser. If
running from browser, then you get around the timeout problem by
having one browser window call another after say, 20 secons, and keep
going until the process is done.

If running from CLI, then I may suggest going with a VPS account which
will get you the flexibility to run the bath job uninterrupted. VPS
can be had these days for $30/month or even less.

You mentioned distributing the script to others? Can you clarify.
Obviously, most users would not be able to run CLI. Then they would
run this process via browser, and the multiple window solution would
do the job (although not reliably, as browser is NOT a good way of
running a long batch job, since it can break easily).

--
http://www.dbForumz.com/ This article was posted by author's request
Articles individually checked for conformance to usenet standards
Topic URL: http://www.dbForumz.com/PHP-script-e...ict178903.html
Visit Topic URL to contact author (reg. req'd). Report abuse: http://www.dbForumz.com/eform.php?p=602510
Jul 17 '05 #5

Let me explain, there is a zipped text file of about 72000 lines (with
zip code data), that must be added to the DB. Once it gets to the
server, it is unzipped, and the zip codes are updated and/or added.

-d

konsu wrote:
well, you are talking about a web server processing a request with some html
form data and not being able to finish the prcessing because the database
operation takes too long. the reason why it takes too long is not because
there is a lot of data in the form, otherwise it would choke with all that
data much earlier, maybe even on the client side, so it is not the form, it
is the design of the database. did you try to test it on an empty database?
does it work? if it does then one thing to look at is to segment the
database into pieces. you have to do something with the database because
from what you are saying it looks like it is unmanageable and hence is not
practical to use anymore. it is useless in other words. ;-)

konstantin
"Doug" <do*****@XXXXre movetheXearthli nk.net> wrote in message
news:EE******** *********@newsr ead3.news.atl.e arthlink.net...
Thanks.... That is an option, but, there is a very large amount of data
involved. Plus the amount of indexes on the data slow down the entry into
the DB.

-d

konsu wrote:
i would redesign the database to make it faster and make it so that no
admin intervention would be needed.


Jul 17 '05 #6

What's VPS and multiple window solution?

there is a zipped text file of about 72000 lines (with US, us
territories zip code data), that must be added to the DB. Once it gets
to the server, it is unzipped, and the zip codes are updated and/or added.

steve wrote:
"Doug48" wrote:
> I have a script that does a huge database update, which takes
> a long
> time. However, the host I am running on (and I suspect most
> hosts) are
> running in "safe mode." This means I cannot change the maximum
> execution
> time of scripts. So, the script stops right in the middle of
> doing the
> update.
>
> On top of that, I am trying to make code that I can distribute
> to
> others. As of now, the only way that this script can be run
> is with sys
> admin intervention. I would like to make it run without any
> help from
> the system administrator.
>
> Does any one have a good solution for this problem?
>
> -d


Hi Doug,

More info please. Are you running from CLI, or via browser. If
running from browser, then you get around the timeout problem by
having one browser window call another after say, 20 secons, and keep
going until the process is done.

If running from CLI, then I may suggest going with a VPS account which
will get you the flexibility to run the bath job uninterrupted. VPS
can be had these days for $30/month or even less.

You mentioned distributing the script to others? Can you clarify.
Obviously, most users would not be able to run CLI. Then they would
run this process via browser, and the multiple window solution would
do the job (although not reliably, as browser is NOT a good way of
running a long batch job, since it can break easily).

Jul 17 '05 #7
i see ;-)

would you explain why you are processing the file as part of a web request?
(i am still assuming you have a web request ;-)) is there a user who might
want to submit such a file from a web page?

can you create a cron job (assuming you are running unix) or something
similar that would use awk or what else to read the file and update the
database during some night time hours when everyone is sleeping and not
watching? ;-)

konstantin

"Doug" <do*****@XXXXre movetheXearthli nk.net> wrote in message
news:78******** ********@newsre ad2.news.atl.ea rthlink.net...

Let me explain, there is a zipped text file of about 72000 lines (with zip
code data), that must be added to the DB. Once it gets to the server, it
is unzipped, and the zip codes are updated and/or added.

-d

Jul 17 '05 #8

Zip codes are updated and changed all the time.

So, PHP scripts run by cron have no time limit?

-d

konsu wrote:
i see ;-)

would you explain why you are processing the file as part of a web request?
(i am still assuming you have a web request ;-)) is there a user who might
want to submit such a file from a web page?

can you create a cron job (assuming you are running unix) or something
similar that would use awk or what else to read the file and update the
database during some night time hours when everyone is sleeping and not
watching? ;-)

konstantin

"Doug" <do*****@XXXXre movetheXearthli nk.net> wrote in message
news:78******** ********@newsre ad2.news.atl.ea rthlink.net...
Let me explain, there is a zipped text file of about 72000 lines (with zip
code data), that must be added to the DB. Once it gets to the server, it
is unzipped, and the zip codes are updated and/or added.

-d


Jul 17 '05 #9
> So, PHP scripts run by cron have no time limit?

i do not know about that but i think you have a better control over it. and
if you do not you can always use a non-PHP solution.
Jul 17 '05 #10

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

9
2330
by: Sims | last post by:
Hi, I recall a function that would tell me how long it took to generate an output. But for the life of me I cannot remember it. Dow anybody know it? Or do I have to write something myself, (using time?). Sims
7
3646
by: Damien | last post by:
Hello to all, I've written a script that take quite a long time to execute (email sending). Last time I ran it, PHP gave me a "execution time limit exceeded" (or something like that), so I changed the value in php.ini. Now, the questions are : - How can I make sure I'm not going to have problems when execution time lengthens still ? I'm...
5
6290
by: Boris Nikolaevich | last post by:
This is backwards of what I usually want--normally if you have a long-running ASP script, it's a good idea to check to see whether the client is still connected so you can cancel execution. However, I have a script that absolutely MUST finish one it's been started--is there a way to cause the entire script to execute, even if the client...
3
47529
by: iam980 | last post by:
Hello All. We have tested following SQL script from query analyzer: -- Script begin DECLARE @I int; SET @I = 1; WHILE @I < 10000000 BEGIN SET @I = @I + 1; END -- Script end
2
2069
by: Jonah Bishop | last post by:
What is the appropriate way to start a PHP script in the background? I have a script that manipulates image data, and it takes a long time to run. I'd like to send this process to the background at the appropriate time, so that it can go do its job without holding up the user. I've seen something like this used: system("php myscript.php...
13
5989
by: wattersmt | last post by:
Hello, I am trying to write a python cgi that calls a script over ssh, the problem is the script takes a very long time to execute so Apache makes the CGI time out and I never see any output. The script is set to print a progress report to stdout every 3 seconds but I never see any output until the child process is killed. Here's what I...
13
3971
by: Marvin Zhang | last post by:
Hi, I'm not familiar with web programming, but I have a problem here. I have a page. When a user click one button on it, I will use AJAX to request a PHP script which will do a bunch of tasks, asynchronously. These tasks might take long time so I want to keep the user informed of the progress. The problem is that only the PHP script knows...
4
4405
by: J. Frank Parnell | last post by:
Hi there, I have a list of links which point to e.g. thescript.php?album=somePictures1 thescript.php?album=somePictures2 This list is about 3000 links. Each album may have 500 or more pictures in it. the script looks in the specified dir, and creates thumbnails if they are not present. So, displaying a particular album often takes...
15
3230
by: Lawrence Krubner | last post by:
Does anything about this script look expensive, in terms of resources or execution time? This script dies after processing about 20 or 25 numbers, yet it leaves no errors in the error logs. This is on a server that handles a fairly demanding site. The defaults, in php.ini, have all been cranked fairly high: scripts get 180 seconds to run, and...
0
7600
marktang
by: marktang | last post by:
ONU (Optical Network Unit) is one of the key components for providing high-speed Internet services. Its primary function is to act as an endpoint device located at the user's premises. However, people are often confused as to whether an ONU can Work As a Router. In this blog post, we’ll explore What is ONU, What Is Router, ONU & Router’s main...
0
7521
by: Hystou | last post by:
Most computers default to English, but sometimes we require a different language, especially when relocating. Forgot to request a specific language before your computer shipped? No problem! You can effortlessly switch the default language on Windows 10 without reinstalling. I'll walk you through it. First, let's disable language...
1
7563
by: Hystou | last post by:
Overview: Windows 11 and 10 have less user interface control over operating system update behaviour than previous versions of Windows. In Windows 11 and 10, there is no way to turn off the Windows Update option using the Control Panel or Settings app; it automatically checks for updates and installs any it finds, whether you like it or not. For...
0
7889
tracyyun
by: tracyyun | last post by:
Dear forum friends, With the development of smart home technology, a variety of wireless communication protocols have appeared on the market, such as Zigbee, Z-Wave, Wi-Fi, Bluetooth, etc. Each protocol has its own unique characteristics and advantages, but as a user who is planning to build a smart home system, I am a bit confused by the...
0
6145
agi2029
by: agi2029 | last post by:
Let's talk about the concept of autonomous AI software engineers and no-code agents. These AIs are designed to manage the entire lifecycle of a software development project—planning, coding, testing, and deployment—without human intervention. Imagine an AI that can take a project description, break it down, write the code, debug it, and then...
0
5155
by: conductexam | last post by:
I have .net C# application in which I am extracting data from word file and save it in database particularly. To store word all data as it is I am converting the whole word file firstly in HTML and then checking html paragraph one by one. At the time of converting from word file to html my equations which are in the word document file was convert...
0
3560
by: TSSRALBI | last post by:
Hello I'm a network technician in training and I need your help. I am currently learning how to create and manage the different types of VPNs and I have a question about LAN-to-LAN VPNs. The last exercise I practiced was to create a LAN-to-LAN VPN between two Pfsense firewalls, by using IPSEC protocols. I succeeded, with both firewalls in...
1
2020
by: 6302768590 | last post by:
Hai team i want code for transfer the data from one system to another through IP address by using C# our system has to for every 5mins then we have to update the data what the data is updated we have to send another system
0
841
bsmnconsultancy
by: bsmnconsultancy | last post by:
In today's digital era, a well-designed website is crucial for businesses looking to succeed. Whether you're a small business owner or a large corporation in Toronto, having a strong online presence can significantly impact your brand's success. BSMN Consultancy, a leader in Website Development in Toronto offers valuable insights into creating...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.