HTTP in the background in IE

[Steve Kirkendall]

I need a work-around for an IE limitation, regarding fetching HTTP
documents in the background. It takes a bit of explaining; please
be patient...

I'm working on an project that will run on an intranet. I need to
have pages be reloaded automatically when the data they reflect is
changed at the server. For Netscape/Mozilla/Firefox, I do this by
having the page use JavaScript to read a special "/update.html" URL
in the background. For this URL, the server doesn't return anything
until it detects a change in the data, at which point it sends a
meaningless response and closes the connection. The browser's
JavaScript code then reloads the main page, showing the new data.
It works great!

But there's this other browser, "Internet Explorer", which I must
also support. The "slow /update.html" strategy *seems* to work until
I exit & reload the main page multiple times, without any changes
being detected in the data. If I do that, then the browser gets stuck.

Apparently IE doesn't terminate backgrounded JavaScript when you exit
the page that started it. For my applicatation, this means IE tries
to accumulate extra backgrounded "/update.html" requests each time
you exit & reload the main page. Also, it seems that IE only allows
about 6 TCP connection in total, so once it's accumulated 6
"/update.html" requests, it can't read anything else.

I got the background "/update.html" strategy to work in IE, more or
less, by making it remember which thread is being used to serve the
slow "/update.html" request for each client, and then when a new thread
wants to do serve a new "/update.html" for the same client, it first
kills the old thread and closes the old connection. I consider this
a nasty hack, but it works.

Except that now I want to allow each client to perform two "/update.html"
requests in parallel -- one for the main window, and one for a pop-up
window. (The /update.html request accepts parameters describing what,
exactly, the client wants to wait for. The main window and pop-up window
will wait for different things.) I tried extending the same "kill the
old thread" hack to handle two connections, but it doesn't seem to be
working. I may yet get it to work, but before I waste a lot of time
on it, I thought I'd ask whether anybody else can suggest something
better.

To reitterate: I want to allow each client to have two windows that
immediately reload their pages when the server's data changes.

Is there some option that IE's JavaScript environment uses to indicate
whether backgrounded tasks should die when the page is unloaded?

Is there a way to kill them on a document unload event, or some
similar event?

Thanks in advance for any advice you can give.

[Thomas 'PointedEars' Lahn]

Steve Kirkendall wrote:

I'm working on an project that will run on an intranet. I need to
have pages be reloaded automatically when the data they reflect is
changed at the server. For Netscape/Mozilla/Firefox, I do this by
having the page use JavaScript to read a special "/update.html" URL
in the background. For this URL, the server doesn't return anything
until it detects a change in the data, at which point it sends a
meaningless response and closes the connection.

^^^^^^^^^^^^^^^^^^^^
Either you use cache control headers which would result in responding
with HTTP status code 304 (Not Modified) if the data was not changed,
or you should respond _properly_ with 204 (No Content) in the case you
detect no change of data server-side. This will probably fix your IE
problem as well.

Also note that your checks should be done with reasonable delay, so use
setTimeout/setInterval() with a timeout/interval length greater than 50
(ms; probably 5000 [5 s] is a nice value), and do not run it in a tight
loop.
HTH

PointedEars

[Steve Kirkendall]

Thomas 'PointedEars' Lahn wrote:

Steve Kirkendall wrote:

I'm working on an project that will run on an intranet. I need to
have pages be reloaded automatically when the data they reflect is
changed at the server. For Netscape/Mozilla/Firefox, I do this by
having the page use JavaScript to read a special "/update.html" URL
in the background. For this URL, the server doesn't return anything
until it detects a change in the data, at which point it sends a
meaningless response and closes the connection.

^^^^^^^^^^^^^^^^^^^^
Either you use cache control headers which would result in responding
with HTTP status code 304 (Not Modified) if the data was not changed,
or you should respond _properly_ with 204 (No Content) in the case you
detect no change of data server-side. This will probably fix your IE
problem as well.

Also note that your checks should be done with reasonable delay, so use
setTimeout/setInterval() with a timeout/interval length greater than 50
(ms; probably 5000 [5 s] is a nice value), and do not run it in a tight
loop.

There is no looping involved. Maybe I was too vague in describing my
application.

It's a scheduling system. The users spend most of their time looking
at a page called /sasked.html. That page contains JavaScript which
sends a single /update.html request. When that request is complete
(which is likely to be several minutes later), the browser reloads
the /sasked.html page. The /update.html request is loaded in the
background so it won't interfere with the user's ability to view and
interact with the /sasked.html page. The JavaScript code in
/sasked.html looks like this...

var mygetsig = XmlHttp.create();
mygetsig.open("GET",
"/update.html?timeout=600&before=today&after=today&p oolid=1", true);
mygetsig.onreadystatechange = function()
{
if (mygetsig.readyState == 4)
{
if (mygetsig.responseText != "")
{
location.reload();
}
}
}
mygetsig.send(null);

.... where XmlHttp is a wrapper to hide the XmlHttpRequest/ActiveX
incompatibility.

When I said /update.html sends a "meaningless response", I meant that it
sends back a normal HTTP 200 response containing a nicely formatted HTML
document, with a table listing all of the changes... which the browser
ignores since *any* change tells it to reload /sasked.html from scratch.

[RobG]

Steve Kirkendall wrote:

I need a work-around for an IE limitation, regarding fetching HTTP
documents in the background. It takes a bit of explaining; please
be patient...

I'm working on an project that will run on an intranet. I need to
have pages be reloaded automatically when the data they reflect is
changed at the server. For Netscape/Mozilla/Firefox, I do this by
having the page use JavaScript to read a special "/update.html" URL
in the background. For this URL, the server doesn't return anything
until it detects a change in the data, at which point it sends a
meaningless response and closes the connection. The browser's
JavaScript code then reloads the main page, showing the new data.
It works great!
How are you reloading the page, e.g. window.location.reload(true) or
some other method?

But there's this other browser, "Internet Explorer", which I must
also support. The "slow /update.html" strategy *seems* to work until
I exit & reload the main page multiple times, without any changes
being detected in the data. If I do that, then the browser gets stuck.
"Exit & reload" - exit what, IE? Or do you just navigate elsewhere then
come back to the page? Or do you mean close the popup then open it again?

IE has some known memory issues, particularly in regard to closures. If
you show how you are calling the function that sends periodic requests
to the server, more help may be available.

Apparently IE doesn't terminate backgrounded JavaScript when you exit
the page that started it. For my applicatation, this means IE tries
to accumulate extra backgrounded "/update.html" requests each time
you exit & reload the main page. Also, it seems that IE only allows
about 6 TCP connection in total, so once it's accumulated 6
"/update.html" requests, it can't read anything else.
This may indicate a problem with cloures, but without seeing the code,
it's impossible to say. Guesses are possible but likely not helpful.
[...]
Is there some option that IE's JavaScript environment uses to indicate
whether backgrounded tasks should die when the page is unloaded?
Have you tried using XMLHttpRequest and script to update the page,
rather than using a reload? It is likely faster and the user won't be
bothered by the re-loads (flashing, trashing any updates in progress,
etc.). You may have to do more session management though, as a user
might be in the middle of updating an entry when you update it from
behind...

Is there a way to kill them on a document unload event, or some
similar event?

Try Richard Cornford's Finalizer:

<URL:http://www.litotes.demon.co.uk/example_scripts/finalizer.html>

"Finalizer is a general function designed to arrange that a function
passed to it as an argument is executed during the onunload event in a
browser. Its primary purpose is to address the memory leak problem on IE
browsers but has been written to provide the facility cross-browser as
it may have other applications."
--
Rob

[Thomas 'PointedEars' Lahn]

Steve Kirkendall wrote:

Thomas 'PointedEars' Lahn wrote:

Also note that your checks should be done with reasonable delay, so use
setTimeout/setInterval() with a timeout/interval length greater than 50
(ms; probably 5000 [5 s] is a nice value), and do not run it in a tight
loop.
There is no looping involved. Maybe I was too vague in describing my
application.

[asynchronous request] The JavaScript code in /sasked.html looks like
this...

var mygetsig = XmlHttp.create();
mygetsig.open("GET",
"/update.html?timeout=600&before=today&after=today&p oolid=1", true);
mygetsig.onreadystatechange = function()
{
if (mygetsig.readyState == 4)
{
if (mygetsig.responseText != "")
{
location.reload();
}
}
}
mygetsig.send(null);
[...]

OK, but the handler applies to but one request; HTTP is not stateful.
As you want to check for changes server-side and update automatically,
you will have to send that repeatedly, have you not? And probably
there is the problem.
When I said /update.html sends a "meaningless response", I meant that it
sends back a normal HTTP 200 response containing a nicely formatted HTML
document

Wait a minute. The above looks like client-side code, so /update.html does
not send a response at all; it sends a request (to the server) or receives
a response (from the server). Are we talking about server-side XMLHTTP
scripting here?
PointedEars

[Steve Kirkendall]

Thomas 'PointedEars' Lahn wrote:

Steve Kirkendall wrote:

Thomas 'PointedEars' Lahn wrote:

Also note that your checks should be done with reasonable delay, so use
setTimeout/setInterval() with a timeout/interval length greater than 50
(ms; probably 5000 [5 s] is a nice value), and do not run it in a tight
loop.

There is no looping involved. Maybe I was too vague in describing my
application.

[asynchronous request] The JavaScript code in /sasked.html looks like
this...

var mygetsig = XmlHttp.create();
mygetsig.open("GET",
"/update.html?timeout=600&before=today&after=today&p oolid=1", true);
mygetsig.onreadystatechange = function()
{
if (mygetsig.readyState == 4)
{
if (mygetsig.responseText != "")
{
location.reload();
}
}
}
mygetsig.send(null);
[...]

OK, but the handler applies to but one request; HTTP is not stateful.
As you want to check for changes server-side and update automatically,
you will have to send that repeatedly, have you not? And probably
there is the problem.

The odd part here is that the /update.html request isn't instantaneous.
It keeps the connection open until something changes on the server,
which may take many minutes. The /sasked.html page uses the above
JavaScript code in the browser to submit a single /update.html request,
and reload the /sasked.html page when the /update.html request completes.

The reloaded /sasked.html page will have its own copy of the above
JavaScript, so it will begin a new /update.html request to wait for
the next change. I guess in that sense, /update.html will be sent
repeatedly.

When I said /update.html sends a "meaningless response", I meant that it
sends back a normal HTTP 200 response containing a nicely formatted HTML
document

Wait a minute. The above looks like client-side code, so /update.html
does not send a response at all; it sends a request (to the server) or
receives a response (from the server). Are we talking about
server-side XMLHTTP scripting here?

The server is responsible for detecting changes. If you're curious,
it does this by maintaining a change log. When the /update.html request
is received, it waits for at least one record to be added to the log
that matches any parameters passed with the request. The server then
sends those records to the client and closes the connection.

For example, the /sasked.html page shows the schedule for a single
date, for a specific scheduling pool. It passes /update.html parameters
describing that date and pool, so that somebody looking at today's
schedule for pool#1 won't have their page refreshed when somebody
changes a different date, or a different pool. But as soon as somebody
changes today's pool#1 then /update.html will send back the change
log record, and /sasked.html will reload itself to show the altered
data.

And, really, that part works okay. Where I run into trouble is if
the user hits the "next day" button repeatedly. All non-IE browsers
terminate the /update.html request automatically, but IE keeps it
open. Each time the user hits the "next day" button, another
/update.html is opened until, rather quickly, IE hits a limit and
refuses to open any more connections, not even to load the next
/sasked.html page. It's stuck, totally unresponsive; even the "Stop"
toolbar button doesn't work.

Thanks again for taking time to help me with this.

[Steve Kirkendall]

RobG wrote:

Steve Kirkendall wrote:

I need a work-around for an IE limitation, regarding fetching HTTP
documents in the background. It takes a bit of explaining; please
be patient...

I'm working on an project that will run on an intranet. I need to
have pages be reloaded automatically when the data they reflect is
changed at the server. For Netscape/Mozilla/Firefox, I do this by
having the page use JavaScript to read a special "/update.html" URL
in the background. For this URL, the server doesn't return anything
until it detects a change in the data, at which point it sends a
meaningless response and closes the connection. The browser's
JavaScript code then reloads the main page, showing the new data.
It works great!
How are you reloading the page, e.g. window.location.reload(true) or
some other method?

Yes, location.reload(). I didn't know it accepted a Boolean flag though.

But there's this other browser, "Internet Explorer", which I must
also support. The "slow /update.html" strategy *seems* to work until
I exit & reload the main page multiple times, without any changes
being detected in the data. If I do that, then the browser gets stuck.

"Exit & reload" - exit what, IE? Or do you just navigate elsewhere then
come back to the page? Or do you mean close the popup then open it again?

For the main window, "exit & reload" basically means to resubmit the
same request, possibly with different parameters. I'm implementing a
scheduling system, in which /sasked.html shows a single day's schedule
for a particular scheduling pool. The page has a "next day" button to
increment the date. If the user hits "next day" repeatedly, then every
browser except IE works just fine, but IE gets stuck after about 6 clicks.
It goes completely unresponsive; even the "Stop" toolbar button doesn't
work. If I try to kill it with <Ctrl-Alt-Delete>, I get a warning that
it is "not responding".
IE has some known memory issues, particularly in regard to closures. If
you show how you are calling the function that sends periodic requests
to the server, more help may be available.

Apparently IE doesn't terminate backgrounded JavaScript when you exit
the page that started it. For my applicatation, this means IE tries
to accumulate extra backgrounded "/update.html" requests each time
you exit & reload the main page. Also, it seems that IE only allows
about 6 TCP connection in total, so once it's accumulated 6
"/update.html" requests, it can't read anything else.

This may indicate a problem with cloures, but without seeing the code,
it's impossible to say. Guesses are possible but likely not helpful.

Nothing fancy. Just use XmlHttpRequest (with a portability wrapper to
get around IE's ActiveX quirk). Each /sasked.html document loads
/update.html once, since it only needs to reload itself once.

var mygetsig = XmlHttp.create();
mygetsig.open("GET",
"/update.html?timeout=600&before=%dflt_date;&after=% dflt_date;&poolid=&dlft_poolid;",
true);
mygetsig.onreadystatechange = function()
{
if (mygetsig.readyState == 4)
{
if (mygetsig.responseText != "")
{
location.reload();
}
}
}
mygetsig.send(null);

Is there some option that IE's JavaScript environment uses to indicate
whether backgrounded tasks should die when the page is unloaded?

Have you tried using XMLHttpRequest and script to update the page,
rather than using a reload? It is likely faster and the user won't be
bothered by the re-loads (flashing, trashing any updates in progress,
etc.). You may have to do more session management though, as a user
might be in the middle of updating an entry when you update it from
behind...

No, I haven't tried that. The /update.html request returns a list of
changes, but it doesn't return the complete details of every changed
timeslot in the schedule. (The /update.html file accesses a change log.
The log is intended to show the modification history of each appointment
in the schedule. So, for example, if an appointment is moved from 10am
to 11am, then it's appropriate to have one "move" record in the log, and
not a pair of "10am" and "11am" records.) This makes patchwork updating
of the page very difficult.

The /sasked.html page contains a link for each timeslot which brings
up a pop-up window for editing the timeslot's details. Very little
inputting is ever done directly on the /sasked.html page, so trashing
an update in progress isn't really an issue. And IE is the only
browser that flashes.

Someday, yeah, it'd be nice to parse the response of something like
/update.html and reconfigure the page contents without reloading it.
But that's a long way off.

Is there a way to kill them on a document unload event, or some
similar event?

Try Richard Cornford's Finalizer:

<URL:http://www.litotes.demon.co.uk/example_scripts/finalizer.html>

"Finalizer is a general function designed to arrange that a function
passed to it as an argument is executed during the onunload event in a
browser. Its primary purpose is to address the memory leak problem on IE
browsers but has been written to provide the facility cross-browser as
it may have other applications."

How is this different from using <body onunload="...">? And what function
should I call to terminate an XmlHttpRequest that's in progress? Or do
I just need to clobber the <script> element that runs the request, to
work around IE's memory leak?

Hmm... This could be interesting. I still have questions but now I have
some new ideas to play with. Thanks for the response, and if you have
anything more to add, I'd be grateful for that too.

[Thomas 'PointedEars' Lahn]

Steve Kirkendall wrote:

Thomas 'PointedEars' Lahn wrote:

Steve Kirkendall wrote:

[XMLHttpRequest code]
OK, but the handler applies to but one request; HTTP is not stateful.
As you want to check for changes server-side and update automatically,
you will have to send that repeatedly, have you not? And probably
there is the problem.

The odd part here is that the /update.html request isn't instantaneous.
It keeps the connection open until something changes on the server,
which may take many minutes. The /sasked.html page uses the above
JavaScript code in the browser to submit a single /update.html request,
and reload the /sasked.html page when the /update.html request completes.
[...]
And, really, that part works okay. Where I run into trouble is if
the user hits the "next day" button repeatedly. All non-IE browsers
terminate the /update.html request automatically,

I doubt that very much. It is more likely that the number of connections
is limited to a greater value there than it is in IE by default, and so it
takes more additional requests to reach that limit.
but IE keeps it open.
It is perfectly reasonable of IE or any HTTP client to open a new connection
for a new request and keep the previous one open instead until a response
is received for the last request submitted through it.
Each time the user hits the "next day" button, another
/update.html is opened until, rather quickly, IE hits a limit and
refuses to open any more connections, not even to load the next
/sasked.html page.
And it does so rightfully.
It's stuck, totally unresponsive; even the "Stop" toolbar button doesn't
work.

Your problem is not due to scripting but that you are trying to make HTTP
stateful, which it is not; hence the unreliable results produced.

You should use timeouts/intervals and proper status code responses instead
as I suggested above. This does not only produce reliable results with any
HTTP client but also reduces client, network, and server load a lot (which
probably was an important reason why HTTP was made a stateful protocol in
the first place).
HTH

PointedEars

[Thomas 'PointedEars' Lahn]

Steve Kirkendall wrote:

Thomas 'PointedEars' Lahn wrote:

Steve Kirkendall wrote:

[XMLHttpRequest code]
OK, but the handler applies to but one request; HTTP is not stateful.
As you want to check for changes server-side and update automatically,
you will have to send that repeatedly, have you not? And probably
there is the problem.

The odd part here is that the /update.html request isn't instantaneous.
It keeps the connection open until something changes on the server,
which may take many minutes. The /sasked.html page uses the above
JavaScript code in the browser to submit a single /update.html request,
and reload the /sasked.html page when the /update.html request completes.
[...]
And, really, that part works okay. Where I run into trouble is if
the user hits the "next day" button repeatedly. All non-IE browsers
terminate the /update.html request automatically,

I doubt that very much. It is more likely that the number of connections
is limited to a greater value there than it is in IE by default, and so it
takes more additional requests to reach that limit.
but IE keeps it open.
It is perfectly reasonable of IE or any HTTP client to open a new connection
for a new request and keep the previous one open instead until a response
is received for the last request submitted through it.
Each time the user hits the "next day" button, another
/update.html is opened until, rather quickly, IE hits a limit and
refuses to open any more connections, not even to load the next
/sasked.html page.
And it does so rightfully.
It's stuck, totally unresponsive; even the "Stop" toolbar button doesn't
work.

Your problem is not due to scripting but that you are trying to make HTTP
stateful, which it is not; hence the unreliable results produced.

You should use timeouts/intervals and proper status code responses instead
as I suggested above. This does not only produce reliable results with any
HTTP client but also reduces client, network, and server load a lot (which
probably was an important reason why HTTP was made a non-stateful protocol
in
the first place).
HTH

PointedEars

[Jasen Betts]

On 2006-01-20, Steve Kirkendall <sk*********@dsl-only.net> wrote:

Thomas 'PointedEars' Lahn wrote:

And, really, that part works okay. Where I run into trouble is if
the user hits the "next day" button repeatedly. All non-IE browsers
terminate the /update.html request automatically, but IE keeps it
open. Each time the user hits the "next day" button, another
/update.html is opened until, rather quickly, IE hits a limit and
refuses to open any more connections, not even to load the next
/sasked.html page. It's stuck, totally unresponsive; even the "Stop"
toolbar button doesn't work.
could ypu put something ing the onclick of the "next day" button to fix that?
there's also document.onbeforeunload that could be used.

I'm not sure of the exact mechanics required to cancel a request.
there may be a "cancel" or "close" method.
Thanks again for taking time to help me with this.

--

Bye.
Jasen

[Richard Cornford]

Steve Kirkendall wrote:
<snip>

And what function should I call to terminate an XmlHttpRequest
that's in progress?

<snip>

XmlHttpRequest objects have an - abort - method for cancelling requests
that are not complete. (Apparently it is not a good idea to abort
requests when their readyState is 0 or 4).

Richard.

[Norman L. DeForest]

On Thu, 19 Jan 2006, Steve Kirkendall wrote:

I need a work-around for an IE limitation, regarding fetching HTTP
documents in the background. It takes a bit of explaining; please
be patient...

I'm working on an project that will run on an intranet. I need to
have pages be reloaded automatically when the data they reflect is
changed at the server. For Netscape/Mozilla/Firefox, I do this by
having the page use JavaScript to read a special "/update.html" URL
in the background. For this URL, the server doesn't return anything
until it detects a change in the data, at which point it sends a
meaningless response and closes the connection. The browser's
JavaScript code then reloads the main page, showing the new data.
It works great!

But there's this other browser, "Internet Explorer", which I must
also support. The "slow /update.html" strategy *seems* to work until
I exit & reload the main page multiple times, without any changes
being detected in the data. If I do that, then the browser gets stuck.

Apparently IE doesn't terminate backgrounded JavaScript when you exit
the page that started it. For my applicatation, this means IE tries
to accumulate extra backgrounded "/update.html" requests each time
you exit & reload the main page. Also, it seems that IE only allows
about 6 TCP connection in total, so once it's accumulated 6
"/update.html" requests, it can't read anything else.

I got the background "/update.html" strategy to work in IE, more or
less, by making it remember which thread is being used to serve the
slow "/update.html" request for each client, and then when a new thread
wants to do serve a new "/update.html" for the same client, it first
kills the old thread and closes the old connection. I consider this
a nasty hack, but it works.

Except that now I want to allow each client to perform two "/update.html"
requests in parallel -- one for the main window, and one for a pop-up
window. (The /update.html request accepts parameters describing what,
exactly, the client wants to wait for. The main window and pop-up window
will wait for different things.) I tried extending the same "kill the
old thread" hack to handle two connections, but it doesn't seem to be
working. I may yet get it to work, but before I waste a lot of time
on it, I thought I'd ask whether anybody else can suggest something
better.

To reitterate: I want to allow each client to have two windows that
immediately reload their pages when the server's data changes.

Is there some option that IE's JavaScript environment uses to indicate
whether backgrounded tasks should die when the page is unloaded?

Is there a way to kill them on a document unload event, or some
similar event?

Thanks in advance for any advice you can give.

I have an idea (which I can't test as I don't have control of any
server and also refuse to use IE for anything but Windows updates and
testing my own pages) but I have no idea whether it would work.

If a page is redirected with reply code 307, can IE detect the new page
name and can JavaScript get the temporary name?

If so, perhaps you could:

1. Have the JavaScript test for IE and request "/updateIE.html"
instead of "/update.html" when IE is detected (and do so
periodically).

2. Have the server act the same as it does now for a request
for "/update.html".

3. Have the server *temporarily* redirect "/updateIE.html requests
(using reply code 307) to "foo.html" or "bar.html" depending of
whether an update is available. "foo.html" could be a tiny file,
<html><body><p>&nbsp;</p></body></html>
that simply flags by its file name that no information is available
and "bar.html" could be a similar file that flags that information
*is* available.

4. Have the IE JavaScript test for the file name of the returned page
and act accordingly depending on whether it's "foo.html" or "bar.html".

Since the files would actually be fetched, no unsatisfied requests would
be left pending.

For two windows, use two different sets of filenames.

Creating two tiny unobtrusive windows to "display" the request results
is left as an exercise. Or try fetching two 1x1 GIFs and redirect
"/update.gif" to "foo.gif" or "bar.gif" depending on the results
and see if IE can detect the redirected image file names.

<discworld>
"Hey, it's a million-to-one chance so it might work."
</discworld>

--
Norman De Forest http://www.chebucto.ns.ca/~af380/Profile.html
af***@chebucto.ns.ca [=||=] (At the Sign of the Flashing Cursor)
"Oh how I miss the days when it was easier to catch gonorhea than a
computer virus." -- Big Will in alt.comp.virus, March 9, 2005