473,704 Members | 4,514 Online
Bytes | Software Development & Data Engineering Community
+ Post

Home Posts Topics Members FAQ

MSXML2.XMLHTTP

I wrote a small script that grabs two CSV files [links to the data files]
from a remote web site, parses them out and displays them in scrolling divs.
The first file has a little over 27k records, the second has less. It
retrieves the data pretty quick but it takes awhile to write the page.

Is there a better alternative to this approach?
This is my page:
http://kiddanger.com/lab/getsaveurl.asp

This is the relevant code to retrieve the data:

function strQuote(strURL )
dim objXML
set objXML = CreateObject("M SXML2.ServerXML HTTP")
objXML.Open "GET", strURL, False
objXML.Send
strQuote = objXML.Response Text
set objXML = nothing
end function

I split the data into an array and then split that into a new array because
the delimeters are line feed and comma, respectively.

TIA...

--
Roland Hall
/* This information is distributed in the hope that it will be useful, but
without any warranty; without even the implied warranty of merchantability
or fitness for a particular purpose. */
Technet Script Center - http://www.microsoft.com/technet/scriptcenter/
WSH 5.6 Documentation - http://msdn.microsoft.com/downloads/list/webdev.asp
MSDN Library - http://msdn.microsoft.com/library/default.asp
Jul 22 '05 #1
23 7484
Roland Hall wrote:
I wrote a small script that grabs two CSV files [links to the data
files] from a remote web site, parses them out and displays them in
scrolling divs. The first file has a little over 27k records, the
second has less. It retrieves the data pretty quick but it takes
awhile to write the page.

Is there a better alternative to this approach?
This is my page:
http://kiddanger.com/lab/getsaveurl.asp

This is the relevant code to retrieve the data:

function strQuote(strURL )
dim objXML
set objXML = CreateObject("M SXML2.ServerXML HTTP")
objXML.Open "GET", strURL, False
objXML.Send
strQuote = objXML.Response Text
set objXML = nothing
end function

I split the data into an array and then split that into a new array
because the delimeters are line feed and comma, respectively.

TIA...


It's pretty tough to comment on this. You've identified the bottleneck as
the process of writing the data to the page, so the strQuote function is not
relevant, is it? What you do with the array contents seems to be more
relevant, at least to me.

Somebody (I think it might have been Chris Hohmann) posted an analysis of
different techniques for generating large blocks of html a few weeks ago
that you may find interesting.

Bob Barrows
--
Microsoft MVP -- ASP/ASP.NET
Please reply to the newsgroup. The email account listed in my From
header is my spam trap, so I don't check it very often. You will get a
quicker response by posting to the newsgroup.
Jul 22 '05 #2
"Bob Barrows [MVP]" wrote in message
news:uC******** ******@TK2MSFTN GP11.phx.gbl...
: Roland Hall wrote:
: > I wrote a small script that grabs two CSV files [links to the data
: > files] from a remote web site, parses them out and displays them in
: > scrolling divs. The first file has a little over 27k records, the
: > second has less. It retrieves the data pretty quick but it takes
: > awhile to write the page.
: >
: > Is there a better alternative to this approach?
: > This is my page:
: > http://kiddanger.com/lab/getsaveurl.asp
: >
: > This is the relevant code to retrieve the data:
: >
: > function strQuote(strURL )
: > dim objXML
: > set objXML = CreateObject("M SXML2.ServerXML HTTP")
: > objXML.Open "GET", strURL, False
: > objXML.Send
: > strQuote = objXML.Response Text
: > set objXML = nothing
: > end function
: >
: > I split the data into an array and then split that into a new array
: > because the delimeters are line feed and comma, respectively.
: >
: > TIA...
: >
:
: It's pretty tough to comment on this. You've identified the bottleneck as
: the process of writing the data to the page, so the strQuote function is
not
: relevant, is it? What you do with the array contents seems to be more
: relevant, at least to me.

Hi Bob. Thanks for responding.

Perhaps. I'm assuming the data is retrieved due to the activity light on my
switch. I have not actually put timers in, which I guess would be the next
test.

:
: Somebody (I think it might have been Chris Hohmann) posted an analysis of
: different techniques for generating large blocks of html a few weeks ago
: that you may find interesting.

I searched in this NG for all of Chris' posting and didn't find anything.
Then I searched for the reference you made and didn't find anything that way
either. Here is my subroutine for parsing the data and perhaps someone will
notice something that will help speed it up.

sub strWrite(str)
dim arr, i, arr2, j
arr = split(str,vbLf)
prt("<fieldset> <legend style=""font-weight: bold"">" & arr(0) & " " &
strURL & "</legend>")
prt("<div style=""height: 200px; overflow: auto; width: 950px"">")
prt("<table style=""padding : 3px"">")
for i = 1 to ubound(arr)
arr2 = split(arr(i),", ")
if i = 1 then
prt("<tr style=""font-weight: bold"">")
else
if i mod 2 = 0 then
prt("<tr style=""backgro und-color: #ddd"">")
else
prt("<tr>")
end if
end if
for j = 0 to ubound(arr2)
prt("<td>" & arr2(j))
next
next
prt("</table>")
prt("</div>")
prt("</fieldset>")
end sub

These are the calls for the two files:

dim strURL
strURL = "http://neustar.us/reports/rgp/domains_in_rgp. csv"
strWrite strQuote(strURL )
strURL = "http://neustar.us/reports/rgp/domains_out_rgp .csv"
strWrite strQuote(strUrl )

I made some changes to my buffer and some variables and it's noticably
faster. It still takes about 4-5 seconds to parse the data but I'm not sure
if that's all that bad for that amount.

I'm testing with two links, one on the Internet and one on my Intranet. The
Internet link normally displays them almost simultaneously. The Intranet
displays the first file, then almost as much of a delay for the next, which
is what I expected.

http://kiddanger.com/lab/getsaveurl.asp Internet
http://netfraud.us/asp/rgpr.asp Intranet

I wonder if I wrote everything to a string and then made only one write
statement if that would be faster. Any ideas?

--
Roland Hall
/* This information is distributed in the hope that it will be useful, but
without any warranty; without even the implied warranty of merchantability
or fitness for a particular purpose. */
Technet Script Center - http://www.microsoft.com/technet/scriptcenter/
WSH 5.6 Documentation - http://msdn.microsoft.com/downloads/list/webdev.asp
MSDN Library - http://msdn.microsoft.com/library/default.asp
Jul 22 '05 #3
I added the record count to the legend and now I know why the second one is
a lot faster. 1/10 the amount of records.
Jul 22 '05 #4
Roland Hall wrote:
...Here is my subroutine for parsing the data and perhaps someone
will notice something that will help speed it up...

for i = 1 to ubound(arr)
arr2 = split(arr(i),", ")
if i = 1 then
prt("<tr style=""font-weight: bold"">")
else
if i mod 2 = 0 then
prt("<tr style=""backgro und-color: #ddd"">")
else
prt("<tr>")
end if
end if
for j = 0 to ubound(arr2)
prt("<td>" & arr2(j))
next
next


Have you tried using Replace() instead of split?

for i = 1 to ubound(arr)
if i = 1 then
prt("<tr style=""font-weight: bold"">")
else
if i mod 2 = 0 then
prt("<tr style=""backgro und-color: #ddd"">")
else
prt("<tr>")
end if
end if

prt(Replace(arr (i),",","<td>") )
next

--
Dave Anderson

Unsolicited commercial email will be read at a cost of $500 per message. Use
of this email address implies consent to these terms. Please do not contact
me directly or ask me to contact you directly for assistance. If your
question is worth asking, it's worth posting.
Jul 22 '05 #5
Roland Hall wrote:
Somebody (I think it might have been Chris Hohmann) posted an
analysis of different techniques for generating large blocks of html
a few weeks ago that you may find interesting.


I searched in this NG for all of Chris' posting and didn't find
anything. Then I searched for the reference you made and didn't find
anything that way either.


Darn. I just tried to find it as well, and failed. ISTR that the consensus
was that adding the individual strings to an array and then using Join to
combine them was the fastest method. Combined with Dave's idea, you would
get something like this:

sub strWrite(str)
dim arr, i, arr2, j
dim arHTML(), sRow
arr = split(str,vbLf)
prt("<fieldset> <legend style=""font-weight: bold"">" & arr(0) & " " &
strURL & "</legend>")
prt("<div style=""height: 200px; overflow: auto; width: 950px"">")
prt("<table style=""padding : 3px"">")
redim arHTML(ubound(a rr))
for i = 1 to ubound(arr)
if i = 1 then
sRow= "<tr style=""font-weight: bold"">"
else
if i mod 2 = 0 then
sRow="<tr style=""backgro und-color: #ddd"">"
else
sRow="<tr>"
end if
end if
sRow=sRow & vbCrLf & vbTab & Replace(arr(i), ",","<td>") )
arHTML(i) =sRow
next
prt(Join(arHTML ,vbCrLf))
prt("</table>")
prt("</div>")
prt("</fieldset>")
end sub
:

Bob Barrows

--
Microsoft MVP -- ASP/ASP.NET
Please reply to the newsgroup. The email account listed in my From
header is my spam trap, so I don't check it very often. You will get a
quicker response by posting to the newsgroup.
Jul 22 '05 #6
"Dave Anderson" wrote in message
news:%2******** ********@TK2MSF TNGP11.phx.gbl. ..
: Roland Hall wrote:
: > ...Here is my subroutine for parsing the data and perhaps someone
: > will notice something that will help speed it up...
: >
: > for i = 1 to ubound(arr)
: > arr2 = split(arr(i),", ")
: > if i = 1 then
: > prt("<tr style=""font-weight: bold"">")
: > else
: > if i mod 2 = 0 then
: > prt("<tr style=""backgro und-color: #ddd"">")
: > else
: > prt("<tr>")
: > end if
: > end if
: > for j = 0 to ubound(arr2)
: > prt("<td>" & arr2(j))
: > next
: > next
:
: Have you tried using Replace() instead of split?
:
: for i = 1 to ubound(arr)
: if i = 1 then
: prt("<tr style=""font-weight: bold"">")
: else
: if i mod 2 = 0 then
: prt("<tr style=""backgro und-color: #ddd"">")
: else
: prt("<tr>")
: end if
: end if
:
: prt(Replace(arr (i),",","<td>") )
: next
:
:
:

Thank Dave. I'll put a timer on it to see if the difference. Hard to tell
just looking. I know it's hard to write this stuff off the top of your
head, especially not seeing the raw data but I needed to make one mod to
your suggestion. There is no leading , (comma) so another <td> had to be
inserted.

prt("<td>" & replace(arr(i), ",","<td>") )

Thanks for your insight. I like that a lot better than the array loop.

Roland
Jul 22 '05 #7
Roland Hall wrote:
I wrote a small script that grabs two CSV files [links to the data files]
from a remote web site, parses them out and displays them in scrolling divs.
The first file has a little over 27k records, the second has less. It
retrieves the data pretty quick but it takes awhile to write the page.

Is there a better alternative to this approach?


How often are the CSV files updated at their remote site? If it's not
too frequently, then the files could be transferred to your server when
updated or by a periodically-executed script or Windows service. The
files could then be accessed locally (more quickly).

BTW 27,000 rows seems like an excessive amount of data for a user to
digest at once. Could the program present a search page (by field, by
alphabetic order, etc.) or summary page (listing categories) first? Then
the user could limit the search somewhat.

I would be tempted to periodically transfer the CSV file(s) to a local
directory and import the data into a database. Then an ASP page would
handle the search and presentation.
Jul 22 '05 #8
Roland Hall wrote:
There is no leading , (comma) so another <td> had to be inserted.

prt("<td>" & replace(arr(i), ",","<td>") )


I would go even further and reach for HTML completeness:

prt("<td>" & replace(arr(i), ",","</td><td>") & "</td>")

--
Dave Anderson

Unsolicited commercial email will be read at a cost of $500 per message. Use
of this email address implies consent to these terms. Please do not contact
me directly or ask me to contact you directly for assistance. If your
question is worth asking, it's worth posting.
Jul 22 '05 #9
Michael D. Kersey wrote:
BTW 27,000 rows seems like an excessive amount of data for
a user to digest at once...


That raises another point I forgot to address in my other post. If the
client machine is Internet Explorer, the table will be displayed all at
once, rather than line-by-line, no matter what buffering you use.

I have jobs that I occasionally run with ASP scripts, and I often set the
script up to spit out every changed record and/or every 100th record, or
something similar. I typically break the table every 10 or 20 rows by
inserting one of these: "</table><table>".

It has been my observation that IE displays nothing at all until the table
is closed, while Mozilla/Firefox/Opera will display each row as it arrives
(buffering must be off to see this in effect).

--
Dave Anderson

Unsolicited commercial email will be read at a cost of $500 per message. Use
of this email address implies consent to these terms. Please do not contact
me directly or ask me to contact you directly for assistance. If your
question is worth asking, it's worth posting.
Jul 22 '05 #10

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

3
12450
by: Dan Sikorsky | last post by:
Uploading from browser to server using Msxml2.XMLHTTP takes a long time about 15 minutes for a 1.5MB file at 37.2Kbps, although it does get there. Is there anyway to speed things up? here's the code ... var sFilename = (document.form1.myFileName.value); // create ADO-stream Object var ado_stream = new ActiveXObject("ADODB.Stream"); // create XML document with default header and primary node
17
14788
by: Patrick | last post by:
I am almost certain that I could use HTTP Post/Get to submit XML Web Service call (over SSL as well, if using Version 3 of MSXML2) from an ASP Application? However, would I only be able to call web-service in a an asynchronous mode (with a callback function)? If so, how?
5
11013
by: RK | last post by:
I am getting the "HTTP /1.1 405 - method not allowed" error when I am sending XML string over Msxml2.XMLHTTP object. I am sending data in POST, also passing querystring variable and here is the code. Also error comes only sometimes. var xmlhttp = new ActiveXObject("Msxml2.XMLHTTP"); xmlhttp.Open("POST", "test.aspx?qsVar="+qsVar, false); xmlhttp.setRequestHeader("Content-Type", "application/x-www-form-urlencoded"); xmlhttp.Send(xmldoc);
2
21803
by: Michael Christensen | last post by:
Hi How do I send an input-param to my web service with MSXML2.ServerXMLHTTP? Can this be done without the soap-toolkit? Can't find anything about it - looking forward getting some help :-) ------------- Function GetLatest() Dim url, xmlhttp, XMLDOM, XMLNode
1
6584
by: Raúl Martín | last post by:
I´ve a function in asp that run correctly but If I tried to change it forasp.net in asp: xmlHTTP = CreateObject("Microsoft.XMLHTTP") And I thought to use this sentence for asp.net but the server don´t response right. xmlHTTP = Server.CreateObject("MSXML2.ServerXMLHTTP")
2
8751
by: Maris Janis Vasilevskis | last post by:
Hi, Is it possible to force HttpWebRequest to do exactly (not approximately) the same as MSXML2.ServerXMLHTTP does? More details. I port JScript to JScript.NET I have a server (ASP invoking binary code; code not accessible). I send XML requests to it. If my request is accepted, ServerXMLHTTP returns XML with accept details. If my request is rejected, ServerXMLHTTP returns XML with reject details.
3
11297
by: BjörnHolmberg | last post by:
I'm trying to consume a WS from Excel (Office 98), seems that I can't send arguments into the WS. In order to pinpoint the problem I've written the following application i C# and VB6. It reproduces exaclty the same problem with less code. Any hints on how to solve this would be greatly appreciated. /Regards Björn The webservice is: using System; using System.Web.Services;
5
3784
by: Brent | last post by:
This AJAX stuff is all new to me. To try it out, I borrowed this code from a website: ======================== var http_request = false; function makeRequest(url) { http_request = false;
2
11308
by: noOby | last post by:
Hi developers, currently i'm doing a school project and i am stuck on MSXML2.XMLHTTP conversion from vb code to c#. Below is the vb Coding that i need to convert to C# Dim sUrl As String Dim xHttp As MSXML2.XMLHTTP
2
19225
by: Dave | last post by:
I am running the following code and I get an error: Set xmlHttp = Server.CreateObject("MSXML2.XMLHTTP.3.0") xmlHttp.Open "Get", URLToRSS, false xmlHttp.Send RSSXML = xmlHttp.ResponseText The error is: msxml3.dll error '800c0005'
0
8677
by: Hystou | last post by:
Most computers default to English, but sometimes we require a different language, especially when relocating. Forgot to request a specific language before your computer shipped? No problem! You can effortlessly switch the default language on Windows 10 without reinstalling. I'll walk you through it. First, let's disable language synchronization. With a Microsoft account, language settings sync across devices. To prevent any complications,...
0
9130
jinu1996
by: jinu1996 | last post by:
In today's digital age, having a compelling online presence is paramount for businesses aiming to thrive in a competitive landscape. At the heart of this digital strategy lies an intricately woven tapestry of website design and digital marketing. It's not merely about having a website; it's about crafting an immersive digital experience that captivates audiences and drives business growth. The Art of Business Website Design Your website is...
1
9020
by: Hystou | last post by:
Overview: Windows 11 and 10 have less user interface control over operating system update behaviour than previous versions of Windows. In Windows 11 and 10, there is no way to turn off the Windows Update option using the Control Panel or Settings app; it automatically checks for updates and installs any it finds, whether you like it or not. For most users, this new feature is actually very convenient. If you want to control the update process,...
0
8974
tracyyun
by: tracyyun | last post by:
Dear forum friends, With the development of smart home technology, a variety of wireless communication protocols have appeared on the market, such as Zigbee, Z-Wave, Wi-Fi, Bluetooth, etc. Each protocol has its own unique characteristics and advantages, but as a user who is planning to build a smart home system, I am a bit confused by the choice of these technologies. I'm particularly interested in Zigbee because I've heard it does some...
0
7879
agi2029
by: agi2029 | last post by:
Let's talk about the concept of autonomous AI software engineers and no-code agents. These AIs are designed to manage the entire lifecycle of a software development project—planning, coding, testing, and deployment—without human intervention. Imagine an AI that can take a project description, break it down, write the code, debug it, and then launch it, all on its own.... Now, this would greatly impact the work of software developers. The idea...
1
6603
isladogs
by: isladogs | last post by:
The next Access Europe User Group meeting will be on Wednesday 1 May 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM). In this session, we are pleased to welcome a new presenter, Adolph Dupré who will be discussing some powerful techniques for using class modules. He will explain when you may want to use classes instead of User Defined Types (UDT). For example, to manage the data in unbound forms. Adolph will...
0
5926
by: conductexam | last post by:
I have .net C# application in which I am extracting data from word file and save it in database particularly. To store word all data as it is I am converting the whole word file firstly in HTML and then checking html paragraph one by one. At the time of converting from word file to html my equations which are in the word document file was convert into image. Globals.ThisAddIn.Application.ActiveDocument.Select();...
2
2468
muto222
by: muto222 | last post by:
How can i add a mobile payment intergratation into php mysql website.
3
2074
bsmnconsultancy
by: bsmnconsultancy | last post by:
In today's digital era, a well-designed website is crucial for businesses looking to succeed. Whether you're a small business owner or a large corporation in Toronto, having a strong online presence can significantly impact your brand's success. BSMN Consultancy, a leader in Website Development in Toronto offers valuable insights into creating effective websites that not only look great but also perform exceptionally well. In this comprehensive...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.