469,920 Members | 2,353 Online
Bytes | Developer Community
New Post

Home Posts Topics Members FAQ

Post your question to a community of 469,920 developers. It's quick & easy.

doc.loadxml(axwebbrowser1.document.body.outerhtml) , doesn't work?

I am automating the navigation of a website. When I reach the end, it
changes to xml.

So my axwebbrowser1 has a bunch of xml data in it.

So here is what I am trying to do

Dim xmlText As String
Dim doc As New XmlDocument()
Dim myElem As XmlElement
xmlText = AxWebBrowser1.Document.Body.OuterHTML
doc.LoadXml(xmlText)

But the problem is I get an exception error at the
doc.loadxml(xmltext)
I found a streamreader example but I already have the xml data in the
axwebbrowser control.

How do I get it so I can parse through the nodes?

Thanks for any information
Nov 20 '05 #1
25 2781
what does the entire document look like?

either post it here, or give the url so we can see.

thx,

steve
"MeNotHome" <dtfa@****nospm****insightbb.com> wrote in message
news:c7********************************@4ax.com...
I am automating the navigation of a website. When I reach the end, it
changes to xml.

So my axwebbrowser1 has a bunch of xml data in it.

So here is what I am trying to do

Dim xmlText As String
Dim doc As New XmlDocument()
Dim myElem As XmlElement
xmlText = AxWebBrowser1.Document.Body.OuterHTML
doc.LoadXml(xmlText)

But the problem is I get an exception error at the
doc.loadxml(xmltext)
I found a streamreader example but I already have the xml data in the
axwebbrowser control.

How do I get it so I can parse through the nodes?

Thanks for any information

Nov 20 '05 #2
I wish I could post the website but it's on a password protected site.
But here is what the page looks like when viewed in ie6.

<?xml version="1.0" encoding="ISO-8859-1" ?>
- <ShipmentForecasts>
- <ShipmentForecast>
<extract-datetime
code="CST">2003-10-15T15:33:06.000-06:00</extract-datetime>
<FileKey>ALL1*23759*20011200*ALL1</FileKey>
<TransactionIdentification>000343717</TransactionIdentification>
<ReleaseIdentification Type="Original"
DateType="Delivery">20011200</ReleaseIdentification>
<DateTime>2003-10-15T21:33:06.000Z</DateTime>
<StartDateTime>2003-10-13T21:33:06.000Z</StartDateTime>
<EndDateTime>2004-10-06T00:00:00.000Z</EndDateTime>
- <ShipFrom>
- <Company>
etc etc....

Does this help at all?

Thanks for any advice you can give.

On Mon, 20 Oct 2003 11:42:55 -0500, "steve" <as*@abc.com> wrote:
what does the entire document look like?

either post it here, or give the url so we can see.

thx,

steve
"MeNotHome" <dtfa@****nospm****insightbb.com> wrote in message
news:c7********************************@4ax.com.. .
I am automating the navigation of a website. When I reach the end, it
changes to xml.

So my axwebbrowser1 has a bunch of xml data in it.

So here is what I am trying to do

Dim xmlText As String
Dim doc As New XmlDocument()
Dim myElem As XmlElement
xmlText = AxWebBrowser1.Document.Body.OuterHTML
doc.LoadXml(xmlText)

But the problem is I get an exception error at the
doc.loadxml(xmltext)
I found a streamreader example but I already have the xml data in the
axwebbrowser control.

How do I get it so I can parse through the nodes?

Thanks for any information


Nov 20 '05 #3
you know...that looks fine to me. i don't see anything wrong with it. what
does the exception state?

here's some "quicky code" that will help you parse xml in the meantime...i
didn't have time to learn how the new xml stuff worked in .net so this was
my "5 minute problem solved" adaptation.

hth,

steve

Private Sub xmlParsing()
Dim xml As String
xml &= "<authorization>" & vbCrLf
xml &= " <record1>" & vbCrLf
xml &= " <id>1</id>" & vbCrLf
xml &= " <role>1</role>" & vbCrLf
xml &= " <description>system account</description>" & vbCrLf
xml &= " </record1>" & vbCrLf
xml &= " <record2>" & vbCrLf
xml &= " <id>2</id>" & vbCrLf
xml &= " <role>2</role>" & vbCrLf
xml &= " <description>administrator</description>" & vbCrLf
xml &= " </record2>" & vbCrLf
xml &= " <record3>" & vbCrLf
xml &= " <id>3</id>" & vbCrLf
xml &= " <role>3</role>" & vbCrLf
xml &= " <description>standard user</description>" & vbCrLf
xml &= " </record3>" & vbCrLf
xml &= "</authorization>" & vbCrLf
xml &= "<authorization>" & vbCrLf
xml &= " <record4>" & vbCrLf
xml &= " <id>4</id>" & vbCrLf
xml &= " <role>4</role>" & vbCrLf
xml &= " <description>system knob</description>" & vbCrLf
xml &= " </record4>" & vbCrLf
xml &= " <record5>" & vbCrLf
xml &= " <id>5</id>" & vbCrLf
xml &= " <role>5</role>" & vbCrLf
xml &= " <description>certified looser</description>" & vbCrLf
xml &= " </record5>" & vbCrLf
xml &= " <record6>" & vbCrLf
xml &= " <id>6</id>" & vbCrLf
xml &= " <role>6</role>" & vbCrLf
xml &= " <description>emmense whiner</description>" & vbCrLf
xml &= " </record6>" & vbCrLf
xml &= "</authorization>" & vbCrLf
Dim authorization() As String = getMultipleSegments("AUTHORIZATION", xml)
If authorization Is Nothing Then Return
Dim segment As String
For Each segment In authorization
Dim records() As String = getMultipleSegments("RECORD", segment)
If Not records Is Nothing Then
Dim description As String
Dim id As Integer
Dim record As String
Dim role As String
For Each record In records
id = CInt(getSegmentValue(getSingleSegment("ID", record), 0))
role = getSegmentValue(getSingleSegment("ROLE", record))
description = getSegmentValue(getSingleSegment("DESCRIPTION", record))
Console.WriteLine("id: " & id & ", role: " & role & ", description: " &
description)
Next
End If
Next
Console.ReadLine()
End Sub

Private Function getMultipleSegments(ByVal segment As String, ByVal xml As
String) As String()
If xml Is Nothing Then Exit Function
Dim pattern As String = "<\s*" & segment & "[^>]*>.*?<\s*/\s*" & segment &
"[^>]*?>"
Dim regExp As New Regex(pattern, RegexOptions.ExplicitCapture Or
RegexOptions.Singleline Or RegexOptions.IgnoreCase)
If Not regExp.IsMatch(xml) Then Exit Function
Dim match As Match
Dim result() As String
Dim index As Integer
For Each match In regExp.Matches(xml)
ReDim Preserve result(index)
result(index) = match.Value
index += 1
Next
Return result
End Function

Private Function getSingleSegment(ByVal segment As String, ByVal xml As
String) As String
If xml Is Nothing Then Exit Function
Dim pattern As String = "<\s*" & segment & "[^>]*>.*?<\s*/\s*" & segment &
"[^>]*?>"
Dim regExp As New Regex(pattern, RegexOptions.ExplicitCapture Or
RegexOptions.Singleline Or RegexOptions.IgnoreCase)
If Not regExp.IsMatch(xml) Then Exit Function
Dim match As Match = regExp.Match(xml)
Return match.Value
End Function

Private Overloads Function getSegmentValue(ByVal xml As String) As String
If xml Is Nothing Then Exit Function
Dim pattern As String = "<[^>]*>"
Dim regExp As New Regex(pattern)
If Not regExp.IsMatch(xml) Then Exit Function
Return regExp.Replace(xml, "")
End Function

Private Overloads Function getSegmentValue(ByVal xml As String, ByVal
emptyDefault As String) As String
Dim value As String = getSegmentValue(xml)
If value Is Nothing OrElse value.Trim = vbNullString Then value =
emptyDefault
Return value
End Function
"MeNotHome" <dtfa@****nospm****insightbb.com> wrote in message
news:6e********************************@4ax.com...
I wish I could post the website but it's on a password protected site.
But here is what the page looks like when viewed in ie6.

<?xml version="1.0" encoding="ISO-8859-1" ?>
- <ShipmentForecasts>
- <ShipmentForecast>
<extract-datetime
code="CST">2003-10-15T15:33:06.000-06:00</extract-datetime>
<FileKey>ALL1*23759*20011200*ALL1</FileKey>
<TransactionIdentification>000343717</TransactionIdentification>
<ReleaseIdentification Type="Original"
DateType="Delivery">20011200</ReleaseIdentification>
<DateTime>2003-10-15T21:33:06.000Z</DateTime>
<StartDateTime>2003-10-13T21:33:06.000Z</StartDateTime>
<EndDateTime>2004-10-06T00:00:00.000Z</EndDateTime>
- <ShipFrom>
- <Company>
etc etc....

Does this help at all?

Thanks for any advice you can give.

On Mon, 20 Oct 2003 11:42:55 -0500, "steve" <as*@abc.com> wrote:
what does the entire document look like?

either post it here, or give the url so we can see.

thx,

steve
"MeNotHome" <dtfa@****nospm****insightbb.com> wrote in message
news:c7********************************@4ax.com.. .
I am automating the navigation of a website. When I reach the end, it
changes to xml.

So my axwebbrowser1 has a bunch of xml data in it.

So here is what I am trying to do

Dim xmlText As String
Dim doc As New XmlDocument()
Dim myElem As XmlElement
xmlText = AxWebBrowser1.Document.Body.OuterHTML
doc.LoadXml(xmlText)

But the problem is I get an exception error at the
doc.loadxml(xmltext)
I found a streamreader example but I already have the xml data in the
axwebbrowser control.

How do I get it so I can parse through the nodes?

Thanks for any information

Nov 20 '05 #4
I am attempting to try out your code.

I am getting a couple of errors when inserting the code.

Inside GetMultipleSegments()
Dim regExp as New Regex(pattern Regex is undefined
Dim match as Match Match is undefined
Inside getSingleSegment
Regex is undefined and Match is undefined

insided Overloads function getSegmentValue
value = emptyDefault() is undefined

thanks for any help

On Mon, 20 Oct 2003 12:43:53 -0500, "steve" <as*@abc.com> wrote:
you know...that looks fine to me. i don't see anything wrong with it. what
does the exception state?

here's some "quicky code" that will help you parse xml in the meantime...i
didn't have time to learn how the new xml stuff worked in .net so this was
my "5 minute problem solved" adaptation.

hth,

steve

Private Sub xmlParsing()
Dim xml As String
xml &= "<authorization>" & vbCrLf
xml &= " <record1>" & vbCrLf
xml &= " <id>1</id>" & vbCrLf
xml &= " <role>1</role>" & vbCrLf
xml &= " <description>system account</description>" & vbCrLf
xml &= " </record1>" & vbCrLf
xml &= " <record2>" & vbCrLf
xml &= " <id>2</id>" & vbCrLf
xml &= " <role>2</role>" & vbCrLf
xml &= " <description>administrator</description>" & vbCrLf
xml &= " </record2>" & vbCrLf
xml &= " <record3>" & vbCrLf
xml &= " <id>3</id>" & vbCrLf
xml &= " <role>3</role>" & vbCrLf
xml &= " <description>standard user</description>" & vbCrLf
xml &= " </record3>" & vbCrLf
xml &= "</authorization>" & vbCrLf
xml &= "<authorization>" & vbCrLf
xml &= " <record4>" & vbCrLf
xml &= " <id>4</id>" & vbCrLf
xml &= " <role>4</role>" & vbCrLf
xml &= " <description>system knob</description>" & vbCrLf
xml &= " </record4>" & vbCrLf
xml &= " <record5>" & vbCrLf
xml &= " <id>5</id>" & vbCrLf
xml &= " <role>5</role>" & vbCrLf
xml &= " <description>certified looser</description>" & vbCrLf
xml &= " </record5>" & vbCrLf
xml &= " <record6>" & vbCrLf
xml &= " <id>6</id>" & vbCrLf
xml &= " <role>6</role>" & vbCrLf
xml &= " <description>emmense whiner</description>" & vbCrLf
xml &= " </record6>" & vbCrLf
xml &= "</authorization>" & vbCrLf
Dim authorization() As String = getMultipleSegments("AUTHORIZATION", xml)
If authorization Is Nothing Then Return
Dim segment As String
For Each segment In authorization
Dim records() As String = getMultipleSegments("RECORD", segment)
If Not records Is Nothing Then
Dim description As String
Dim id As Integer
Dim record As String
Dim role As String
For Each record In records
id = CInt(getSegmentValue(getSingleSegment("ID", record), 0))
role = getSegmentValue(getSingleSegment("ROLE", record))
description = getSegmentValue(getSingleSegment("DESCRIPTION", record))
Console.WriteLine("id: " & id & ", role: " & role & ", description: " &
description)
Next
End If
Next
Console.ReadLine()
End Sub

Private Function getMultipleSegments(ByVal segment As String, ByVal xml As
String) As String()
If xml Is Nothing Then Exit Function
Dim pattern As String = "<\s*" & segment & "[^>]*>.*?<\s*/\s*" & segment &
"[^>]*?>"
Dim regExp As New Regex(pattern, RegexOptions.ExplicitCapture Or
RegexOptions.Singleline Or RegexOptions.IgnoreCase)
If Not regExp.IsMatch(xml) Then Exit Function
Dim match As Match
Dim result() As String
Dim index As Integer
For Each match In regExp.Matches(xml)
ReDim Preserve result(index)
result(index) = match.Value
index += 1
Next
Return result
End Function

Private Function getSingleSegment(ByVal segment As String, ByVal xml As
String) As String
If xml Is Nothing Then Exit Function
Dim pattern As String = "<\s*" & segment & "[^>]*>.*?<\s*/\s*" & segment &
"[^>]*?>"
Dim regExp As New Regex(pattern, RegexOptions.ExplicitCapture Or
RegexOptions.Singleline Or RegexOptions.IgnoreCase)
If Not regExp.IsMatch(xml) Then Exit Function
Dim match As Match = regExp.Match(xml)
Return match.Value
End Function

Private Overloads Function getSegmentValue(ByVal xml As String) As String
If xml Is Nothing Then Exit Function
Dim pattern As String = "<[^>]*>"
Dim regExp As New Regex(pattern)
If Not regExp.IsMatch(xml) Then Exit Function
Return regExp.Replace(xml, "")
End Function

Private Overloads Function getSegmentValue(ByVal xml As String, ByVal
emptyDefault As String) As String
Dim value As String = getSegmentValue(xml)
If value Is Nothing OrElse value.Trim = vbNullString Then value =
emptyDefault
Return value
End Function
"MeNotHome" <dtfa@****nospm****insightbb.com> wrote in message
news:6e********************************@4ax.com.. .
I wish I could post the website but it's on a password protected site.
But here is what the page looks like when viewed in ie6.

<?xml version="1.0" encoding="ISO-8859-1" ?>
- <ShipmentForecasts>
- <ShipmentForecast>
<extract-datetime
code="CST">2003-10-15T15:33:06.000-06:00</extract-datetime>
<FileKey>ALL1*23759*20011200*ALL1</FileKey>
<TransactionIdentification>000343717</TransactionIdentification>
<ReleaseIdentification Type="Original"
DateType="Delivery">20011200</ReleaseIdentification>
<DateTime>2003-10-15T21:33:06.000Z</DateTime>
<StartDateTime>2003-10-13T21:33:06.000Z</StartDateTime>
<EndDateTime>2004-10-06T00:00:00.000Z</EndDateTime>
- <ShipFrom>
- <Company>
etc etc....

Does this help at all?

Thanks for any advice you can give.

On Mon, 20 Oct 2003 11:42:55 -0500, "steve" <as*@abc.com> wrote:
>what does the entire document look like?
>
>either post it here, or give the url so we can see.
>
>thx,
>
>steve
>
>
>"MeNotHome" <dtfa@****nospm****insightbb.com> wrote in message
>news:c7********************************@4ax.com.. .
>> I am automating the navigation of a website. When I reach the end, it
>> changes to xml.
>>
>> So my axwebbrowser1 has a bunch of xml data in it.
>>
>> So here is what I am trying to do
>>
>> Dim xmlText As String
>> Dim doc As New XmlDocument()
>> Dim myElem As XmlElement
>> xmlText = AxWebBrowser1.Document.Body.OuterHTML
>> doc.LoadXml(xmlText)
>>
>> But the problem is I get an exception error at the
>> doc.loadxml(xmltext)
>> I found a streamreader example but I already have the xml data in the
>> axwebbrowser control.
>>
>> How do I get it so I can parse through the nodes?
>>
>> Thanks for any information
>


Nov 20 '05 #5
Ok, I see I forgot an import.

One more thing.....

When I set xml=axwebbrowser1.document.body.outerhtml

I get all kinds of junk in the xml stirng.

It is displayed perfectly as an xml document in the axwebrowser
control but the string that is returned contains lots of extra junk in
it. And the <Node>xyz</Node> is totally missing from the xml string.

Any ideas on how to get the xml string loaded with only the real xml
data that is displayed in the axwebbrowser control?

thanks
Nov 20 '05 #6
Hi MeNo,

If you know when you've reached the xml stage, why not get the file
directly using an HttpRequest? Then you won't need to worry axBrowser messing
it up - you can stick it straight into your XmlDocument.

Regards,
Fergus
Nov 20 '05 #7
go to the command window (ctl + g) and type this in when you have it running
and loaded:

?axwebbrowser1.document.body

what shows up? if you don't see xml anywhere, then try:

?axwebbrowser1.document

i think the whole problem is there. one way to bypass the whole use of the
browser control is to create an http request and get the results. i have an
example if you wanna go that route.

let me know what the command window outputs.

thx,

steve
"MeNotHome" <dtfa@****nospm****insightbb.com> wrote in message
news:m7********************************@4ax.com...
Ok, I see I forgot an import.

One more thing.....

When I set xml=axwebbrowser1.document.body.outerhtml

I get all kinds of junk in the xml stirng.

It is displayed perfectly as an xml document in the axwebrowser
control but the string that is returned contains lots of extra junk in
it. And the <Node>xyz</Node> is totally missing from the xml string.

Any ideas on how to get the xml string loaded with only the real xml
data that is displayed in the axwebbrowser control?

thanks

Nov 20 '05 #8
I can't get it from the httprequest. There is no single URL that will
get me to the necessary page.

I have to step through a series of pages. Selecting links, filling in
data fields and pressing the right buttons. I have all of this
working programatically of course.

If I enter the URL that is displayed when I get to the final page, it
redirects to the entry page where I have to fill in the information.
The website is ASP based.

Thanks for any additional information.

On Tue, 21 Oct 2003 20:33:06 +0100, "Fergus Cooney" <fi*****@post.com>
wrote:
Hi MeNo,

If you know when you've reached the xml stage, why not get the file
directly using an HttpRequest? Then you won't need to worry axBrowser messing
it up - you can stick it straight into your XmlDocument.

Regards,
Fergus


Nov 20 '05 #9
Hi MeNo,

Yes, I've followed your progress with this project. Hurdle after hurdle,
but you're getting there. :-)

If simply using the URL won't work. I'd guess it's because the request has
particular data in the headers. If you could determine what that is, you could
recreate the entire request for yourself.

On way to find out would be to substitute the website's address with your
localhost and have an asp page on a dummy website which simply reports what
the headers are.

Leaving that aside, it may be possible to take the mangled axBrowser
output and 're-xmlify' it. It depends whether it loses information or just
adds in a load of garbage.

Regards,
Fergus
Nov 20 '05 #10
hate to say it, but for the betterment of all concerned..."poppy-cock"

there is one page that has data to post to another page. all you have to do
is get the label/value pairs of what is to be posted and post it through an
http request...simple as that. look at the html source of the entry page,
then post the entry page data programmatically to the final url that's going
to return your xml.

btw, this is common methodology for hacking "secure" sites. explore the
hacker w/n.

;^)

steve
"MeNotHome" <dtfa@****nospm****insightbb.com> wrote in message
news:8i********************************@4ax.com...
I can't get it from the httprequest. There is no single URL that will
get me to the necessary page.

I have to step through a series of pages. Selecting links, filling in
data fields and pressing the right buttons. I have all of this
working programatically of course.

If I enter the URL that is displayed when I get to the final page, it
redirects to the entry page where I have to fill in the information.
The website is ASP based.

Thanks for any additional information.

On Tue, 21 Oct 2003 20:33:06 +0100, "Fergus Cooney" <fi*****@post.com>
wrote:
Hi MeNo,

If you know when you've reached the xml stage, why not get the file
directly using an HttpRequest? Then you won't need to worry axBrowser messingit up - you can stick it straight into your XmlDocument.

Regards,
Fergus

Nov 20 '05 #11
Yes, I am getting there but boy is it slow progress...LOL
I have tried to figure out if they were posting something via the URL
command line to get to the right page.

Here is the form code for the button I press

<form action="Rel.asp" method="post" name="frmReleaseList"
onsubmit="return frmReleaseList_validate();" style="margin:0">

But I cannot figure out what it wants on the URL line to produce the
right page.

If I put in the "Rel.asp" it jumps to the selection form and doesn't
display any xml data.

thanks for trying to help
On Tue, 21 Oct 2003 21:22:48 +0100, "Fergus Cooney" <fi*****@post.com>
wrote:
Hi MeNo,

Yes, I've followed your progress with this project. Hurdle after hurdle,
but you're getting there. :-)

If simply using the URL won't work. I'd guess it's because the request has
particular data in the headers. If you could determine what that is, you could
recreate the entire request for yourself.

On way to find out would be to substitute the website's address with your
localhost and have an asp page on a dummy website which simply reports what
the headers are.

Leaving that aside, it may be possible to take the mangled axBrowser
output and 're-xmlify' it. It depends whether it loses information or just
adds in a load of garbage.

Regards,
Fergus


Nov 20 '05 #12
That's an interesting idea. I'm not sure how to proceed though.
There are 45 records on the page that have checkboxes next to them.
Yes, all 45 are checked.

So are you saying that I would need
http://mywebpage.com/Rel.asp?checkbo...=1&checkbox3=1 all
the way out for 45 entries?

Thanks for any info

On Tue, 21 Oct 2003 15:28:00 -0500, "steve" <as*@abc.com> wrote:
hate to say it, but for the betterment of all concerned..."poppy-cock"

there is one page that has data to post to another page. all you have to do
is get the label/value pairs of what is to be posted and post it through an
http request...simple as that. look at the html source of the entry page,
then post the entry page data programmatically to the final url that's going
to return your xml.

btw, this is common methodology for hacking "secure" sites. explore the
hacker w/n.

;^)

steve
"MeNotHome" <dtfa@****nospm****insightbb.com> wrote in message
news:8i********************************@4ax.com.. .
I can't get it from the httprequest. There is no single URL that will
get me to the necessary page.

I have to step through a series of pages. Selecting links, filling in
data fields and pressing the right buttons. I have all of this
working programatically of course.

If I enter the URL that is displayed when I get to the final page, it
redirects to the entry page where I have to fill in the information.
The website is ASP based.

Thanks for any additional information.

On Tue, 21 Oct 2003 20:33:06 +0100, "Fergus Cooney" <fi*****@post.com>
wrote:
>Hi MeNo,
>
> If you know when you've reached the xml stage, why not get the file
>directly using an HttpRequest? Then you won't need to worry axBrowsermessing >it up - you can stick it straight into your XmlDocument.
>
>Regards,
>Fergus
>


Nov 20 '05 #13
my or hasn't shown for some reason...so here's my $0.02 usd.
I can't get it from the httprequest. There is no single URL that will
get me to the necessary page.

I have to step through a series of pages. Selecting links, filling in
data fields and pressing the right buttons. I have all of this
working programatically of course.


look at it more simply...

there is one page that has data to post to another page. all you have to do
is get the label/value pairs of what is to be posted and post it through an
http request...simple as that. look at the html source of the entry page,
then post the entry page data programmatically to the final url that's going
to return your xml.

btw, this is common methodology for hacking "secure" sites. explore the
hacker w/n.

;^)

steve
Nov 20 '05 #14
absolutely...

you are not presented w/ an "asp" or "php" or any other type of dynamically
generated "page". you are presented with, what you can consider, static
information...beit html, xml, etc. all the browser control is doing
conveniently displaying the results of an http request. there's no magic
involved.

try it out...make your own asp pages that act similarly to your target site.

in the interest of time though, did you print out the document.body and
document properties? i think it may be quicker to resolve where the xml
resides in the control.

steve
"MeNotHome" <dtfa@****nospm****insightbb.com> wrote in message
news:io********************************@4ax.com...
Is this true even if the pages are asp based?

On Tue, 21 Oct 2003 16:48:31 -0500, "steve" <as*@abc.com> wrote:
my or hasn't shown for some reason...so here's my $0.02 usd.
I can't get it from the httprequest. There is no single URL that will
get me to the necessary page.

I have to step through a series of pages. Selecting links, filling in
data fields and pressing the right buttons. I have all of this
working programatically of course.


look at it more simply...

there is one page that has data to post to another page. all you have to dois get the label/value pairs of what is to be posted and post it through anhttp request...simple as that. look at the html source of the entry page,
then post the entry page data programmatically to the final url that's goingto return your xml.

btw, this is common methodology for hacking "secure" sites. explore the
hacker w/n.

;^)

steve

Nov 20 '05 #15
Hi MeNo,

Right from the start, this is what I've been trying to say to you - work
out what each page is sending in its request and use HttpRequest to match it.
But I accept that working that out is not easy if the concept is new to you.
And I admire what you've done in triggering events within the AxBrowser. :-)

If you set up an asp or aspx page on your server at home or work, you can
get it to display the headers of the request that invokes it. If you can
fiddle the link html in the AxBrowser to call <your> page instead of the real
website, you can do the same - find out what the headers are. Then you can use
HttpRequest and set <its> headers to those values.

However, like Steve says, pragmatism favours following the path you're on
rather than researching a whole new area.

I'm curious and would be interested to see the html of the last couple of
web pages in your sequence. Perhaps you could zip and post them?

Regards,
Fergus
Nov 20 '05 #16
Can you give any help on figuring out what is being sent to each page?

Give me more info on using httpRequest. I have not used this before.

Thanks

On Wed, 22 Oct 2003 00:46:18 +0100, "Fergus Cooney" <fi*****@post.com>
wrote:
Hi MeNo,

Right from the start, this is what I've been trying to say to you - work
out what each page is sending in its request and use HttpRequest to match it.
But I accept that working that out is not easy if the concept is new to you.
And I admire what you've done in triggering events within the AxBrowser. :-)

If you set up an asp or aspx page on your server at home or work, you can
get it to display the headers of the request that invokes it. If you can
fiddle the link html in the AxBrowser to call <your> page instead of the real
website, you can do the same - find out what the headers are. Then you can use
HttpRequest and set <its> headers to those values.

However, like Steve says, pragmatism favours following the path you're on
rather than researching a whole new area.

I'm curious and would be interested to see the html of the last couple of
web pages in your sequence. Perhaps you could zip and post them?

Regards,
Fergus


Nov 20 '05 #17
here's an example i've given b4...it sends a soap message, but posting data
is done in a similar fashion. pay attention to headers.add as this is where
session information, etc. is communicated. btw, are you going to attempt to
find where the xml is hidden w/n the control?

hth,

steve

' ============================================

Private Const shopCode As String = "XYZ"
Private Const appId As String = "SOME_GENERATED_PUBLIC_KEY"
Private Const syncServer As String = "http://somecompany.com/websvc.php"
' wow...i thought people just used asp in the real world!

' this is as basic as it gets
' the method is getData(shopcode, appid)
' this is how you'd wrap up the request in soap

Function sendWebRequest() As String
Dim webRequest As HttpWebRequest =
CType(webRequest.Create(syncServer), HttpWebRequest)
Dim webResponse As HttpWebResponse
Dim soapEnvelope As String
soapEnvelope &= "<SOAP:Envelope>" & vbCrLf
soapEnvelope &= " <SOAP:Body>" & vbCrLf
soapEnvelope &= " <getData>" & vbCrLf
soapEnvelope &= " <parameters>" & vbCrLf
soapEnvelope &= " <shopcode xsi:type=""xsd:string"">" &
shopCode & "</shopcode>" & vbCrLf
soapEnvelope &= " <appid xsi:type=""xsd:string"">" & appId
& "</appid>" & vbCrLf
soapEnvelope &= " </parameters>" & vbCrLf
soapEnvelope &= " </getData>" & vbCrLf
soapEnvelope &= " </SOAP:Body>" & vbCrLf
soapEnvelope &= "</SOAP:Envelope>" & vbCrLf
With webRequest
.ContentType = "text/xml"
.Headers.Add("SOAPMethodName", "getData")
.ContentLength = soapEnvelope.Length
.Method = "POST"
.Timeout = 60 * 1000 ' milliseconds to seconds
Dim streamWriter As New StreamWriter(.GetRequestStream())
streamWriter.Write(soapEnvelope)
streamWriter.Close()
webResponse = CType(.GetResponse(), HttpWebResponse)
End With
Dim stream As Stream = webResponse.GetResponseStream
Dim streamReader As New StreamReader(stream)
Dim xmlStream As String = streamReader.ReadToEnd
streamReader.Close()
stream.Close()
Return xmlStream
End Function
Nov 20 '05 #18
Thanks for the example code.

Now is there anyway to know what I fields and values I need to pass to
the httprequest?

Thanks

On Wed, 22 Oct 2003 11:04:03 -0500, "steve" <as*@abc.com> wrote:
here's an example i've given b4...it sends a soap message, but posting data
is done in a similar fashion. pay attention to headers.add as this is where
session information, etc. is communicated. btw, are you going to attempt to
find where the xml is hidden w/n the control?

hth,

steve

' ============================================

Private Const shopCode As String = "XYZ"
Private Const appId As String = "SOME_GENERATED_PUBLIC_KEY"
Private Const syncServer As String = "http://somecompany.com/websvc.php"
' wow...i thought people just used asp in the real world!

' this is as basic as it gets
' the method is getData(shopcode, appid)
' this is how you'd wrap up the request in soap

Function sendWebRequest() As String
Dim webRequest As HttpWebRequest =
CType(webRequest.Create(syncServer), HttpWebRequest)
Dim webResponse As HttpWebResponse
Dim soapEnvelope As String
soapEnvelope &= "<SOAP:Envelope>" & vbCrLf
soapEnvelope &= " <SOAP:Body>" & vbCrLf
soapEnvelope &= " <getData>" & vbCrLf
soapEnvelope &= " <parameters>" & vbCrLf
soapEnvelope &= " <shopcode xsi:type=""xsd:string"">" &
shopCode & "</shopcode>" & vbCrLf
soapEnvelope &= " <appid xsi:type=""xsd:string"">" & appId
& "</appid>" & vbCrLf
soapEnvelope &= " </parameters>" & vbCrLf
soapEnvelope &= " </getData>" & vbCrLf
soapEnvelope &= " </SOAP:Body>" & vbCrLf
soapEnvelope &= "</SOAP:Envelope>" & vbCrLf
With webRequest
.ContentType = "text/xml"
.Headers.Add("SOAPMethodName", "getData")
.ContentLength = soapEnvelope.Length
.Method = "POST"
.Timeout = 60 * 1000 ' milliseconds to seconds
Dim streamWriter As New StreamWriter(.GetRequestStream())
streamWriter.Write(soapEnvelope)
streamWriter.Close()
webResponse = CType(.GetResponse(), HttpWebResponse)
End With
Dim stream As Stream = webResponse.GetResponseStream
Dim streamReader As New StreamReader(stream)
Dim xmlStream As String = streamReader.ReadToEnd
streamReader.Close()
stream.Close()
Return xmlStream
End Function


Nov 20 '05 #19
first, let me re-post what fergus...err...iron/gold guru...said.

"If you set up an asp or aspx page on your server at home or work, you can
get it to display the headers of the request that invokes it. If you can
fiddle the link html in the AxBrowser to call <your> page instead of the
real
website, you can do the same - find out what the headers are. Then you can
use
HttpRequest and set <its> headers to those values."

also, you can use your standard browser, go the the page in question - the
one that submits the information to the xml producing web page. once on that
page, view it's source. you will most likely see <form></form> tags...if
there are multiple forms, then look for the appropriate one that submits to
the target xml page. get the names of all (including type="hidden") it's
elements and their values.

from there you will submit the headers discovered in the manner described by
fergus and the form data discovered as above...i don't know what kind of
example i can get/do but i'll try one.

i assume at this point you neither want to try to continue using the
axbrowser control nor discover where it is placing the xml results?
"MeNotHome" <dtfa@****nospm****insightbb.com> wrote in message
news:m2********************************@4ax.com...
Thanks for the example code.

Now is there anyway to know what I fields and values I need to pass to
the httprequest?

Thanks

On Wed, 22 Oct 2003 11:04:03 -0500, "steve" <as*@abc.com> wrote:
here's an example i've given b4...it sends a soap message, but posting datais done in a similar fashion. pay attention to headers.add as this is wheresession information, etc. is communicated. btw, are you going to attempt tofind where the xml is hidden w/n the control?

hth,

steve

' ============================================

Private Const shopCode As String = "XYZ"
Private Const appId As String = "SOME_GENERATED_PUBLIC_KEY"
Private Const syncServer As String = "http://somecompany.com/websvc.php"' wow...i thought people just used asp in the real world!

' this is as basic as it gets
' the method is getData(shopcode, appid)
' this is how you'd wrap up the request in soap

Function sendWebRequest() As String
Dim webRequest As HttpWebRequest =
CType(webRequest.Create(syncServer), HttpWebRequest)
Dim webResponse As HttpWebResponse
Dim soapEnvelope As String
soapEnvelope &= "<SOAP:Envelope>" & vbCrLf
soapEnvelope &= " <SOAP:Body>" & vbCrLf
soapEnvelope &= " <getData>" & vbCrLf
soapEnvelope &= " <parameters>" & vbCrLf
soapEnvelope &= " <shopcode xsi:type=""xsd:string"">" &
shopCode & "</shopcode>" & vbCrLf
soapEnvelope &= " <appid xsi:type=""xsd:string"">" & appId& "</appid>" & vbCrLf
soapEnvelope &= " </parameters>" & vbCrLf
soapEnvelope &= " </getData>" & vbCrLf
soapEnvelope &= " </SOAP:Body>" & vbCrLf
soapEnvelope &= "</SOAP:Envelope>" & vbCrLf
With webRequest
.ContentType = "text/xml"
.Headers.Add("SOAPMethodName", "getData")
.ContentLength = soapEnvelope.Length
.Method = "POST"
.Timeout = 60 * 1000 ' milliseconds to seconds
Dim streamWriter As New StreamWriter(.GetRequestStream())
streamWriter.Write(soapEnvelope)
streamWriter.Close()
webResponse = CType(.GetResponse(), HttpWebResponse)
End With
Dim stream As Stream = webResponse.GetResponseStream
Dim streamReader As New StreamReader(stream)
Dim xmlStream As String = streamReader.ReadToEnd
streamReader.Close()
stream.Close()
Return xmlStream
End Function

Nov 20 '05 #20
I would love to be able to continue to use the axwebbrowser. I have
everything working up to this point.

If I could just get the xml data from the axwebbrowser, then the
battle would be over.

But no matter what I try, I can't get the xml data from the browser
control.

I will try what you suggested regarding looking for the input and
hidden fields.

Thanks for any additional help

i assume at this point you neither want to try to continue using the
axbrowser control nor discover where it is placing the xml results?
"MeNotHome" <dtfa@****nospm****insightbb.com> wrote in message
news:m2********************************@4ax.com.. .
Thanks for the example code.

Now is there anyway to know what I fields and values I need to pass to
the httprequest?

Thanks

On Wed, 22 Oct 2003 11:04:03 -0500, "steve" <as*@abc.com> wrote:
>here's an example i've given b4...it sends a soap message, but postingdata >is done in a similar fashion. pay attention to headers.add as this iswhere >session information, etc. is communicated. btw, are you going to attemptto >find where the xml is hidden w/n the control?
>
>hth,
>
>steve
>
>' ============================================
>
> Private Const shopCode As String = "XYZ"
> Private Const appId As String = "SOME_GENERATED_PUBLIC_KEY"
> Private Const syncServer As String ="http://somecompany.com/websvc.php" >' wow...i thought people just used asp in the real world!
>
>' this is as basic as it gets
>' the method is getData(shopcode, appid)
>' this is how you'd wrap up the request in soap
>
> Function sendWebRequest() As String
> Dim webRequest As HttpWebRequest =
>CType(webRequest.Create(syncServer), HttpWebRequest)
> Dim webResponse As HttpWebResponse
> Dim soapEnvelope As String
> soapEnvelope &= "<SOAP:Envelope>" & vbCrLf
> soapEnvelope &= " <SOAP:Body>" & vbCrLf
> soapEnvelope &= " <getData>" & vbCrLf
> soapEnvelope &= " <parameters>" & vbCrLf
> soapEnvelope &= " <shopcode xsi:type=""xsd:string"">" &
>shopCode & "</shopcode>" & vbCrLf
> soapEnvelope &= " <appid xsi:type=""xsd:string"">" &appId >& "</appid>" & vbCrLf
> soapEnvelope &= " </parameters>" & vbCrLf
> soapEnvelope &= " </getData>" & vbCrLf
> soapEnvelope &= " </SOAP:Body>" & vbCrLf
> soapEnvelope &= "</SOAP:Envelope>" & vbCrLf
> With webRequest
> .ContentType = "text/xml"
> .Headers.Add("SOAPMethodName", "getData")
> .ContentLength = soapEnvelope.Length
> .Method = "POST"
> .Timeout = 60 * 1000 ' milliseconds to seconds
> Dim streamWriter As New StreamWriter(.GetRequestStream())
> streamWriter.Write(soapEnvelope)
> streamWriter.Close()
> webResponse = CType(.GetResponse(), HttpWebResponse)
> End With
> Dim stream As Stream = webResponse.GetResponseStream
> Dim streamReader As New StreamReader(stream)
> Dim xmlStream As String = streamReader.ReadToEnd
> streamReader.Close()
> stream.Close()
> Return xmlStream
> End Function
>


Nov 20 '05 #21
Hi MeNo,

There's still that option of posting a zip of the web pages that you are
playing with (including the one with the xml). I can't promise but it might be
useful.

Regards,
Fergus
Nov 20 '05 #22
Hi Steve,

ROFL. ;-))

Regards,
Fergus
Nov 20 '05 #23
;^)

btw...i've posted data to web sites before through apis in vb6. the example
i was trying to "cook up" for the op isn't quite working. any ideas on the
following (btw, it has mutated several times in an attempt to get a working
one...the code either reflects that or my ignorance...you pick). would
including "Expect" in the header and waiting for "100 (Continue)" before i
post the data make a difference here...haven't had to before, but it is part
of the rfc.

' the accounts.php page posts u/pass to itself...if successful, redirects to
siteauth.php

Imports System.Net
Imports System.Net.HttpWebRequest
Imports System.Net.HttpWebResponse

Private Const authorizationPage As String =
"http://www.renderbar.com/secure/siteauth.php"
Private Const targetPage As String =
"http://www.renderbar.com/secure/accounts.php" ' should redirect to the
above page.

Function sendWebRequest() As String
Dim webRequest As HttpWebRequest = webRequest.Create(authorizationPage)
Dim webResponse As HttpWebResponse
Dim pageData As String = "username=support&password=diver"
With webRequest
.ContentType = "application/x-url-formencoded" ' tried many
combinations...thought text/html w/b the best bet.
.ContentLength = pageData.Length
.KeepAlive = True
.Method = "POST"
.Timeout = 60 * 1000 ' milliseconds to seconds
Dim streamWriter As New StreamWriter(.GetRequestStream())
streamWriter.Write(pageData)
streamWriter.Close()
webResponse = DirectCast(.GetResponse(), HttpWebResponse)
End With
Dim stream As Stream = webResponse.GetResponseStream
Dim streamReader As New StreamReader(stream)
Dim xmlStream As String = streamReader.ReadToEnd
streamReader.Close()
stream.Close()
Return xmlStream
End Function
"Fergus Cooney" <fi*****@post.com> wrote in message
news:eL**************@TK2MSFTNGP12.phx.gbl...
Hi Steve,

ROFL. ;-))

Regards,
Fergus

Nov 20 '05 #24
username s/b su*****@renderbar.com if you wanna be able to log in correctly.
"steve" <as*@abc.com> wrote in message
news:vp************@corp.supernews.com...
;^)

btw...i've posted data to web sites before through apis in vb6. the example i was trying to "cook up" for the op isn't quite working. any ideas on the
following (btw, it has mutated several times in an attempt to get a working one...the code either reflects that or my ignorance...you pick). would
including "Expect" in the header and waiting for "100 (Continue)" before i post the data make a difference here...haven't had to before, but it is part of the rfc.

' the accounts.php page posts u/pass to itself...if successful, redirects to siteauth.php

Imports System.Net
Imports System.Net.HttpWebRequest
Imports System.Net.HttpWebResponse

Private Const authorizationPage As String =
"http://www.renderbar.com/secure/siteauth.php"
Private Const targetPage As String =
"http://www.renderbar.com/secure/accounts.php" ' should redirect to the
above page.

Function sendWebRequest() As String
Dim webRequest As HttpWebRequest = webRequest.Create(authorizationPage)
Dim webResponse As HttpWebResponse
Dim pageData As String = "username=support&password=diver"
With webRequest
.ContentType = "application/x-url-formencoded" ' tried many
combinations...thought text/html w/b the best bet.
.ContentLength = pageData.Length
.KeepAlive = True
.Method = "POST"
.Timeout = 60 * 1000 ' milliseconds to seconds
Dim streamWriter As New StreamWriter(.GetRequestStream())
streamWriter.Write(pageData)
streamWriter.Close()
webResponse = DirectCast(.GetResponse(), HttpWebResponse)
End With
Dim stream As Stream = webResponse.GetResponseStream
Dim streamReader As New StreamReader(stream)
Dim xmlStream As String = streamReader.ReadToEnd
streamReader.Close()
stream.Close()
Return xmlStream
End Function
"Fergus Cooney" <fi*****@post.com> wrote in message
news:eL**************@TK2MSFTNGP12.phx.gbl...
Hi Steve,

ROFL. ;-))

Regards,
Fergus


Nov 20 '05 #25
ok mnh! this works on my website...modify to your liking.

Imports System.IO
Imports System.Net
Imports System.Net.HttpWebRequest
Imports System.Net.HttpWebResponse
Imports System.Web
Imports System.Web.HttpUtility

Module webTest

Private Const authorizationPage As String =
"http://www.renderbar.com/secure/siteauth.php"
Private Const targetPage As String =
"http://www.renderbar.com/secure/accounts.php"

Public Sub Main()
' post web data test
Dim params(1) As String
params(0) = "us**************@renderbar.com"
params(1) = "password=diver"
getWebResponse(authorizationPage, params)
End Sub

#Region " http request "

Function getWebResponse(ByVal url As String, ByVal params As String()) As
String
If url Is Nothing OrElse params Is Nothing Then Return Nothing
Dim webRequest As HttpWebRequest = webRequest.Create(url)
Dim webResponse As HttpWebResponse
Dim param As String
Dim postData As String
For Each param In params
postData &= HttpUtility.HtmlEncode(param) & "&"
Next
postData = postData.Substring(0, postData.Length - 1)
With webRequest
.AllowAutoRedirect = True
.ContentType = "application/x-www-form-urlencoded"
.ContentLength = postData.Length
.KeepAlive = False
.Method = "POST"
.Timeout = 60 * 1000 ' milliseconds to seconds
Dim streamWriter As New StreamWriter(.GetRequestStream())
streamWriter.Write(postData)
streamWriter.Close()
webResponse = .GetResponse()
End With
Dim stream As Stream = webResponse.GetResponseStream
Dim streamReader As New StreamReader(stream)
Dim xmlStream As String = streamReader.ReadToEnd
streamReader.Close()
stream.Close()
Return xmlStream
End Function

#End Region

End Module

Nov 20 '05 #26

This discussion thread is closed

Replies have been disabled for this discussion.

Similar topics

2 posts views Thread by Robert | last post: by
4 posts views Thread by Cathie | last post: by
2 posts views Thread by Dave H | last post: by
10 posts views Thread by =?Utf-8?B?YzY3NjIyOA==?= | last post: by
reply views Thread by Waqarahmed | last post: by
By using this site, you agree to our Privacy Policy and Terms of Use.