WebBrowser and Returing the raw HTML

[Craig Francis]

Ok I'm making a fairly simple application. It contains 2
web browsers, the top one is used so that you can view a
website (i.e. one you have created). Every time you load
a page, the HTML which was received is then sent to the
http://validator.w3.org website to validate your HTML /
XHTML.

So far I've got everything to work, even the part where
the HTML is posted to the w3.org website.

But all of the following commands (Browser1 is the main
WebBrowser control) produce a form of HTML for the
document, but all the tags get converted to uppercase and
parts of the document go missing such as the "DOCTYPE".. .

Browser1.Docume nt.ToString()
Browser1.Docume nt.documentelem ent.outerhtml
Browser1.Docume nt.documentelem ent.innerhtml
Browser1.Docume nt.Body.outerht ml
Browser1.Docume nt.Body.innerht ml
Browser1.Docume nt.All(0).outer html
Browser1.Docume nt.All(0).inner html
Browser1.Docume nt.All(1).outer html
Browser1.Docume nt.All(1).inner html
Browser1.Docume nt.All(2).outer html
Browser1.Docume nt.All(2).inner html
NOTE: The HTML sent to the w3.org website must be exactly
the same as what the server sends otherwise what's the
point in validating it?

Finally, because it will be used on interactive websites
(with a user login), you cant use controls such as the
Inet to return the HTML as then the user (main browser)
will make a request to the server (which may delete a
record) then the Inet or Winsock (etc) will make a
request, but this will then return a different page
(saying you cant delete a record).

[Charles Law]

Hi Craig

Because the DOCTYPE tag is outside the main document, it is not included
when you retrieve inner and outer HTML. To include the entire file you will
need to use the IPersistStreamI nit interface, e.g.

<interface>
Imports System.Runtime. InteropServices

' IPersistStreamI nit interface
<ComVisible(Tru e), ComImport(),
Guid("7FD52380-4E07-101B-AE2D-08002B2EC713"), _
InterfaceTypeAt tribute(ComInte rfaceType.Inter faceIsIUnknown) > _
Public Interface IPersistStreamI nit
Sub GetClassID(ByRe f pClassID As Guid)

<PreserveSig( )> Function IsDirty() As Integer
<PreserveSig( )> Function Load(ByVal pstm As UCOMIStream) As Integer
<PreserveSig( )> Function Save(ByVal pstm As UCOMIStream, ByVal fClearDirty
As Boolean) As Integer
<PreserveSig( )> Function GetSizeMax(<InA ttribute(), Out(),
MarshalAs(Unman agedType.U8)> ByRef pcbSize As Long) As Integer
<PreserveSig( )> Function InitNew() As Integer
End Interface
</interface>

<code>
Dim ips as IPersistStreamI nit

ips = DirectCast(Brow ser1.document, IPersistStreamI nit)

ips.Save(strm, False)
</code>

This will save the complete HTML to a stream, which you can turn into a
string.

Regarding the conversion to uppercase, is this actually a problem? The
change of case should not affect the validity of the parsing.

There also two particular newsgroups which may give further help:

microsoft.publi c.inetsdk.progr amming.mshtml_h osting
microsoft.publi c.inetsdk.progr amming.webbrows er_ctl

HTH

Charles

"Craig Francis" <[e-mail]> wrote in message
news:08******** *************** *****@phx.gbl.. .

Ok I'm making a fairly simple application. It contains 2
web browsers, the top one is used so that you can view a
website (i.e. one you have created). Every time you load
a page, the HTML which was received is then sent to the
http://validator.w3.org website to validate your HTML /
XHTML.

So far I've got everything to work, even the part where
the HTML is posted to the w3.org website.

But all of the following commands (Browser1 is the main
WebBrowser control) produce a form of HTML for the
document, but all the tags get converted to uppercase and
parts of the document go missing such as the "DOCTYPE".. .

Browser1.Docume nt.ToString()
Browser1.Docume nt.documentelem ent.outerhtml
Browser1.Docume nt.documentelem ent.innerhtml
Browser1.Docume nt.Body.outerht ml
Browser1.Docume nt.Body.innerht ml
Browser1.Docume nt.All(0).outer html
Browser1.Docume nt.All(0).inner html
Browser1.Docume nt.All(1).outer html
Browser1.Docume nt.All(1).inner html
Browser1.Docume nt.All(2).outer html
Browser1.Docume nt.All(2).inner html
NOTE: The HTML sent to the w3.org website must be exactly
the same as what the server sends otherwise what's the
point in validating it?

Finally, because it will be used on interactive websites
(with a user login), you cant use controls such as the
Inet to return the HTML as then the user (main browser)
will make a request to the server (which may delete a
record) then the Inet or Winsock (etc) will make a
request, but this will then return a different page
(saying you cant delete a record).

[Craig Francis]

Thank you for your quick reply.

But is that VB code? I've been using VB5/6 for several
years and that looks slightly C like - this project is
being written in VB.NET, but I've only just upgraded and
finding some of these new methods a little strange.

Also RE the tags being changed to uppercase - The reason
I mentioned it was because it shows that the HTML
document is being changed, probably into a form that the
browser can easily understand (and is probably strict XML
even if the input wasn't XML based).

Anyway, thanks for giving me something else to try.

Craig

[Craig Francis]

Got it, you put all the

<interface></interface>

before the "Public Class Form1" bit - so the first part
of the form, then the

<code></code>

in the function which returns the HTML code. Well that
method doesn't bring up any errors apart from what "strm"
should be dimed as - I've never used a stream before.

But thanks again - this is the most progress I've made in
the past 2 days!

[Charles Law]

Hi Craig

Yes, sorry about that. It's just a habit I have got into to show where code
and stuff begins and ends. Add the following for the stream handling:

<code>
<DllImport("OLE 32.DLL")> _
Public Shared Sub CreateStreamOnH Global(ByVal hGlobal As IntPtr, ByVal
fDelete As Boolean, ByRef stm As UCOMIStream)
' LEAVE THIS BLANK - PLACEHOLDER
End Sub

<DllImport("OLE 32.DLL")> _
Public Shared Sub GetHGlobalFromS tream(ByVal stm As UCOMIStream, ByRef
hGlobal As IntPtr)
' LEAVE THIS BLANK - PLACEHOLDER
End Sub

Private Function GetStream(ByVal size As Integer) As UCOMIStream

Dim iptr As IntPtr
Dim strm As UCOMIStream

iptr = Marshal.AllocHG lobal(size)
CreateStreamOnH Global(iptr, True, strm)

Return strm

End Function

Private Function StreamToString( ByVal strm As UCOMIStream) As String

Dim iptr As IntPtr
Dim s As String

GetHGlobalFromS tream(strm, iptr)
s = Marshal.PtrToSt ringAnsi(iptr)

Return s

End Function
</code>

<code>
Dim strm As UCOMIStream
Dim s As String

' Allocate a reasonably high value!
strm = GetStream(2048)

' Save HTML and convert to a string
ips.Save(strm, False)
s = StreamToString( strm)
</code>

The code above should allow you to be able get the full HTML. The only issue
with this is the allocation of the stream. IPersistStreamI nit.GetSizeMax( )
should return a value indicating the size of the stream required, but it
always returns zero. The best way, therefore is to read the stream a bit at
a time until the buffer is empty, but for simplicity I have just allocated a
stream that should be big enough to take it all in one go. You can make it
bigger of course if you need to.

HTH

Charles
"Craig Francis" <[e-mail]> wrote in message
news:09******** *************** *****@phx.gbl.. .

Got it, you put all the

<interface></interface>

before the "Public Class Form1" bit - so the first part
of the form, then the

<code></code>

in the function which returns the HTML code. Well that
method doesn't bring up any errors apart from what "strm"
should be dimed as - I've never used a stream before.

But thanks again - this is the most progress I've made in
the past 2 days!

[Craig Francis]

You are a f________ god!

Thankyou, it works perfectly!!!

[Herfried K. Wagner [MVP]]

Hello,

"Craig Francis" <[e-mail]> schrieb:

Ok I'm making a fairly simple application. It contains 2
web browsers, the top one is used so that you can view a
website (i.e. one you have created). Every time you load
a page, the HTML which was received is then sent to the
http://validator.w3.org website to validate your HTML /
XHTML.

So far I've got everything to work, even the part where
the HTML is posted to the w3.org website.

But all of the following commands (Browser1 is the main
WebBrowser control) produce a form of HTML for the
document, but all the tags get converted to uppercase and
parts of the document go missing such as the "DOCTYPE".. .

I don't really understand why you use the WebBrowser control to download the
web page. Why not use, for example, the 'WebRequest' class?

--
Herfried K. Wagner
MVP · VB Classic, VB.NET
http://www.mvps.org/dotnet

[Craig Francis]

>I don't really understand why you use the WebBrowser
control to download the

web page. Why not use, for example, the 'WebRequest'

class?

Because im fairly new to VB.NET and wanted a simple
application to create - well what I thought might be
simple.

Also I've used the WebBrowser control before and it was a
simple way to add a browser to the application where the
user could navigate in exactly the same way as in IE.

[Cor]

Charles,
Just a question, I have seen you uses always the mshtml.IHtmldoc ument2
I use the mshtml.Htmldocu ment.
I have the idea, that with that I can access all <tags> including the src,
innertext and innerhtml etc per framepage.

What do I mis?
Cor

[Charles Law]

Hi Cor

Long time no speak.

The simple answer is speed. Try the following on an initialised WebBrowser
control and you may be surprised:

<code>
Dim doc As mshtml.HTMLDocu ment
Dim doc2 As mshtml.IHTMLDoc ument2
Dim elem As mshtml.IHTMLEle ment

Dim dt As Date

MsgBox("Start")

dt = Now

For i As Integer = 1 To 1000
doc = DirectCast(AxWe bBrowser1.Docum ent, mshtml.HTMLDocu ment)
elem = doc.createEleme nt("INPUT")
Next i

MsgBox(Now.Subt ract(dt).ToStri ng)

dt = Now

For i As Integer = 1 To 1000
doc2 = DirectCast(AxWe bBrowser1.Docum ent, mshtml.IHTMLDoc ument2)
elem = doc2.createElem ent("INPUT")
Next i

MsgBox(Now.Subt ract(dt).ToStri ng)
</code>

I used mshtml.HTMLDocu ment once in the earlier post because New doesn't work
on interfaces of course. But otherwise I use the interfaces. It means a bit
more code to cast to the correct one all the time [long live Option Strict
On], but it's worth it in performance.

Regards

Charles
"Cor" <no*@non.com> wrote in message
news:3f******** *************** @reader22.wxs.n l...

Charles,
Just a question, I have seen you uses always the mshtml.IHtmldoc ument2
I use the mshtml.Htmldocu ment.
I have the idea, that with that I can access all <tags> including the src,
innertext and innerhtml etc per framepage.

What do I mis?
Cor