post large xml files

[andy]

I have a routine I call to post XML from vb.net to a url over http

It can be the XML file can be quite a size sometime I expect it to
exceed 10mb.
I started using a routine to post the file to a URL from vb.net, but
didnt realise there would be a restiction on post size.
So from this function :

Dim result As String = ""
Dim mywriter As IO.StreamWriter
Dim strXML = xmlDoc.InnerXml ()
Dim bojRequest As System.Net.Http WebRequest =
System.Net.Http WebRequest.Crea te(postUrl)

bojRequest.Meth od = "POST"
bojRequest.Cont entLength = strXML.Length
bojRequest.Cont entType = "text/xml"

Dim myCred As New System.Net.Netw orkCredential(p ostUsername,
postPassword)
bojRequest.Cred entials = myCred

Try
mywriter = New
IO.StreamWriter (bojRequest.Get RequestStream() )
mywriter.Write( strXML)
mywriter.Flush( )
mywriter.Close( )
mywriter = Nothing

Dim objResponse As System.Net.Http WebResponse =
bojRequest.GetR esponse()
Dim sr As IO.StreamReader
sr = New IO.StreamReader (objResponse.Ge tResponseStream ())
result = sr.ReadToEnd
sr.Close()
sr = Nothing
Catch ex As Exception
Return ex.Message
End Try

I got this return:
Bytes to be written to the stream exceed the Content-Length bytes size
specified, but the function works when sizes are good.
Im not sure how to fix this so I googled areound and came up with this:

Try
Dim req As WebRequest
Dim RequestStream As Stream
Dim ReceiveStream As Stream
Dim encode As Encoding
Dim sr As StreamReader
Dim payload As String
req = WebRequest.Crea te(postUrl)
req.Method = "POST"
req.ContentType = "text/xml"
Dim SomeBytes() As Byte
Dim UrlEncoded As New StringBuilder
Dim reserved() As Char = {ChrW(63), ChrW(61), ChrW(38)}
Dim result As WebResponse

payload = xmlDoc.InnerXml
If payload <Nothing Then
Dim i As Integer = 0
Dim j As Integer
While i < payload.Length
j = payload.IndexOf Any(reserved, i)
If j = -1 Then

UrlEncoded.Appe nd(HttpUtility. UrlEncode(paylo ad.Substring(i,
payload.Length - i)))

'UrlEncoded.App end(URLEncode(p ayload.Substrin g(i, payload.Length - i)))
Exit While
End If

UrlEncoded.Appe nd(HttpUtility. UrlEncode(paylo ad.Substring(i, j - i)))
UrlEncoded.Appe nd(payload.Subs tring(j, 1))
i = j + 1
End While
SomeBytes =
System.Text.Enc oding.UTF8.GetB ytes(UrlEncoded .ToString())
req.ContentLeng th = SomeBytes.Lengt h
RequestStream = req.GetRequestS tream()
RequestStream.W rite(SomeBytes, 0, SomeBytes.Lengt h)
RequestStream.C lose()
Else
req.ContentLeng th = 0
End If
result = req.GetResponse ()
ReceiveStream = result.GetRespo nseStream()
encode = System.Text.Enc oding.GetEncodi ng("utf-8")
sr = New StreamReader(Re ceiveStream, encode)

Console.WriteLi ne()
Console.WriteLi ne("Response stream received")
Dim read(256) As Char
Dim count As Integer = sr.Read(read, 0, 256)

Console.WriteLi ne("HTML...")
Console.WriteLi ne()
Do While count 0
Dim str As String = New String(read, 0, count)
Console.Write(s tr)
count = sr.Read(read, 0, 256)
Loop
Console.WriteLi ne("")
Catch Exc As Exception
Console.WriteLi ne()
Console.WriteLi ne("The request URI could not be found or
was malformed")
Finally
If Not result Is Nothing Then
result.Close()
End If
End Try

The function runs through without error, but doesnt post
anything..im not sure why its not posting, the url and credentials are
fine.
So I end up with 2 questions, one how to run a procedure to
post large xml files over http, and if anyone knows how to use
compression to reduce the file size sent?

[Martin Honnen]

andy wrote:

Dim strXML = xmlDoc.InnerXml ()

What is xmlDoc? A System.Xml.XmlD ocument instance? If so if you want to
write that to a stream then you can simply do e.g.

Dim bojRequest As System.Net.Http WebRequest =
System.Net.Http WebRequest.Crea te(postUrl)

bojRequest.Meth od = "POST"

bojRequest.Cont entType = "text/xml"

Dim myCred As New System.Net.Netw orkCredential(p ostUsername,
postPassword)
bojRequest.Cred entials = myCred

Stream requestStream = bojRequest.GetR equestStream()
xmlDoc.Save(req uestStream)

--

Martin Honnen --- MVP XML
http://JavaScript.FAQTs.com/

[andy]

Thanks for replying Martin.
I could see what you were saying but couldnt get it to work, anyway
here is the solution i found earlier for anyone else trawling google.

Dim inputStream As New FileStream(file Path, FileMode.Open,
FileAccess.Read , FileShare.Read)
Dim inputLength As Integer = CInt(inputStrea m.Length)
Dim inputBody(input Length) As Byte
inputStream.Rea d(inputBody, 0, inputLength)
inputStream.Clo se()
' Create WebRequest and set headers
Dim req As HttpWebRequest =
CType(WebReques t.Create(postUr l), HttpWebRequest)
req.Credentials = CredentialCache .DefaultCredent ials
req.Method = "POST"
req.ProtocolVer sion = HttpVersion.Ver sion11
req.ContentType = "text/xml"
req.ContentLeng th = inputLength
If 0 < mTimeout Then
req.Timeout = mTimeout
End If
' Write the input to the request stream
Dim reqStream As Stream = req.GetRequestS tream()
reqStream.Write (inputBody, 0, inputLength)
reqStream.Close ()
It works althought im not at all sure about this functions limitations
when faced with large file sizes and would still be interested in
comments on passing xml over http with compression.
Seeing as compression reduces significantly the data been sent over the

wire and can go someway to securing raw data....
Anyone comments ?