post large xml files

[andy]

I have a routine I call to post XML from vb.net to a url over http

It can be the XML file can be quite a size sometime I expect it to
exceed 10mb.
I started using a routine to post the file to a URL from vb.net, but
didnt realise there would be a restiction on post size.
So from this function :

Dim result As String = ""
Dim mywriter As IO.StreamWriter
Dim strXML = xmlDoc.InnerXml()
Dim bojRequest As System.Net.HttpWebRequest =
System.Net.HttpWebRequest.Create(postUrl)

bojRequest.Method = "POST"
bojRequest.ContentLength = strXML.Length
bojRequest.ContentType = "text/xml"

Dim myCred As New System.Net.NetworkCredential(postUsername,
postPassword)
bojRequest.Credentials = myCred

Try
mywriter = New
IO.StreamWriter(bojRequest.GetRequestStream())
mywriter.Write(strXML)
mywriter.Flush()
mywriter.Close()
mywriter = Nothing

Dim objResponse As System.Net.HttpWebResponse =
bojRequest.GetResponse()
Dim sr As IO.StreamReader
sr = New IO.StreamReader(objResponse.GetResponseStream())
result = sr.ReadToEnd
sr.Close()
sr = Nothing
Catch ex As Exception
Return ex.Message
End Try

I got this return:
Bytes to be written to the stream exceed the Content-Length bytes size
specified, but the function works when sizes are good.
Im not sure how to fix this so I googled areound and came up with this:

Try
Dim req As WebRequest
Dim RequestStream As Stream
Dim ReceiveStream As Stream
Dim encode As Encoding
Dim sr As StreamReader
Dim payload As String
req = WebRequest.Create(postUrl)
req.Method = "POST"
req.ContentType = "text/xml"
Dim SomeBytes() As Byte
Dim UrlEncoded As New StringBuilder
Dim reserved() As Char = {ChrW(63), ChrW(61), ChrW(38)}
Dim result As WebResponse

payload = xmlDoc.InnerXml
If payload <Nothing Then
Dim i As Integer = 0
Dim j As Integer
While i < payload.Length
j = payload.IndexOfAny(reserved, i)
If j = -1 Then

UrlEncoded.Append(HttpUtility.UrlEncode(payload.Su bstring(i,
payload.Length - i)))

'UrlEncoded.Append(URLEncode(payload.Substring(i, payload.Length - i)))
Exit While
End If

UrlEncoded.Append(HttpUtility.UrlEncode(payload.Su bstring(i, j - i)))
UrlEncoded.Append(payload.Substring(j, 1))
i = j + 1
End While
SomeBytes =
System.Text.Encoding.UTF8.GetBytes(UrlEncoded.ToSt ring())
req.ContentLength = SomeBytes.Length
RequestStream = req.GetRequestStream()
RequestStream.Write(SomeBytes, 0, SomeBytes.Length)
RequestStream.Close()
Else
req.ContentLength = 0
End If
result = req.GetResponse()
ReceiveStream = result.GetResponseStream()
encode = System.Text.Encoding.GetEncoding("utf-8")
sr = New StreamReader(ReceiveStream, encode)

Console.WriteLine()
Console.WriteLine("Response stream received")
Dim read(256) As Char
Dim count As Integer = sr.Read(read, 0, 256)

Console.WriteLine("HTML...")
Console.WriteLine()
Do While count 0
Dim str As String = New String(read, 0, count)
Console.Write(str)
count = sr.Read(read, 0, 256)
Loop
Console.WriteLine("")
Catch Exc As Exception
Console.WriteLine()
Console.WriteLine("The request URI could not be found or
was malformed")
Finally
If Not result Is Nothing Then
result.Close()
End If
End Try

The function runs through without error, but doesnt post
anything..im not sure why its not posting, the url and credentials are
fine.
So I end up with 2 questions, one how to run a procedure to
post large xml files over http, and if anyone knows how to use
compression to reduce the file size sent?

[Martin Honnen]

andy wrote:

Dim strXML = xmlDoc.InnerXml()

What is xmlDoc? A System.Xml.XmlDocument instance? If so if you want to
write that to a stream then you can simply do e.g.

Dim bojRequest As System.Net.HttpWebRequest =
System.Net.HttpWebRequest.Create(postUrl)

bojRequest.Method = "POST"

bojRequest.ContentType = "text/xml"

Dim myCred As New System.Net.NetworkCredential(postUsername,
postPassword)
bojRequest.Credentials = myCred

Stream requestStream = bojRequest.GetRequestStream()
xmlDoc.Save(requestStream)

--

Martin Honnen --- MVP XML
http://JavaScript.FAQTs.com/

[andy]

Thanks for replying Martin.
I could see what you were saying but couldnt get it to work, anyway
here is the solution i found earlier for anyone else trawling google.

Dim inputStream As New FileStream(filePath, FileMode.Open,
FileAccess.Read, FileShare.Read)
Dim inputLength As Integer = CInt(inputStream.Length)
Dim inputBody(inputLength) As Byte
inputStream.Read(inputBody, 0, inputLength)
inputStream.Close()
' Create WebRequest and set headers
Dim req As HttpWebRequest =
CType(WebRequest.Create(postUrl), HttpWebRequest)
req.Credentials = CredentialCache.DefaultCredentials
req.Method = "POST"
req.ProtocolVersion = HttpVersion.Version11
req.ContentType = "text/xml"
req.ContentLength = inputLength
If 0 < mTimeout Then
req.Timeout = mTimeout
End If
' Write the input to the request stream
Dim reqStream As Stream = req.GetRequestStream()
reqStream.Write(inputBody, 0, inputLength)
reqStream.Close()
It works althought im not at all sure about this functions limitations
when faced with large file sizes and would still be interested in
comments on passing xml over http with compression.
Seeing as compression reduces significantly the data been sent over the

wire and can go someway to securing raw data....
Anyone comments ?