473,778 Members | 1,804 Online
Bytes | Software Development & Data Engineering Community
+ Post

Home Posts Topics Members FAQ

WebClient and chunked data

Hi all!

I use a webClient for requesting data from a server of mine.
Should I worry about long ammount of data sent by server in the client side?
Or, another way, should I send some kind of a terminator sign in the data in
order to realise in the client that all data arrived?

--
Ceers,
Crirus

------------------------------
If work were a good thing, the boss would take it all from you

------------------------------
Nov 20 '05 #1
7 3943
Cor
Hi Crirus,
The webclient is in my idea the best placed in a seperated thread, then you
could abort it also, I think that will be your next problem if you have not
done that already.

I do use it with a try, catch end block an then empty string is a good
result.

Public Function download(Uri, Filename) as string
Dim browser As New System.Net.WebC lient
Try
browser.Downloa dFile(Uri, Filename)
Return ""
Catch Ex As Exception
Return Uri & " " & Ex.Message
End Try
End Function

Cor
I use a webClient for requesting data from a server of mine.
Should I worry about long ammount of data sent by server in the client side? Or, another way, should I send some kind of a terminator sign in the data in order to realise in the client that all data arrived?

Nov 20 '05 #2
That's exactly how I use it..

I'm asking if the Download method will return by itself after all data is
received or should I take care to check if all data was received by parsing
and interpreting the data..

Data is sent by a server I designed with TCP sockets following some MS
sample.
However, in the server, I receive data request in multiple steps. First I
receive only the headers, and in a second receive I have the body if the
method of WebClient was UploadData...

I may miss something at the server side anyway, there should be some length
that I can check against received buffer but I cant figure out now.

I cant find that length
The BeginReceive of the receive socket have the following function as
callback:


Private Sub Receive(ByVal ar As IAsyncResult) 'este apelata cand au ajuns
date
Try
Dim content As [String] = [String].Empty
Dim state As StateObject = CType(ar.AsyncS tate, StateObject) '
Retrieve the state object and the handler socket from the async state
object.
Dim handler As Socket = state.workSocke t 'this is client socket

Dim bytesRead As Integer = handler.EndRece ive(ar) '
Read data from client socket.

If bytesRead > 0 Then
'state.sb.Appen d(Encoding.ASCI I.GetString(sta te.buffer, 0,
bytesRead))
' There might be more data, so store the data received so
far.
Dim lastLength As Integer
If state.received Is Nothing Then
lastLength = 0
Else
lastLength = UBound(state.re ceived)
End If
ReDim Preserve state.received( lastLength + bytesRead - 1)
Array.Copy(stat e.buffer, 0, state.received, lastLength,
bytesRead)
content = Chr(state.recei ved(state.recei ved.Length - 1))
Dim bc As BitConverter
'Dim lenBuff As Long = bc.ToInt64(stat e.buffer, 0)
If content = state.endTransm ition Then
' All the data has been read from the client
Dim utilData(state. received.Length - 102 -
state.endTransm ition.Length) As Byte 'pana la 100 sunt headere ultimul e
terminator
Array.Copy(stat e.received, 101, utilData, 0,
state.received. Length - 101 - state.endTransm ition.Length)
'If lenBuff = bytesRead Then
'Request received; processing ...
Dim response As Byte()
If Not myGame Is Nothing Then
response = myGame.ProcessC lientRequest(ut ilData)
Else
response = myManager.Proce ssClientRequest (utilData)
End If
'"Accept-Language:en-us" + vbCrLf +
"Accept-Encoding:gzip, deflate" + vbCrLf +
'Dim response As Byte() =
Encoding.ASCII. GetBytes("<html ><head></head><body>Noth ing to see
here!</body></html>")
allBytesRead += bytesRead
RaiseEvent BytesReceived(a llBytesRead)
Send(handler, response)
Else
handler.BeginRe ceive(state.buf fer, 0,
StateObject.Buf ferSize, 0, New AsyncCallback(A ddressOf Receive), state) '
Not all data received. Get more.
End If
Else
'nothing to read: client closed
handler.Shutdow n(SocketShutdow n.Both)
handler.Close()
openConnections -= 1
RaiseEvent NewConnection(o penConnections)
End If
Catch e As SocketException
'client close
openConnections -= 1
RaiseEvent NewConnection(o penConnections)
End Try
End Sub 'ReadCallback



--
Ceers,
Crirus

------------------------------
If work were a good thing, the boss would take it all from you

------------------------------

"Cor" <no*@non.com> wrote in message
news:%2******** *******@TK2MSFT NGP10.phx.gbl.. .
Hi Crirus,
The webclient is in my idea the best placed in a seperated thread, then you could abort it also, I think that will be your next problem if you have not done that already.

I do use it with a try, catch end block an then empty string is a good
result.

Public Function download(Uri, Filename) as string
Dim browser As New System.Net.WebC lient
Try
browser.Downloa dFile(Uri, Filename)
Return ""
Catch Ex As Exception
Return Uri & " " & Ex.Message
End Try
End Function

Cor
I use a webClient for requesting data from a server of mine.
Should I worry about long ammount of data sent by server in the client side?
Or, another way, should I send some kind of a terminator sign in the

data in
order to realise in the client that all data arrived?


Nov 20 '05 #3
Cor
Hi Crirus,
If you want to know what is the filelength in advance, you can use the
httpwebrequest with the httpwebheaderco llection,

But that will of course never be sure.

If it are your own files, why not set something as a filesize or other
hashitem in an endblock of the file and test that when you got it?

Cor
Nov 20 '05 #4
I just tested this ideea and dont work:

I never receive a 0 length from EndReceive when all data was read. They say
this in MSDN:

<QUOTE>If you are using a connection-oriented protocol, the EndReceive
method will read as much data as is available up to the number of bytes you
specified in the size parameter of the BeginReceive method. If the remote
host shuts down the Socket connection with the Shutdown method, and all
available data has been received, the EndReceive method will complete
immediately and return zero bytes.To obtain the received data, call the
AsyncState method of the IAsyncResult, and extract the buffer contained in
the resulting state object.</QUOTE>

I upload data with WebClient and my sever never return the final 0 if I do
it like below. (I was forced to add a terminator to the data uploaded to
figurea out when data was all read)

FROM MSDN

Dim so As StateObject = CType(ar.AsyncS tate, StateObject)
Dim s As Socket = so.workSocket

Dim read As Integer = s.EndReceive(ar )

If read > 0 Then
so.sb.Append(En coding.ASCII.Ge tString(so.buff er, 0, read))
s.BeginReceive( so.buffer, 0, StateObject.BUF FER_SIZE, 0, New
AsyncCallback(A ddressOf Async_Send_Rece ive.Read_Callba ck), so)
Else
If so.sb.Length > 1 Then
'All the data has been read, so displays it to the console
Dim strContent As String
strContent = so.sb.ToString( )
Console.WriteLi ne([String].Format("Read {0} byte from socket" + "data
= {1} ", strContent.Leng th, strContent))
End If
s.Close()
End If
End Sub 'Read_Callback
--
Ceers,
Crirus

------------------------------
If work were a good thing, the boss would take it all from you

------------------------------

"Crirus" <Cr****@datagro up.ro> wrote in message
news:%2******** ********@TK2MSF TNGP09.phx.gbl. ..
That's exactly how I use it..

I'm asking if the Download method will return by itself after all data is
received or should I take care to check if all data was received by parsing and interpreting the data..

Data is sent by a server I designed with TCP sockets following some MS
sample.
However, in the server, I receive data request in multiple steps. First I
receive only the headers, and in a second receive I have the body if the
method of WebClient was UploadData...

I may miss something at the server side anyway, there should be some length that I can check against received buffer but I cant figure out now.

I cant find that length
The BeginReceive of the receive socket have the following function as
callback:


Private Sub Receive(ByVal ar As IAsyncResult) 'este apelata cand au ajuns
date
Try
Dim content As [String] = [String].Empty
Dim state As StateObject = CType(ar.AsyncS tate, StateObject) '
Retrieve the state object and the handler socket from the async state
object.
Dim handler As Socket = state.workSocke t 'this is client socket
Dim bytesRead As Integer = handler.EndRece ive(ar) '
Read data from client socket.

If bytesRead > 0 Then
'state.sb.Appen d(Encoding.ASCI I.GetString(sta te.buffer, 0,
bytesRead))
' There might be more data, so store the data received so far.
Dim lastLength As Integer
If state.received Is Nothing Then
lastLength = 0
Else
lastLength = UBound(state.re ceived)
End If
ReDim Preserve state.received( lastLength + bytesRead - 1)
Array.Copy(stat e.buffer, 0, state.received, lastLength,
bytesRead)
content = Chr(state.recei ved(state.recei ved.Length - 1))
Dim bc As BitConverter
'Dim lenBuff As Long = bc.ToInt64(stat e.buffer, 0)
If content = state.endTransm ition Then
' All the data has been read from the client
Dim utilData(state. received.Length - 102 -
state.endTransm ition.Length) As Byte 'pana la 100 sunt headere ultimul e
terminator
Array.Copy(stat e.received, 101, utilData, 0,
state.received. Length - 101 - state.endTransm ition.Length)
'If lenBuff = bytesRead Then
'Request received; processing ...
Dim response As Byte()
If Not myGame Is Nothing Then
response = myGame.ProcessC lientRequest(ut ilData)
Else
response = myManager.Proce ssClientRequest (utilData) End If
'"Accept-Language:en-us" + vbCrLf +
"Accept-Encoding:gzip, deflate" + vbCrLf +
'Dim response As Byte() =
Encoding.ASCII. GetBytes("<html ><head></head><body>Noth ing to see
here!</body></html>")
allBytesRead += bytesRead
RaiseEvent BytesReceived(a llBytesRead)
Send(handler, response)
Else
handler.BeginRe ceive(state.buf fer, 0,
StateObject.Buf ferSize, 0, New AsyncCallback(A ddressOf Receive), state) '
Not all data received. Get more.
End If
Else
'nothing to read: client closed
handler.Shutdow n(SocketShutdow n.Both)
handler.Close()
openConnections -= 1
RaiseEvent NewConnection(o penConnections)
End If
Catch e As SocketException
'client close
openConnections -= 1
RaiseEvent NewConnection(o penConnections)
End Try
End Sub 'ReadCallback



--
Ceers,
Crirus

------------------------------
If work were a good thing, the boss would take it all from you

------------------------------

"Cor" <no*@non.com> wrote in message
news:%2******** *******@TK2MSFT NGP10.phx.gbl.. .
Hi Crirus,
The webclient is in my idea the best placed in a seperated thread, then

you
could abort it also, I think that will be your next problem if you have

not
done that already.

I do use it with a try, catch end block an then empty string is a good
result.

Public Function download(Uri, Filename) as string
Dim browser As New System.Net.WebC lient
Try
browser.Downloa dFile(Uri, Filename)
Return ""
Catch Ex As Exception
Return Uri & " " & Ex.Message
End Try
End Function

Cor
I use a webClient for requesting data from a server of mine.
Should I worry about long ammount of data sent by server in the client

side?
Or, another way, should I send some kind of a terminator sign in the

data
in
order to realise in the client that all data arrived?



Nov 20 '05 #5
I send strings... yes, I have a final transmision identifier, just asking if
the sockets itself cant figure out somehow

--
Ceers,
Crirus

------------------------------
If work were a good thing, the boss would take it all from you

------------------------------

"Cor" <no*@non.com> wrote in message
news:eB******** ******@TK2MSFTN GP09.phx.gbl...
Hi Crirus,
If you want to know what is the filelength in advance, you can use the
httpwebrequest with the httpwebheaderco llection,

But that will of course never be sure.

If it are your own files, why not set something as a filesize or other
hashitem in an endblock of the file and test that when you got it?

Cor

Nov 20 '05 #6

"Crirus" <Cr****@datagro up.ro> wrote in message
news:%2******** ********@TK2MSF TNGP12.phx.gbl. ..
Hi all!

I use a webClient for requesting data from a server of mine.
Should I worry about long ammount of data sent by server in the client side? Or, another way, should I send some kind of a terminator sign in the data in order to realise in the client that all data arrived?

--
Ceers,
Crirus


Read the HTTP protocol.
With an HTTP 1.0 request or an HTTP 1.1 request with the Connection:clos e
header, the web server will close the connection after all the data has been
sent. So you can just read to the end of the socket.

With HTTP 1.1 without the Connection:clos e request header the web server
will not close the connection, and you should determine the length of the
data the Content-Length response header.

For the server, if transmitting a known quantity of data (like a string or
file), the server should set the Content-Length header to tell the client
how much data to read from the socket.

David
Nov 20 '05 #7
The server is mine as well.. So I have to deal with the protocol....
I just want to know about this mechanism

--
Ceers,
Crirus

------------------------------
If work were a good thing, the boss would take it all from you

------------------------------

"David Browne" <davidbaxterbro wne no potted me**@hotmail.co m> wrote in
message news:%2******** ********@TK2MSF TNGP12.phx.gbl. ..

"Crirus" <Cr****@datagro up.ro> wrote in message
news:%2******** ********@TK2MSF TNGP12.phx.gbl. ..
Hi all!

I use a webClient for requesting data from a server of mine.
Should I worry about long ammount of data sent by server in the client side?
Or, another way, should I send some kind of a terminator sign in the

data in
order to realise in the client that all data arrived?

--
Ceers,
Crirus

Read the HTTP protocol.
With an HTTP 1.0 request or an HTTP 1.1 request with the Connection:clos e
header, the web server will close the connection after all the data has

been sent. So you can just read to the end of the socket.

With HTTP 1.1 without the Connection:clos e request header the web server
will not close the connection, and you should determine the length of the
data the Content-Length response header.

For the server, if transmitting a known quantity of data (like a string or
file), the server should set the Content-Length header to tell the client
how much data to read from the socket.

David

Nov 20 '05 #8

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

0
1714
by: d21mike | last post by:
When I get the following input stream of chunked data I ge Request.TotalBytes = -1 because there is not Content-Length. The following will work because I hardcoded the correct length: TotalBytes = 16 strBRequest = Request.BinaryRead(TotalBytes) I get: 16 bytes of data (see below) However, if I don't know the length and try to use something > 16 li 100 then I get at error and the TotalBytes is set to 16 and th
0
1140
by: Ali Mazaheri | last post by:
Hi there, Anyone knows how to capture a Chunked data (Transfer-Encoding=chunked) through httpwebresponse? Regards
1
2620
by: Ali Mazaheri | last post by:
Hi there, Has anyone tried to capture a Chunked data (Transfer-Encoding=chunked) through httpwebresponse? Regards
0
1443
by: Jeremy | last post by:
Hi all, I'm using the webclient class in VB.net to download a web page and save it locally. I have a few chars in the html that appear fine when the html page is viewed itself in IE, but once the webclient downloads the file, and saves it, if I then try to view the file saved by the webclient, a few chars are completely garbled. Here are a few examples of chars that get wacky...
0
996
by: Peter Hayes | last post by:
I have successfully used the WebClient to access a web site using the name-value list to send name=value pairs for a database query that is run on the far end. I get the results back and all is fine. The problem is that the queries I can send are limited because of the way the namevalue collection works. The database query 'decoder' at the far end expects that multiple values for the same field (or name) be sent in multiple tags, so that...
12
2748
by: =?Utf-8?B?Sm9uYXMgRXJpY3Nzb24=?= | last post by:
The release notes for IIS 6.0 talks about ASP support for POSTs from clients that are using chunked Transfer-encoding. Does anyone know what this support means? It does not seem to be possible to use Request.BinaryRead, and Request.TotalBytes returns -1. Thanks
2
5866
by: VeeraLakshmi | last post by:
Can anybody tell me how to get or read the value of transfer encoding. I got the HTTP Response header as "Transfer-Encoding: chunked".But i can't get the chunk size or the chunked data. Without getting those details i cant read the content of the site.If Content-Length is in the HTTP header,i can read upto that length.But in this Transfer-Encoding case,i cant know any other details except the value "chunked".So suggest me to read the content...
1
2132
by: Tom | last post by:
Hey, On the server side, i am using <?php print_r($_REQUEST); ?> To show me the data posted, so i know its not the server at fault..
2
3397
by: letisoft | last post by:
Hello, I am sorry but I am not ASP developer, anyhow I have problems communicating with ASP pages so I am trying to help the developer of the page. The problem is with posting chunked data. According to HTTP 1.1 Content-Length header is missing so Request.TotalBytes is -1. The only idea I have is that we should use Request.ReadBinary till end of the stream is reached?
0
9628
marktang
by: marktang | last post by:
ONU (Optical Network Unit) is one of the key components for providing high-speed Internet services. Its primary function is to act as an endpoint device located at the user's premises. However, people are often confused as to whether an ONU can Work As a Router. In this blog post, we’ll explore What is ONU, What Is Router, ONU & Router’s main usage, and What is the difference between ONU and Router. Let’s take a closer look ! Part I. Meaning of...
0
9464
by: Hystou | last post by:
Most computers default to English, but sometimes we require a different language, especially when relocating. Forgot to request a specific language before your computer shipped? No problem! You can effortlessly switch the default language on Windows 10 without reinstalling. I'll walk you through it. First, let's disable language synchronization. With a Microsoft account, language settings sync across devices. To prevent any complications,...
0
10292
Oralloy
by: Oralloy | last post by:
Hello folks, I am unable to find appropriate documentation on the type promotion of bit-fields when using the generalised comparison operator "<=>". The problem is that using the GNU compilers, it seems that the internal comparison operator "<=>" tries to promote arguments from unsigned to signed. This is as boiled down as I can make it. Here is my compilation command: g++-12 -std=c++20 -Wnarrowing bit_field.cpp Here is the code in...
0
9923
tracyyun
by: tracyyun | last post by:
Dear forum friends, With the development of smart home technology, a variety of wireless communication protocols have appeared on the market, such as Zigbee, Z-Wave, Wi-Fi, Bluetooth, etc. Each protocol has its own unique characteristics and advantages, but as a user who is planning to build a smart home system, I am a bit confused by the choice of these technologies. I'm particularly interested in Zigbee because I've heard it does some...
0
8954
agi2029
by: agi2029 | last post by:
Let's talk about the concept of autonomous AI software engineers and no-code agents. These AIs are designed to manage the entire lifecycle of a software development project—planning, coding, testing, and deployment—without human intervention. Imagine an AI that can take a project description, break it down, write the code, debug it, and then launch it, all on its own.... Now, this would greatly impact the work of software developers. The idea...
1
7471
isladogs
by: isladogs | last post by:
The next Access Europe User Group meeting will be on Wednesday 1 May 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM). In this session, we are pleased to welcome a new presenter, Adolph Dupré who will be discussing some powerful techniques for using class modules. He will explain when you may want to use classes instead of User Defined Types (UDT). For example, to manage the data in unbound forms. Adolph will...
1
4031
by: 6302768590 | last post by:
Hai team i want code for transfer the data from one system to another through IP address by using C# our system has to for every 5mins then we have to update the data what the data is updated we have to send another system
2
3627
muto222
by: muto222 | last post by:
How can i add a mobile payment intergratation into php mysql website.
3
2860
bsmnconsultancy
by: bsmnconsultancy | last post by:
In today's digital era, a well-designed website is crucial for businesses looking to succeed. Whether you're a small business owner or a large corporation in Toronto, having a strong online presence can significantly impact your brand's success. BSMN Consultancy, a leader in Website Development in Toronto offers valuable insights into creating effective websites that not only look great but also perform exceptionally well. In this comprehensive...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.