By using this site, you agree to our updated Privacy Policy and our Terms of Use. Manage your Cookies Settings.
434,677 Members | 1,094 Online
Bytes IT Community
+ Ask a Question
Need help? Post your question and get tips & solutions from a community of 434,677 IT Pros & Developers. It's quick & easy.

WebClient and chunked data

P: n/a
Hi all!

I use a webClient for requesting data from a server of mine.
Should I worry about long ammount of data sent by server in the client side?
Or, another way, should I send some kind of a terminator sign in the data in
order to realise in the client that all data arrived?

--
Ceers,
Crirus

------------------------------
If work were a good thing, the boss would take it all from you

------------------------------
Nov 20 '05 #1
Share this Question
Share on Google+
7 Replies


P: n/a
Cor
Hi Crirus,
The webclient is in my idea the best placed in a seperated thread, then you
could abort it also, I think that will be your next problem if you have not
done that already.

I do use it with a try, catch end block an then empty string is a good
result.

Public Function download(Uri, Filename) as string
Dim browser As New System.Net.WebClient
Try
browser.DownloadFile(Uri, Filename)
Return ""
Catch Ex As Exception
Return Uri & " " & Ex.Message
End Try
End Function

Cor
I use a webClient for requesting data from a server of mine.
Should I worry about long ammount of data sent by server in the client side? Or, another way, should I send some kind of a terminator sign in the data in order to realise in the client that all data arrived?

Nov 20 '05 #2

P: n/a
That's exactly how I use it..

I'm asking if the Download method will return by itself after all data is
received or should I take care to check if all data was received by parsing
and interpreting the data..

Data is sent by a server I designed with TCP sockets following some MS
sample.
However, in the server, I receive data request in multiple steps. First I
receive only the headers, and in a second receive I have the body if the
method of WebClient was UploadData...

I may miss something at the server side anyway, there should be some length
that I can check against received buffer but I cant figure out now.

I cant find that length
The BeginReceive of the receive socket have the following function as
callback:


Private Sub Receive(ByVal ar As IAsyncResult) 'este apelata cand au ajuns
date
Try
Dim content As [String] = [String].Empty
Dim state As StateObject = CType(ar.AsyncState, StateObject) '
Retrieve the state object and the handler socket from the async state
object.
Dim handler As Socket = state.workSocket 'this is client socket

Dim bytesRead As Integer = handler.EndReceive(ar) '
Read data from client socket.

If bytesRead > 0 Then
'state.sb.Append(Encoding.ASCII.GetString(state.bu ffer, 0,
bytesRead))
' There might be more data, so store the data received so
far.
Dim lastLength As Integer
If state.received Is Nothing Then
lastLength = 0
Else
lastLength = UBound(state.received)
End If
ReDim Preserve state.received(lastLength + bytesRead - 1)
Array.Copy(state.buffer, 0, state.received, lastLength,
bytesRead)
content = Chr(state.received(state.received.Length - 1))
Dim bc As BitConverter
'Dim lenBuff As Long = bc.ToInt64(state.buffer, 0)
If content = state.endTransmition Then
' All the data has been read from the client
Dim utilData(state.received.Length - 102 -
state.endTransmition.Length) As Byte 'pana la 100 sunt headere ultimul e
terminator
Array.Copy(state.received, 101, utilData, 0,
state.received.Length - 101 - state.endTransmition.Length)
'If lenBuff = bytesRead Then
'Request received; processing ...
Dim response As Byte()
If Not myGame Is Nothing Then
response = myGame.ProcessClientRequest(utilData)
Else
response = myManager.ProcessClientRequest(utilData)
End If
'"Accept-Language:en-us" + vbCrLf +
"Accept-Encoding:gzip, deflate" + vbCrLf +
'Dim response As Byte() =
Encoding.ASCII.GetBytes("<html><head></head><body>Nothing to see
here!</body></html>")
allBytesRead += bytesRead
RaiseEvent BytesReceived(allBytesRead)
Send(handler, response)
Else
handler.BeginReceive(state.buffer, 0,
StateObject.BufferSize, 0, New AsyncCallback(AddressOf Receive), state) '
Not all data received. Get more.
End If
Else
'nothing to read: client closed
handler.Shutdown(SocketShutdown.Both)
handler.Close()
openConnections -= 1
RaiseEvent NewConnection(openConnections)
End If
Catch e As SocketException
'client close
openConnections -= 1
RaiseEvent NewConnection(openConnections)
End Try
End Sub 'ReadCallback



--
Ceers,
Crirus

------------------------------
If work were a good thing, the boss would take it all from you

------------------------------

"Cor" <no*@non.com> wrote in message
news:%2***************@TK2MSFTNGP10.phx.gbl...
Hi Crirus,
The webclient is in my idea the best placed in a seperated thread, then you could abort it also, I think that will be your next problem if you have not done that already.

I do use it with a try, catch end block an then empty string is a good
result.

Public Function download(Uri, Filename) as string
Dim browser As New System.Net.WebClient
Try
browser.DownloadFile(Uri, Filename)
Return ""
Catch Ex As Exception
Return Uri & " " & Ex.Message
End Try
End Function

Cor
I use a webClient for requesting data from a server of mine.
Should I worry about long ammount of data sent by server in the client side?
Or, another way, should I send some kind of a terminator sign in the

data in
order to realise in the client that all data arrived?


Nov 20 '05 #3

P: n/a
Cor
Hi Crirus,
If you want to know what is the filelength in advance, you can use the
httpwebrequest with the httpwebheadercollection,

But that will of course never be sure.

If it are your own files, why not set something as a filesize or other
hashitem in an endblock of the file and test that when you got it?

Cor
Nov 20 '05 #4

P: n/a
I just tested this ideea and dont work:

I never receive a 0 length from EndReceive when all data was read. They say
this in MSDN:

<QUOTE>If you are using a connection-oriented protocol, the EndReceive
method will read as much data as is available up to the number of bytes you
specified in the size parameter of the BeginReceive method. If the remote
host shuts down the Socket connection with the Shutdown method, and all
available data has been received, the EndReceive method will complete
immediately and return zero bytes.To obtain the received data, call the
AsyncState method of the IAsyncResult, and extract the buffer contained in
the resulting state object.</QUOTE>

I upload data with WebClient and my sever never return the final 0 if I do
it like below. (I was forced to add a terminator to the data uploaded to
figurea out when data was all read)

FROM MSDN

Dim so As StateObject = CType(ar.AsyncState, StateObject)
Dim s As Socket = so.workSocket

Dim read As Integer = s.EndReceive(ar)

If read > 0 Then
so.sb.Append(Encoding.ASCII.GetString(so.buffer, 0, read))
s.BeginReceive(so.buffer, 0, StateObject.BUFFER_SIZE, 0, New
AsyncCallback(AddressOf Async_Send_Receive.Read_Callback), so)
Else
If so.sb.Length > 1 Then
'All the data has been read, so displays it to the console
Dim strContent As String
strContent = so.sb.ToString()
Console.WriteLine([String].Format("Read {0} byte from socket" + "data
= {1} ", strContent.Length, strContent))
End If
s.Close()
End If
End Sub 'Read_Callback
--
Ceers,
Crirus

------------------------------
If work were a good thing, the boss would take it all from you

------------------------------

"Crirus" <Cr****@datagroup.ro> wrote in message
news:%2****************@TK2MSFTNGP09.phx.gbl...
That's exactly how I use it..

I'm asking if the Download method will return by itself after all data is
received or should I take care to check if all data was received by parsing and interpreting the data..

Data is sent by a server I designed with TCP sockets following some MS
sample.
However, in the server, I receive data request in multiple steps. First I
receive only the headers, and in a second receive I have the body if the
method of WebClient was UploadData...

I may miss something at the server side anyway, there should be some length that I can check against received buffer but I cant figure out now.

I cant find that length
The BeginReceive of the receive socket have the following function as
callback:


Private Sub Receive(ByVal ar As IAsyncResult) 'este apelata cand au ajuns
date
Try
Dim content As [String] = [String].Empty
Dim state As StateObject = CType(ar.AsyncState, StateObject) '
Retrieve the state object and the handler socket from the async state
object.
Dim handler As Socket = state.workSocket 'this is client socket
Dim bytesRead As Integer = handler.EndReceive(ar) '
Read data from client socket.

If bytesRead > 0 Then
'state.sb.Append(Encoding.ASCII.GetString(state.bu ffer, 0,
bytesRead))
' There might be more data, so store the data received so far.
Dim lastLength As Integer
If state.received Is Nothing Then
lastLength = 0
Else
lastLength = UBound(state.received)
End If
ReDim Preserve state.received(lastLength + bytesRead - 1)
Array.Copy(state.buffer, 0, state.received, lastLength,
bytesRead)
content = Chr(state.received(state.received.Length - 1))
Dim bc As BitConverter
'Dim lenBuff As Long = bc.ToInt64(state.buffer, 0)
If content = state.endTransmition Then
' All the data has been read from the client
Dim utilData(state.received.Length - 102 -
state.endTransmition.Length) As Byte 'pana la 100 sunt headere ultimul e
terminator
Array.Copy(state.received, 101, utilData, 0,
state.received.Length - 101 - state.endTransmition.Length)
'If lenBuff = bytesRead Then
'Request received; processing ...
Dim response As Byte()
If Not myGame Is Nothing Then
response = myGame.ProcessClientRequest(utilData)
Else
response = myManager.ProcessClientRequest(utilData) End If
'"Accept-Language:en-us" + vbCrLf +
"Accept-Encoding:gzip, deflate" + vbCrLf +
'Dim response As Byte() =
Encoding.ASCII.GetBytes("<html><head></head><body>Nothing to see
here!</body></html>")
allBytesRead += bytesRead
RaiseEvent BytesReceived(allBytesRead)
Send(handler, response)
Else
handler.BeginReceive(state.buffer, 0,
StateObject.BufferSize, 0, New AsyncCallback(AddressOf Receive), state) '
Not all data received. Get more.
End If
Else
'nothing to read: client closed
handler.Shutdown(SocketShutdown.Both)
handler.Close()
openConnections -= 1
RaiseEvent NewConnection(openConnections)
End If
Catch e As SocketException
'client close
openConnections -= 1
RaiseEvent NewConnection(openConnections)
End Try
End Sub 'ReadCallback



--
Ceers,
Crirus

------------------------------
If work were a good thing, the boss would take it all from you

------------------------------

"Cor" <no*@non.com> wrote in message
news:%2***************@TK2MSFTNGP10.phx.gbl...
Hi Crirus,
The webclient is in my idea the best placed in a seperated thread, then

you
could abort it also, I think that will be your next problem if you have

not
done that already.

I do use it with a try, catch end block an then empty string is a good
result.

Public Function download(Uri, Filename) as string
Dim browser As New System.Net.WebClient
Try
browser.DownloadFile(Uri, Filename)
Return ""
Catch Ex As Exception
Return Uri & " " & Ex.Message
End Try
End Function

Cor
I use a webClient for requesting data from a server of mine.
Should I worry about long ammount of data sent by server in the client

side?
Or, another way, should I send some kind of a terminator sign in the

data
in
order to realise in the client that all data arrived?



Nov 20 '05 #5

P: n/a
I send strings... yes, I have a final transmision identifier, just asking if
the sockets itself cant figure out somehow

--
Ceers,
Crirus

------------------------------
If work were a good thing, the boss would take it all from you

------------------------------

"Cor" <no*@non.com> wrote in message
news:eB**************@TK2MSFTNGP09.phx.gbl...
Hi Crirus,
If you want to know what is the filelength in advance, you can use the
httpwebrequest with the httpwebheadercollection,

But that will of course never be sure.

If it are your own files, why not set something as a filesize or other
hashitem in an endblock of the file and test that when you got it?

Cor

Nov 20 '05 #6

P: n/a

"Crirus" <Cr****@datagroup.ro> wrote in message
news:%2****************@TK2MSFTNGP12.phx.gbl...
Hi all!

I use a webClient for requesting data from a server of mine.
Should I worry about long ammount of data sent by server in the client side? Or, another way, should I send some kind of a terminator sign in the data in order to realise in the client that all data arrived?

--
Ceers,
Crirus


Read the HTTP protocol.
With an HTTP 1.0 request or an HTTP 1.1 request with the Connection:close
header, the web server will close the connection after all the data has been
sent. So you can just read to the end of the socket.

With HTTP 1.1 without the Connection:close request header the web server
will not close the connection, and you should determine the length of the
data the Content-Length response header.

For the server, if transmitting a known quantity of data (like a string or
file), the server should set the Content-Length header to tell the client
how much data to read from the socket.

David
Nov 20 '05 #7

P: n/a
The server is mine as well.. So I have to deal with the protocol....
I just want to know about this mechanism

--
Ceers,
Crirus

------------------------------
If work were a good thing, the boss would take it all from you

------------------------------

"David Browne" <davidbaxterbrowne no potted me**@hotmail.com> wrote in
message news:%2****************@TK2MSFTNGP12.phx.gbl...

"Crirus" <Cr****@datagroup.ro> wrote in message
news:%2****************@TK2MSFTNGP12.phx.gbl...
Hi all!

I use a webClient for requesting data from a server of mine.
Should I worry about long ammount of data sent by server in the client side?
Or, another way, should I send some kind of a terminator sign in the

data in
order to realise in the client that all data arrived?

--
Ceers,
Crirus

Read the HTTP protocol.
With an HTTP 1.0 request or an HTTP 1.1 request with the Connection:close
header, the web server will close the connection after all the data has

been sent. So you can just read to the end of the socket.

With HTTP 1.1 without the Connection:close request header the web server
will not close the connection, and you should determine the length of the
data the Content-Length response header.

For the server, if transmitting a known quantity of data (like a string or
file), the server should set the Content-Length header to tell the client
how much data to read from the socket.

David

Nov 20 '05 #8

This discussion thread is closed

Replies have been disabled for this discussion.