file transfer help?

[Bonzol]

vb.net 2003 Windows application

We have a Client/Server system set up by communicating through a
TCPClient object. The server loops continuously using a tcplistener
device and each time a client object attempts a connection, a new
instance of a Client object is created (in the server's clients
hashtable). The client object on the server (the client solution itself
has same communication setup only with the code in the main form)
handles communiction by using:
visual basic
code:--------------------------------------------------------------------------------
Private Sub StreamReceiver(ByVal ar As IAsyncResult)
Dim BytesRead As Integer
Dim strMessage As String

Try
' Ensure that no other threads try to use the stream at the
same time.
SyncLock client.GetStream
' Finish asynchronous read into readBuffer and get
number of bytes read.
BytesRead = client.GetStream.EndRead(ar)
End SyncLock

' Convert the byte array the message was saved into, minus
one for the
' Chr(13).
strMessage = Encoding.ASCII.GetString(readBuffer, 0,
BytesRead - 1)
'LineReceived event points to the server frm (which has the hashtable
of object of type Client (i.e. 'Me')) method 'OnLineReceived'
which splits the strMessage into "command" and which ever data is
supplied
Then performs a function based on the command.
RaiseEvent LineReceived(Me, strMessage)

' Ensure that no other threads try to use the stream at the
same time.
SyncLock client.GetStream
' Start a new asynchronous read into readBuffer.
client.GetStream.BeginRead(readBuffer, 0,
READ_BUFFER_SIZE, AddressOf StreamReceiver, Nothing)
End SyncLock
Catch e As Exception
End Try
End Sub

On the Client solution side, when run a tcpclient object is created:

client = New TcpClient(serverIP, serverPort)

' Start an asynchronous read invoking DoRead to avoid
lagging the user
' interface.
client.GetStream.BeginRead(readBuffer, 0,
READ_BUFFER_SIZE, AddressOf DoRead, Nothing)

and a command is sent to login ("CONNECT|username|password")
'|' is the delimeter used (or Chr(124))
--------------------------------------------------------------------------------

The problem starts when we've tried to implement a file transfer
between the server and client. We have created loops that read from a
file into a 'buffer(1024) as byte' in 1024 byte chunks, then
writing that buffer to the 'netstream = client.getstream' object
using:
visual basic
code:--------------------------------------------------------------------------------
Public Sub SendFile(ByVal filename As String, ByVal category As String,
ByVal size As Integer)
'Integer storing the total amount of bytes read when filling
the buffer
Dim totalbytesread As Integer = 0
Dim bytesread As Integer
'Byte array buffer of size specified by BUFFER_SIZE
Dim buffer(BUFFER_SIZE) As Byte
'Dim size As Integer = info.Length

If fs Is Nothing Then
fs = New FileStream("C:\i.doc", FileMode.Open,
FileAccess.Read, FileShare.Read, BUFFER_SIZE)
End If
Try

SyncLock netstream
While (totalbytesread < size And netstream.CanWrite)
bytesread = fs.Read(buffer, 0, BUFFER_SIZE)

netstream.Write(buffer, 0, bytesread)

totalbytesread = totalbytesread + bytesread
End While
End SyncLock
fs.Close()
Catch ex As Exception
MsgBox(ex.ToString)
End Try
End sub
--------------------------------------------------------------------------------

And at the client end we have tried a similar loop that reads from the
tcpclient object's stream in 1024 byte chunks and then writes to a
file stream. For some reason, they can read some of the byte chunks and
write it to the file. But not all of the byte chunks are received, and
the chunks are often out of order. We have tried setting the receiving
end of the file transfer
Client.getstream.beginread(buffer,0,BUFFER_SIZE,Ad dressOf
ReceiveFile,Nothing) where ReceiveFile loops through the stream and
writes to a file.

We've based the communication model on an msdn chat client/server
example, and have attempted as many different ways to loop through the
stream on both the sending and receiving end, including:
Send file end:
· Read file into buffer(filesize)
· Write buffer into stream

Receive file end:
· Read buffer(filesize) from stream
(client.getstream.read(buffer,0,buffer.length)
· Write buffer to filestream (fs.write(buffer,0,buffer.length)

Which worked for small txt and doc files (without images) and worked
for one 64.1kb bmp image. Any help would be appreciated on how to get
this to work.Some code examples would be greatly appreciated

Thanx in advance

[teslar91]

You and I learned from the same example code. :)

Bonzol wrote:

And at the client end we have tried a similar loop that reads from the
tcpclient object's stream in 1024 byte chunks and then writes to a
file stream. For some reason, they can read some of the byte chunks and
write it to the file. But not all of the byte chunks are received, and
the chunks are often out of order.

<snip>

I didn't spot anything wrong with the partial code you provided, so I'm
going to take a guess based on the way you worded this.

TCP breaks up or combines what to you were individual transmissions, as
needed and according to its own rules; though it always maintains the
order.

As an example, let's say your server does this:
1) Wait for a connection
2) .Send(buf, 0, 1024)
3) .Send(buf, 0, 1024)

And your client does this:
1) Connect to server
2) .Read(buf, 0, 1024)
3) .Read(buf, 0, 1024)

Those two reads might actually get *less* than 1024 bytes each.
Meaning it would take additional reads to get all the data.

And here's where I'm guessing. I'm betting since your client *knows*
the file size, it *expects* each .Read or .EndRead to return a certain
amount of data, say 1024 bytes at a time for most of the file, and it
doesn't check the return value to see how many bytes were actually
read. It then .Writes the entire 1024 byte buffer to a file. The end
of that buffer may contain data from a previous read that wasn't
overwritten (perceived as out-of-order data). And since the total
amount of bytes read is less than the file size, you're also missing
data. The received file size would be correct, with the difference
made of duplicated, out-of-order data.

Here's what a file receive on the client using .Read would look like:

While (totalbytesread < size And netstream.CanWrite)
bytesread = fs.Read(buffer, 0, BUFFER_SIZE)
netstream.Write(buffer, 0, bytesread)
totalbytesread = totalbytesread + bytesread
End While

One more tip that might be applicable: Make sure you don't attempt to
convert a byte array containing binary data, such as from a picture
file, to a string using ASCIIEncoding. It doesn't handle byte values
128-255. Other encoding types I've tried also garble binary data in
various ways. There may yet be one that works, but I've decided it's
better to make sure binary data stays in byte arrays and never gets
converted to a string.

Well, I hope something here solves your problem. I'll continue
monitoring this thread until you have resolution.

[Bonzol]

Thanks heaps for the help, it explained what was happening
in an earlier attempt and gave me ideas on how to fix the problem.
I managed to get the transfer going by using the asynchronous
stream writing (beginWrite then endwrite) and reading (beginread and
endread) functions.

The basic pseudocode for that was

For Sending the file through the stream:

Begin an Asynchronous write to the stream connecting the client and
server
Loop untill the count of bytes read from file is >= filesize
-read 1024 bytes from file into byte array (which gives an
integer of how many bytes were read)
-write the amount of bytes read to the stream from byte array
-increase the count of bytes by amount of bytes read
end loop
end Asynchronous write
close filestream
For receiving files from the stream:

Loop until count of bytes read from file is >= filesize
- begin Asynchronous read from stream into byte array
- end Asynchronous read from stream (which gives the number of bytes
read)
- write the number of bytes read to file from byte array
end loop
close filestream
I think it was a combination asynchronous reads without async writes,
or
asyn writes without async reads that caused the headache. Thanks again,
hopefully
this might help some other people avoid an overload of stress.