Streaming files

[=?Utf-8?B?TWljaGFlbA==?=]

Hello,
I need some help. I am trying to write my own web server for serving up
large files
on my own network. I am using FileStream, BinaryReader, TcpClient, and
Sockets.
I am trying to break up the large file and send it a chunck at a time. But
I keep getting
and error stating that "Error occured: An operation on a socket could not be
performed
because the system lacked sufficient buffer space or because a queue was
full".

Here is some of the code:

Socket s;
byte[] incomingBuffer;

s = this.listener.AcceptSocket();

incomingBuffer = new byte[1024]; // 1Kb
bytesRead = s.Receive(incomingBuffer);
string buffer = Encoding.ASCII.GetString(incomingBuffer);
totalBytesRead = fileBytes.Length;
int iCount = 1;
int lIndex = 0;
int iTemp = 1;
int iDivides = totalBytesRead / 100000;
int iRemander = totalBytesRead % 100000;

while (iCount <= iDivides)
{
FileStream fs = new FileStream(virtualDir + requestedFile, FileMode.Open,
FileAccess.Read, FileShare.Read);

BinaryReader reader = new BinaryReader(fs);
reader.Read(fileBytes, lIndex, (100000 * iCount));

if ((numBytes = s.Send(fileBytes, fileBytes.Length, 0)) == -1)
this.Log("Socket error: Cannot send packet");
else
this.Log(String.Format("Bytes sent: {0}", numBytes)); lIndex +=
(100000 + iTemp);

iTemp = 0;
iCount++;
reader.Close();
fs.Close();
}
Can anyone give any direction? If I'm not going at it the right way,please
let me know.
One of the reasons I'm trying to break up the is because my app kept trying
to load the
whole thing into memory before it sent the file. So my memory for the app
would grow to
over a gig. I just need some way to read in the file and then stream it out.
Or stream it
as I read it.

Thanks,
Michael

[=?UTF-8?B?R8O2cmFuIEFuZGVyc3Nvbg==?=]

Michael wrote:

Hello,
I need some help. I am trying to write my own web server for serving up
large files
on my own network. I am using FileStream, BinaryReader, TcpClient, and
Sockets.
I am trying to break up the large file and send it a chunck at a time. But
I keep getting
and error stating that "Error occured: An operation on a socket could not be
performed
because the system lacked sufficient buffer space or because a queue was
full".

Here is some of the code:

Socket s;
byte[] incomingBuffer;

s = this.listener.AcceptSocket();

incomingBuffer = new byte[1024]; // 1Kb
bytesRead = s.Receive(incomingBuffer);
string buffer = Encoding.ASCII.GetString(incomingBuffer);

What do you do here? You know that characters are 16 bit values, right?
And you know that the ASCII encoding only uses 7-bit character codes, right?

How are you going to use the string?

>
totalBytesRead = fileBytes.Length;

What does fileBytes contain? How large is it? The way that you use it,
it looks like it contains all the data from the file. Wasn't that what
you were trying to avoid?

int iCount = 1;
int lIndex = 0;
int iTemp = 1;
int iDivides = totalBytesRead / 100000;
int iRemander = totalBytesRead % 100000;

while (iCount <= iDivides)
{
FileStream fs = new FileStream(virtualDir + requestedFile, FileMode.Open,
FileAccess.Read, FileShare.Read);

Why are you opening and closing the file for each iteration of the loop?

>
BinaryReader reader = new BinaryReader(fs);
reader.Read(fileBytes, lIndex, (100000 * iCount));

As you are increasing iCount for every iteration in the loop, you will
be reading larger and larger pieces of the file.

As you are increasing lIndex for every iteration in the loop, you will
be reading several copies of some data into different locations of the
array.

How large is fileBytes? If you want to fit all that data into the array,
it has to be much larger than the file.

>
if ((numBytes = s.Send(fileBytes, fileBytes.Length, 0)) == -1)

You are sending the entire contents of the array fileBytes, not just
what you read into it. That means that you will be sending several
copies of part of the file mixed with the data that was in the array
from the beginning.

this.Log("Socket error: Cannot send packet");
else
this.Log(String.Format("Bytes sent: {0}", numBytes)); lIndex +=
(100000 + iTemp);

Why are you increasing lIndex by 100001 the first time?

>
iTemp = 0;
iCount++;
reader.Close();
fs.Close();
}
Can anyone give any direction? If I'm not going at it the right way,please
let me know.
One of the reasons I'm trying to break up the is because my app kept trying
to load the
whole thing into memory before it sent the file. So my memory for the app
would grow to
over a gig. I just need some way to read in the file and then stream it out.
Or stream it
as I read it.

Thanks,
Michael

--
Göran Andersson
_____
http://www.guffa.com

[=?Utf-8?B?TWljaGFlbA==?=]

Then what should I do?

"Göran Andersson" wrote:

Michael wrote:

Hello,
I need some help. I am trying to write my own web server for serving up
large files
on my own network. I am using FileStream, BinaryReader, TcpClient, and
Sockets.
I am trying to break up the large file and send it a chunck at a time. But
I keep getting
and error stating that "Error occured: An operation on a socket could not be
performed
because the system lacked sufficient buffer space or because a queue was
full".

Here is some of the code:

Socket s;
byte[] incomingBuffer;

s = this.listener.AcceptSocket();

incomingBuffer = new byte[1024]; // 1Kb
bytesRead = s.Receive(incomingBuffer);
string buffer = Encoding.ASCII.GetString(incomingBuffer);

What do you do here? You know that characters are 16 bit values, right?
And you know that the ASCII encoding only uses 7-bit character codes, right?

How are you going to use the string?

totalBytesRead = fileBytes.Length;

What does fileBytes contain? How large is it? The way that you use it,
it looks like it contains all the data from the file. Wasn't that what
you were trying to avoid?

int iCount = 1;
int lIndex = 0;
int iTemp = 1;
int iDivides = totalBytesRead / 100000;
int iRemander = totalBytesRead % 100000;

while (iCount <= iDivides)
{
FileStream fs = new FileStream(virtualDir + requestedFile, FileMode.Open,
FileAccess.Read, FileShare.Read);

Why are you opening and closing the file for each iteration of the loop?

BinaryReader reader = new BinaryReader(fs);
reader.Read(fileBytes, lIndex, (100000 * iCount));

As you are increasing iCount for every iteration in the loop, you will
be reading larger and larger pieces of the file.

As you are increasing lIndex for every iteration in the loop, you will
be reading several copies of some data into different locations of the
array.

How large is fileBytes? If you want to fit all that data into the array,
it has to be much larger than the file.

if ((numBytes = s.Send(fileBytes, fileBytes.Length, 0)) == -1)

You are sending the entire contents of the array fileBytes, not just
what you read into it. That means that you will be sending several
copies of part of the file mixed with the data that was in the array
from the beginning.

this.Log("Socket error: Cannot send packet");
else
this.Log(String.Format("Bytes sent: {0}", numBytes)); lIndex +=
(100000 + iTemp);

Why are you increasing lIndex by 100001 the first time?

iTemp = 0;
iCount++;
reader.Close();
fs.Close();
}
Can anyone give any direction? If I'm not going at it the right way,please
let me know.
One of the reasons I'm trying to break up the is because my app kept trying
to load the
whole thing into memory before it sent the file. So my memory for the app
would grow to
over a gig. I just need some way to read in the file and then stream it out.
Or stream it
as I read it.

Thanks,
Michael

--
Göran Andersson
_____
http://www.guffa.com

[Peter Duniho]

On Thu, 19 Jul 2007 07:38:10 -0700, Michael
<Mi*****@discussions.microsoft.comwrote:

Then what should I do?

For receiving, you should:

* not convert your byte data to ASCII characters

There may be other things you should or should not do, but since you
didn't post all of your receiving code, it's hard to comment on what else
is wrong.

For sending, you should:

* calculate bytes read based on actual bytes read, rather than some
other variable or property
* open the file once, rather than each time through your loop
* only read the same amount of bytes each time you read
* update your index to read from based on the bytes read, rather than
on some incorrect calculation
* not have arbitrary things like "iTemp" that add 1 for no good reason
to some calculation

There may be other things you need to change, but I'd start with the above
list and see what you have at that point.

Pete

[Rad [Visual C# MVP]]

On Tue, 17 Jul 2007 09:00:01 -0700, Michael
<Mi*****@discussions.microsoft.comwrote:

>Hello,
I need some help. I am trying to write my own web server for serving up
large files
on my own network. I am using FileStream, BinaryReader, TcpClient, and
Sockets.
I am trying to break up the large file and send it a chunck at a time. But
I keep getting
and error stating that "Error occured: An operation on a socket could not be
performed
because the system lacked sufficient buffer space or because a queue was
full".

Here is some of the code:

Socket s;
byte[] incomingBuffer;

s = this.listener.AcceptSocket();

incomingBuffer = new byte[1024]; // 1Kb
bytesRead = s.Receive(incomingBuffer);
string buffer = Encoding.ASCII.GetString(incomingBuffer);
totalBytesRead = fileBytes.Length;
int iCount = 1;
int lIndex = 0;
int iTemp = 1;
int iDivides = totalBytesRead / 100000;
int iRemander = totalBytesRead % 100000;

while (iCount <= iDivides)
{
FileStream fs = new FileStream(virtualDir + requestedFile, FileMode.Open,
FileAccess.Read, FileShare.Read);

BinaryReader reader = new BinaryReader(fs);
reader.Read(fileBytes, lIndex, (100000 * iCount));

if ((numBytes = s.Send(fileBytes, fileBytes.Length, 0)) == -1)
this.Log("Socket error: Cannot send packet");
else
this.Log(String.Format("Bytes sent: {0}", numBytes)); lIndex +=
(100000 + iTemp);

iTemp = 0;
iCount++;
reader.Close();
fs.Close();
}
Can anyone give any direction? If I'm not going at it the right way,please
let me know.
One of the reasons I'm trying to break up the is because my app kept trying
to load the
whole thing into memory before it sent the file. So my memory for the app
would grow to
over a gig. I just need some way to read in the file and then stream it out.
Or stream it
as I read it.

Thanks,
Michael

Is there any particular reason you're not using plain old FTP?

--
http://bytes.thinkersroom.com