471,089 Members | 1,276 Online
Bytes | Software Development & Data Engineering Community
Post +

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 471,089 software developers and data experts.

Http stream performance

Hi.
I'm reading a 5 meg (or so) file from the response stream of a WebRequest,
and it's pitifully slow. I mean *really* slow, in the order of 15 minutes to
download. Now obviously I'm caching this on the client (Windows app /
IsolatedStorage), but I can't have it taking this long to download in the
first instance. We've got Gigabit LAN. what's the bottleneck?

I could make the resource available via a different protocol / means if
necessary to overcome the problem.

Has anyone had similar issues or know of a better method of large network
downloads?

Cheers,
Leon
Jul 21 '05 #1
4 2708
Leon Jollans <Le**********************@xaman.com> wrote:
I'm reading a 5 meg (or so) file from the response stream of a WebRequest,
and it's pitifully slow. I mean *really* slow, in the order of 15 minutes to
download. Now obviously I'm caching this on the client (Windows app /
IsolatedStorage), but I can't have it taking this long to download in the
first instance. We've got Gigabit LAN. what's the bottleneck?


How are you reading it, exactly? (Please show some code, in other
words.) Have you tried reading it with a plain browser (or wget) to see
whether it's the client or the server that's holding it up?

--
Jon Skeet - <sk***@pobox.com>
http://www.pobox.com/~skeet
If replying to the group, please do not mail me too
Jul 21 '05 #2
It was definitely the client all along. but I think I've sussed it now: I
was reading the stream byte by byte into a pre-initialised array using
ReadByte from the stream directly and checking for -1 each time.

now I'm doing it using a BinaryReader in chunks and the speed is as per
expectations. code below for reference.

void readStream(Stream stream, long contentLength){
int i=0;
byte[] bytes;
BinaryReader reader = new BinaryReader(stream);
MemoryStream ms = new MemoryStream((int)contentLength);
do
{
bytes = reader.ReadBytes(128);
if(bytes.Length > 0)
{
ms.Write(bytes,0,bytes.Length);
}
else{
break;
}
}while( true );
reader.Close();
stream.Close();
ms.Flush();
bytes = ms.ToArray();
ms.Close();

...

cheers

Leon

"Jon Skeet" <sk***@pobox.com> wrote in message
news:MP************************@news.microsoft.com ...
Leon Jollans <Le**********************@xaman.com> wrote:
I'm reading a 5 meg (or so) file from the response stream of a WebRequest, and it's pitifully slow. I mean *really* slow, in the order of 15 minutes to download. Now obviously I'm caching this on the client (Windows app /
IsolatedStorage), but I can't have it taking this long to download in the first instance. We've got Gigabit LAN. what's the bottleneck?


How are you reading it, exactly? (Please show some code, in other
words.) Have you tried reading it with a plain browser (or wget) to see
whether it's the client or the server that's holding it up?

--
Jon Skeet - <sk***@pobox.com>
http://www.pobox.com/~skeet
If replying to the group, please do not mail me too

Jul 21 '05 #3
Leon Jollans <Le**********************@xaman.com> wrote:
It was definitely the client all along. but I think I've sussed it now: I
was reading the stream byte by byte into a pre-initialised array using
ReadByte from the stream directly and checking for -1 each time.
Yes, that would definitely slow it down :)
now I'm doing it using a BinaryReader in chunks and the speed is as per
expectations. code below for reference.

void readStream(Stream stream, long contentLength){
int i=0;
byte[] bytes;
BinaryReader reader = new BinaryReader(stream);
MemoryStream ms = new MemoryStream((int)contentLength);
do
{
bytes = reader.ReadBytes(128);
if(bytes.Length > 0)
{
ms.Write(bytes,0,bytes.Length);
}
else{
break;
}
}while( true );
reader.Close();
stream.Close();
ms.Flush();
bytes = ms.ToArray();
ms.Close();


Rather than doing that, I'd do something like:
(assuming you can make contentLength an int instead of a long)
byte[] data = new byte[contentLength];

int offset=0;
int remaining = contentLength;

while (remaining > 0)
{
int read = stream.Read (data, offset, remaining);
if (read==-1)
// Up to you - throw an exception or something similar
// because we didn't get as much as we expected
offset+=read;
remaining-=read;
}

--
Jon Skeet - <sk***@pobox.com>
http://www.pobox.com/~skeet
If replying to the group, please do not mail me too
Jul 21 '05 #4
That's much faster! thanks :)

Leon

"Jon Skeet" <sk***@pobox.com> wrote in message
news:MP************************@news.microsoft.com ...
Leon Jollans <Le**********************@xaman.com> wrote:
It was definitely the client all along. but I think I've sussed it now: I was reading the stream byte by byte into a pre-initialised array using
ReadByte from the stream directly and checking for -1 each time.


Yes, that would definitely slow it down :)
now I'm doing it using a BinaryReader in chunks and the speed is as per
expectations. code below for reference.

void readStream(Stream stream, long contentLength){
int i=0;
byte[] bytes;
BinaryReader reader = new BinaryReader(stream);
MemoryStream ms = new MemoryStream((int)contentLength);
do
{
bytes = reader.ReadBytes(128);
if(bytes.Length > 0)
{
ms.Write(bytes,0,bytes.Length);
}
else{
break;
}
}while( true );
reader.Close();
stream.Close();
ms.Flush();
bytes = ms.ToArray();
ms.Close();


Rather than doing that, I'd do something like:
(assuming you can make contentLength an int instead of a long)
byte[] data = new byte[contentLength];

int offset=0;
int remaining = contentLength;

while (remaining > 0)
{
int read = stream.Read (data, offset, remaining);
if (read==-1)
// Up to you - throw an exception or something similar
// because we didn't get as much as we expected
offset+=read;
remaining-=read;
}

--
Jon Skeet - <sk***@pobox.com>
http://www.pobox.com/~skeet
If replying to the group, please do not mail me too

Jul 21 '05 #5

This discussion thread is closed

Replies have been disabled for this discussion.

Similar topics

6 posts views Thread by Ian Robertson | last post: by
15 posts views Thread by Adam H. Peterson | last post: by
6 posts views Thread by Microsoft News | last post: by
4 posts views Thread by Leon Jollans | last post: by
reply views Thread by David Purton | last post: by
7 posts views Thread by Lee | last post: by
4 posts views Thread by Scott F. Brown | last post: by

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.