How to force StreamReader.Close() not to Read?

[tovishal2001]

Hi,

I am trying to read first 6000 bytes from a webpage, using StreamReader.Read(buffer, offset, no_of_bytes_to_read) method and trying to close the connection. Because, the useful data I need is present in the first 6000 bytes of the webpage and webpage size is atleast 100Kb. And, I need to repeat this operation for around 100s of such pages. Thats why I would close the connection after I am done reading initial 6000 bytes.

Now the problem is, I noticed that when code fires streamRead.Close(), it actually reads those remaining 94Kb of Data and then closes the connection !! This consumes extra 8 to 10 seconds just for closing the reader.(Verified via network transfer logs and the debug-time timestamps before and after this streamRead.Close() statement.)

Isn't this strange that why the reader should read the remaining bytes when I am actually telling it to close the connection and quit. Maybe I am doing it incorrectly, I tried different methods but none working. Is there other to achieve what I just need?

                byte[] bytes = new byte[6000];
 
                HttpWebRequest request = (HttpWebRequest)WebRequest.Create(query);
                HttpWebResponse resp = (HttpWebResponse)request.GetResponse();
                Stream reader = resp.GetResponseStream(); 
 
                int bytes_read = 0;
                int count;
                int bytes_to_read = 6000;
 
                while (bytes_read < 6000)
                {
                    count = reader.Read(bytes, bytes_read, bytes_to_read);
                    MessageBox.Show(System.Text.ASCIIEncoding.ASCII.GetString(bytes));
                    bytes_read += count;
                    bytes_to_read -= count;
                }
 
                reader.Close(); /// <-- This Line takes 8 to 10 seconds to finish its work because it reads remaining page from server and then closes
                resp.Close();
                string response = System.Text.ASCIIEncoding.ASCII.GetString(bytes);

- Vishal

[Plater]

I am guessing it has something to do with the Flush properties of the stream.
There is probably a low level stream control flag that you can set to tell it not to flush its buffer on close.
You could also maybe just try doing a .Dispose()?

[tovishal2001]

Yes, I thought on same line about the flag thing to change this behavior but didn't find one so far.

I tried Dispose() already, it doesn't solve the issue.

Also note that, this happens with this particular web-server. Same code works fine on other web-servers.

Here is the value of query which causes this issue.

String query = @"http://siteexplorer.search.yahoo.com/search?p=www.msn.com&bwm=i&bwms=p&bwmf=u&fr=sfp&fr 2=seo-rd-se"