473,800 Members | 2,529 Online
Bytes | Software Development & Data Engineering Community
+ Post

Home Posts Topics Members FAQ

HttpWebRequest - data is truncated?

I am looking for some help about the method below which partly works.
The problem is that the data returned is truncated in that the
returned data has missing bytes near the beginning of the data. I am
using VS 2005 and C#.

Has anyone got any ideas?

public string getWebPage(stri ng url)
{
// add error handling
StringBuilder sb = new StringBuilder() ;
byte[] buf = new byte[8192];
try
{
HttpWebRequest request = (HttpWebRequest )WebRequest.Cre ate(url);
request.UserAge nt = "Mozilla/5.0 (Windows; U; Windows NT 5.1; de-
DE;"+
"rv:1.7.5) Gecko/20041108 Firefox/1.0";
request.Timeout =20000;
HttpWebResponse response = (HttpWebRespons e)request.GetRe sponse();
Stream responseStream = response.GetRes ponseStream();
StreamReader myTextReader = new StreamReader(re sponseStream);
char[] strBuffer = new char[25];
myTextReader.Re adBlock(strBuff er,0,25);
string stringBuffer = new string(strBuffe r);
if (stringBuffer.I ndexOf("GIF8")>-1 ||
stringBuffer.In dexOf("JFIF")>-1)
{
// image found
return "image";
}
else
{
// this is a text page
}
string tempString = null;
int count = 0;
do
{
count = responseStream. Read(buf,0,buf. Length);
if (count!=0)
{
tempString=Enco ding.ASCII.GetS tring(buf,0,cou nt);
sb.Append(tempS tring);
}
}while (count>0);
}
catch (WebException e)
{
return "ZenoBot getWebPage web failed to fetch "+e.ToString(); ;
}
catch (Exception e)
{
return "ZenoBot getWebPage web failed to fetch "+e.ToString(); ;
}

return sb.ToString();
}
}
Jun 27 '08 #1
2 3593
Logician <sa***@logician s.comwrote:
I am looking for some help about the method below which partly works.
The problem is that the data returned is truncated in that the
returned data has missing bytes near the beginning of the data. I am
using VS 2005 and C#.
The StreamReader is probably buffering data - reading more than 25
bytes and then saving it in its buffer.

Instead of using a StreamReader, just use the Stream directly to start
with. You're interested in the bytes which represent "GIF8", not the
actual *text* "GIF8", right?

--
Jon Skeet - <sk***@pobox.co m>
http://www.pobox.com/~skeet Blog: http://www.msmvps.com/jon.skeet
World class .NET training in the UK: http://iterativetraining.co.uk
Jun 27 '08 #2
On Apr 24, 10:15*pm, Jon Skeet [C# MVP] <sk...@pobox.co mwrote:
Logician <sa...@logician s.comwrote:
I am looking for some help about the method below which partly works.
The problem is that the data returned is truncated in that the
returned data has missing bytes near the beginning of the data. I am
using VS 2005 and C#.

The StreamReader is probably buffering data - reading more than 25
bytes and then saving it in its buffer.

Instead of using a StreamReader, just use the Stream directly to start
with. You're interested in the bytes which represent "GIF8", not the
actual *text* "GIF8", right?

--
Jon Skeet - <sk...@pobox.co m>http://www.pobox.com/~skeet* Blog:http://www.msmvps.com/jon.skeet
World class .NET training in the UK:http://iterativetraining.co.uk
Yes the 25 bytes was the issue. I saw code at
http://weblogs.asp.net/guys/archive/...21/414118.aspx which
helped a lot.
Jun 27 '08 #3

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

10
19362
by: Gregory A Greenman | last post by:
I'm trying to write a program in vb.net to automate filling out a series of forms on a website. There are three forms I need to fill out in sequence. The first one is urlencoded. My program is able to fill that one out just fine. The second form is multipart/form-data. Unfortunately, I haven't been able to fill that out in a way that makes the server happy. I set up a copy of this form at my web site so that I could see exactly what a...
9
8194
by: Mike Cronin via DotNetMonster.com | last post by:
Hi there, Can anyone tell me what level of encryption is used when making an HTTPS POST request through an instance of the System.Net.HttpWebRequest object? Thanks much in advance! Mike Cronin Data On Call - Programmer
5
7315
by: uthuras | last post by:
Machine : AIX 5.2 Product : UDB DB2 Release 8.1 FP4a I have problem loading data into destination table. The data file is huge with more than 6 Million records. This what i have done 1. Export the data into flat file (del format) 2. use the load command to load the data At step 2, i found that there are some errors registered in the load
0
1562
by: boxboy | last post by:
Hi, I'm writing a console application and am having a problem with HttpWebRequest when posting data to a webserver. A "System.Net.WebException: The server committed a protocol violation" is always being thrown when getting the response from a specific host. I'm pretty sure the framework's implementation doesn't like the formating or status code of the returned response header. It throws the exception without giving me a way of handling it...
6
8755
by: James MA | last post by:
I'm now writing a small program to communicate a web server to simulate a web client. I use te httpwebrequest to talk with the server, and it works find for "POST" method, however, when i test other link using "GET" method, i found that the cookies data has not included in the request. Here is the sample: ' sURL is the URL of server page ' pCookies is a varible contain the cookies data
0
14428
by: Grip | last post by:
Hi, I have gone throught the group and Microsoft's online help and have seen many suggestions but I am still seeking clarity: 1. I have an excel spreadsheet. Column A contains text that may be greater than 255 characters. 2. I have an access database. I link (not import) to the contents of the excel spreadsheet. In the design view in access, Column A has the data type "memo".
0
2588
by: joshblair | last post by:
Hello, I am trying to post XML documents to a third party using the HttpWebRequest. This URL uses HTTPS (SSL) but I don't have a client certificate to deal with. Apparently they are using WebMethods as the platform that receives these postings. I don't have any experience with that technology. The sample below is the from test app that I put together to post the XML (cXML Order Requests) documents. These documents, at least my
7
7692
by: Marc Bartsch | last post by:
Hi, I have a background worker in my C# app that makes a synchronous HttpWebRequest.GetResponse() call. The idea is to POST a file to a server on the internet. When I call HttpWebRequest.Abort() on the request object on another thread, GetResponse() returns with an exception as expected. However, when I monitor the network traffic, it does not seem to stop, but to continue to be active and to upload the file. The network is active even...
1
8048
by: Proogeren | last post by:
I have a problem with a httpwebrequest that I am creating. The request in itself looks correct but using fiddler I see that a www-authentication header is sent along as well. The code is pasted below. I do not add any www-authentication header here so I was wondering if anyone knows how to remove it. I have used almost 2 days trying to figure this out so help would be highly appreciated. CORRECT No proxy-authenticate header is present no...
0
9690
marktang
by: marktang | last post by:
ONU (Optical Network Unit) is one of the key components for providing high-speed Internet services. Its primary function is to act as an endpoint device located at the user's premises. However, people are often confused as to whether an ONU can Work As a Router. In this blog post, we’ll explore What is ONU, What Is Router, ONU & Router’s main usage, and What is the difference between ONU and Router. Let’s take a closer look ! Part I. Meaning of...
0
10504
Oralloy
by: Oralloy | last post by:
Hello folks, I am unable to find appropriate documentation on the type promotion of bit-fields when using the generalised comparison operator "<=>". The problem is that using the GNU compilers, it seems that the internal comparison operator "<=>" tries to promote arguments from unsigned to signed. This is as boiled down as I can make it. Here is my compilation command: g++-12 -std=c++20 -Wnarrowing bit_field.cpp Here is the code in...
1
10251
by: Hystou | last post by:
Overview: Windows 11 and 10 have less user interface control over operating system update behaviour than previous versions of Windows. In Windows 11 and 10, there is no way to turn off the Windows Update option using the Control Panel or Settings app; it automatically checks for updates and installs any it finds, whether you like it or not. For most users, this new feature is actually very convenient. If you want to control the update process,...
0
10033
tracyyun
by: tracyyun | last post by:
Dear forum friends, With the development of smart home technology, a variety of wireless communication protocols have appeared on the market, such as Zigbee, Z-Wave, Wi-Fi, Bluetooth, etc. Each protocol has its own unique characteristics and advantages, but as a user who is planning to build a smart home system, I am a bit confused by the choice of these technologies. I'm particularly interested in Zigbee because I've heard it does some...
0
6811
by: conductexam | last post by:
I have .net C# application in which I am extracting data from word file and save it in database particularly. To store word all data as it is I am converting the whole word file firstly in HTML and then checking html paragraph one by one. At the time of converting from word file to html my equations which are in the word document file was convert into image. Globals.ThisAddIn.Application.ActiveDocument.Select();...
0
5469
by: TSSRALBI | last post by:
Hello I'm a network technician in training and I need your help. I am currently learning how to create and manage the different types of VPNs and I have a question about LAN-to-LAN VPNs. The last exercise I practiced was to create a LAN-to-LAN VPN between two Pfsense firewalls, by using IPSEC protocols. I succeeded, with both firewalls in the same network. But I'm wondering if it's possible to do the same thing, with 2 Pfsense firewalls...
0
5606
by: adsilva | last post by:
A Windows Forms form does not have the event Unload, like VB6. What one acts like?
2
3764
muto222
by: muto222 | last post by:
How can i add a mobile payment intergratation into php mysql website.
3
2945
bsmnconsultancy
by: bsmnconsultancy | last post by:
In today's digital era, a well-designed website is crucial for businesses looking to succeed. Whether you're a small business owner or a large corporation in Toronto, having a strong online presence can significantly impact your brand's success. BSMN Consultancy, a leader in Website Development in Toronto offers valuable insights into creating effective websites that not only look great but also perform exceptionally well. In this comprehensive...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.