C# Webresponse Error with special server

[Lukas S]

Hi,

I have written a short code snippet, that schould response the html source code of a website. Normally it wokrs fine I have testet it on some websites, but at thegreek.com I get an error.
here is the code:

public static string getSourceCode(string url)
        {
            HttpWebRequest req = (HttpWebRequest)WebRequest.Create(url);
            HttpWebResponse resp = (HttpWebResponse)req.GetResponse();
            StreamReader sr = new StreamReader(resp.GetResponseStream());
            string SourceCode = sr.ReadToEnd();
            sr.Close();
            resp.Close();
            return SourceCode;
        }
 

But when the url is http://thegreek.com I get the following error:

System.Net.WebException wurde nicht behandelt.
  Message=Die zugrunde liegende Verbindung wurde geschlossen: Unbekannter Fehler beim Empfangen..
  Source=System
  StackTrace:
       bei System.Net.HttpWebRequest.GetResponse()
       bei Scraper_TheGreek.WebWorker.getSourceCode(String url) in C:\Users\Admin\Desktop\Scraper_TheGreek\Scraper_TheGreek\WebWorker.cs:Zeile 16.
       bei Scraper_TheGreek.Program.Main(String[] args) in C:\Users\Admin\Desktop\Scraper_TheGreek\Scraper_TheGreek\Program.cs:Zeile 16.
       bei System.AppDomain._nExecuteAssembly(RuntimeAssembly assembly, String[] args)
       bei System.AppDomain.ExecuteAssembly(String assemblyFile, Evidence assemblySecurity, String[] args)
       bei Microsoft.VisualStudio.HostingProcess.HostProc.RunUsersAssembly()
       bei System.Threading.ThreadHelper.ThreadStart_Context(Object state)
       bei System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean ignoreSyncCtx)
       bei System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state)
       bei System.Threading.ThreadHelper.ThreadStart()
  InnerException: System.IO.IOException
       Message=Von der Übertragungsverbindung können keine Daten gelesen werden: Eine vorhandene Verbindung wurde vom Remotehost geschlossen.
       Source=System
       StackTrace:
            bei System.Net.Sockets.NetworkStream.Read(Byte[] buffer, Int32 offset, Int32 size)
            bei System.Net.PooledStream.Read(Byte[] buffer, Int32 offset, Int32 size)
            bei System.Net.Connection.SyncRead(HttpWebRequest request, Boolean userRetrievedStream, Boolean probeRead)
       InnerException: System.Net.Sockets.SocketException
            Message=Eine vorhandene Verbindung wurde vom Remotehost geschlossen
            Source=System
            ErrorCode=10054
            NativeErrorCode=10054
            StackTrace:
                 bei System.Net.Sockets.Socket.Receive(Byte[] buffer, Int32 offset, Int32 size, SocketFlags socketFlags)
                 bei System.Net.Sockets.NetworkStream.Read(Byte[] buffer, Int32 offset, Int32 size)
            InnerException: 
 

Does anybody know, what the mistake is?
In the browser it is now problem to diplay the website, it is possible with Firefox and Internet Explorer

[Aimee Bailey]

the problem is you are not passing the request a useragent string. simply ammend your code to the following (for example):

public static string getSourceCode(string url)
{
    HttpWebRequest req = (HttpWebRequest)WebRequest.Create(url);
    req.UserAgent = "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/534.21 (KHTML, like Gecko) Chrome/11.0.682.0 Safari/534.21";
    HttpWebResponse resp = (HttpWebResponse)req.GetResponse();
    StreamReader sr = new StreamReader(resp.GetResponseStream());
    string SourceCode = sr.ReadToEnd();
    sr.Close();
    resp.Close();
    return SourceCode;
}