TCPClient in HTTp Request

[jin]

hi, i'm trying using the tcpClient to get a html file from net, instead of
using WebClient or WebRequest,

the main part of the source code is like this:

private void tcpconnect()
{
tcp=new TcpClient("www. yahoo.com",80);
tcp.NoDelay=fal se;
tcp.ReceiveTime out=60000;
tcp.ReceiveBuff erSize=25000;
stream = tcp.GetStream() ;
byte[] send = Encoding.ASCII. GetBytes("GET /index.html HTTP/1.0\r\n\r\n");
stream.Write(se nd,0,send.Lengt h);

byte[] receive = new byte[tcp.ReceiveBuff erSize];

int lastreceive=str eam.Read(receiv e,0,tcp.Receive BufferSize);
string str = Encoding.ASCII. GetString(recei ve,0,tcp.Receiv eBufferSize);

textBox1.Text=s tr;
tcp.Close();
stream.Close();
}

but the problem is, while i try to run this function, it didn't read all the
html source code for me, but just a part, i try to run twice of the read
method in the function, and it does continue reading for me. I think there
might be some other function which allow us to check whether the html file is
finished loading or not, but i'm dunno which is it and i can't find it
through the msdn library. Is that anyone could help?

Thank for all

[Johann Blake]

The TcpClient is only a communications protocol. It knows nothing of
HTML or even when something has been entirely read. It's up to your app
to figure that out.

To determine whether you have read in all of the HTML, either the
remote server will close the TCP session, in which case you can check
some flag for that (can't remember which one) or if the connection is
not closed, it is implied that you have read all of the data when you
encounter an "</HTML> tag.

Best Regards
Johann Blake

[Brett Romero]

Have you tried reading form a smaller webpage on a different site?
Also try increasing the buffer. Using a tool that reads you packets is
also helpful, such as MS Fiddler. There are others available as well.

Brett

[Ignacio Machin \( .NET/ C# MVP \)]

Hi,

and why are you doing this?

you will need to implement the HTTP protocol.

In your case I think that the problem is in the receiving end, what if you
change it like:

StreamReader reader = new StreamReader( stream );

string line;
while ( (line=reader.Re adLine())!=null )
Console.Write( line );

reader.Close();

cheers,

--
Ignacio Machin,
ignacio.machin AT dot.state.fl.us
Florida Department Of Transportation

"jin" <ji*@discussion s.microsoft.com > wrote in message
news:AB******** *************** ***********@mic rosoft.com...

hi, i'm trying using the tcpClient to get a html file from net, instead of
using WebClient or WebRequest,

the main part of the source code is like this:

private void tcpconnect()
{
tcp=new TcpClient("www. yahoo.com",80);
tcp.NoDelay=fal se;
tcp.ReceiveTime out=60000;
tcp.ReceiveBuff erSize=25000;
stream = tcp.GetStream() ;
byte[] send = Encoding.ASCII. GetBytes("GET /index.html HTTP/1.0\r\n\r\n");
stream.Write(se nd,0,send.Lengt h);

byte[] receive = new byte[tcp.ReceiveBuff erSize];

int lastreceive=str eam.Read(receiv e,0,tcp.Receive BufferSize);
string str = Encoding.ASCII. GetString(recei ve,0,tcp.Receiv eBufferSize);

textBox1.Text=s tr;
tcp.Close();
stream.Close();
}

but the problem is, while i try to run this function, it didn't read all
the
html source code for me, but just a part, i try to run twice of the read
method in the function, and it does continue reading for me. I think there
might be some other function which allow us to check whether the html file
is
finished loading or not, but i'm dunno which is it and i can't find it
through the msdn library. Is that anyone could help?

Thank for all

[Joerg Jooss]

jin wrote:

hi, i'm trying using the tcpClient to get a html file from net,
instead of using WebClient or WebRequest,

the main part of the source code is like this:

private void tcpconnect()
{
tcp=new TcpClient("www. yahoo.com",80);
tcp.NoDelay=fal se;
tcp.ReceiveTime out=60000;
tcp.ReceiveBuff erSize=25000;
stream = tcp.GetStream() ;
byte[] send = Encoding.ASCII. GetBytes("GET /index.html
HTTP/1.0\r\n\r\n"); stream.Write(se nd,0,send.Lengt h);

byte[] receive = new byte[tcp.ReceiveBuff erSize];

int lastreceive=str eam.Read(receiv e,0,tcp.Receive BufferSize);
string str =
Encoding.ASCII. GetString(recei ve,0,tcp.Receiv eBufferSize);

textBox1.Text=s tr;
tcp.Close();
stream.Close();
}

but the problem is, while i try to run this function, it didn't read
all the html source code for me, but just a part, i try to run twice
of the read method in the function, and it does continue reading for
me. I think there might be some other function which allow us to
check whether the html file is finished loading or not, but i'm dunno
which is it and i can't find it through the msdn library. Is that
anyone could help?

Classical NetworkStream programming error: You just cannot expect to
read all data with a single call to Read(), regardless of the receive
buffer's size. You have to loop until Read() returns 0.
Cheers,
--
http://www.joergjooss.de
mailto:ne****** **@joergjooss.d e

[Chad Z. Hower aka Kudzu]

=?Utf-8?B?amlu?= <ji*@discussion s.microsoft.com > wrote in
news:AB******** *************** ***********@mic rosoft.com:

hi, i'm trying using the tcpClient to get a html file from net,
instead of using WebClient or WebRequest,
Why? I would STRONGLY recommend against this. HTTP is NOT a simple protocol. Sure you might
get a simple GET to work - but then wait till the extras of the protocl kick in, chunked transfers, etc.

byte[] receive = new byte[tcp.ReceiveBuff erSize];
int
lastreceive=str eam.Read(receiv e,0,tcp.Receive BufferSize)
; string str =
Encoding.ASCII. GetString(recei ve,0,tcp.Receiv eBufferSize
);

TCP is split into packets - what you are trying will not work. Even when you solve this - you have
solved only the tip of the iceberg.
--
Chad Z. Hower (a.k.a. Kudzu) - http://www.hower.org/Kudzu/
"Programmin g is an art form that fights back"

Empower ASP.NET with IntraWeb
http://www.atozed.com/IntraWeb/

[Chad Z. Hower aka Kudzu]

"Johann Blake" <jo*********@ya hoo.com> wrote in news:1121852438 .817575.71720
@g44g2000cwa.go oglegroups.com:

To determine whether you have read in all of the HTML, either the
remote server will close the TCP session, in which case you can check
some flag for that (can't remember which one) or if the connection is
not closed, it is implied that you have read all of the data when you
encounter an "</HTML> tag.

No - thats totally wrong. Use the HTTP protocol - not the content. HTTP can transport binary files,
XML, and other. And in HTML whitespace and other things could follow the </HTML> tag. Stopping
the transfer at that is a horrible hack that will lead to very poor results.

HTTP 1.1 for example keeps the connection open.
--
Chad Z. Hower (a.k.a. Kudzu) - http://www.hower.org/Kudzu/
"Programmin g is an art form that fights back"

Get your ASP.NET in gear with IntraWeb!
http://www.atozed.com/IntraWeb/

[Johann Blake]

There is nothing wrong with my suggestion. I've done this many times
without any problems. He wants to read HTML using a TcpClient. There's
nothing wrong with that approach. It has its advantages and
disadvantages. The thing he wasn't aware of is that HTML has nothing to
do with the actual TcpClient protocol itself. Anything can be sent with
a TcpClient.

Johann

[Chad Z. Hower aka Kudzu]

"Johann Blake" <jo*********@ya hoo.com> wrote in news:1121926001 .566551.261750
@g44g2000cwa.go oglegroups.com:

There is nothing wrong with my suggestion. I've done this many times
without any problems. He wants to read HTML using a TcpClient. There's
nothing wrong with that approach. It has its advantages and
Yes there is - he will have troubles later. HTTP is NOT the simple protocol it seems.

Its like using a search and replace to transform an XML document. It will work only in the short
term and break very quickly. Implementing 10% of the HTTP protocol is a hack - and very poor style
guaranteed to break quickly.

The proper solution is to use an HTTP component.
disadvantages. The thing he wasn't aware of is that HTML has nothing to
do with the actual TcpClient protocol itself. Anything can be sent with
a TcpClient.

http://www.indyproject.org/

Believe me, I know how TCP works.
--
Chad Z. Hower (a.k.a. Kudzu) - http://www.hower.org/Kudzu/
"Programmin g is an art form that fights back"

Make your ASP.NET applications run faster
http://www.atozed.com/IntraWeb/

[Wessel Troost]

> int lastreceive=str eam.Read(receiv e,0,tcp.Receive BufferSize);

string str =
Encoding.ASCII. GetString(recei ve,0,tcp.Receiv eBufferSize);

Call these functions recursively until stream.Read() returns 0 to indicate
end-of-stream.

Regards,
Wessel