Socket returns EOF prematurely

[Nathan Davis]

Hi,

I am having problems reading data from a socket. The problem only
occurs while trying to read from one particular server, and does not
occur on Windows (as far as I know, this may occur only on Linux).

The server is free.teranews.com, and I am trying to connect to the NNTP
port (119). To make things even more bizar, I only have trouble when I
try to issue the LIST command. After issuing the command, Java will not
return any more data, and will behave as if EOF was reached on the socket.

The server *is* returning data. On windows (reading with Java), the
data is returned, but there are a bunch of NULs in the output. I have
not been able to find any strange characters returned by the server
using a logging tcp proxy called ProxyView, however, so that might be a
bug in the Windows version, or possibly a bug on the part of the test
program I wrote.

At any rate, the server interoperates as expected with every news reader
I've tried, including Netscape and KNode, so I am a bit perplexed why
Java on Linux is have trouble reading from the socket. Reproduction
should be easy: just open a socket to free.teranews.net on port 119,
write "list", and read all data until EOF. You should get a list of
newsgroups if it work, or, if it doesn't, you'll get the Welcome message
and then EOF.

Has anyone seen this behaviour before. Any workarounds? I've tried
this with both JDK1.4 and JDK1.5 (beta), and neither work. If you do
get it to work, would you please post the details of your setup --
version of Linux, JDK, etc.?

Thanks,

--Nathan Davis

[Gordon Beaton]

On Fri, 11 Jun 2004 01:03:45 GMT, Nathan Davis wrote:

I am having problems reading data from a socket. The problem only
occurs while trying to read from one particular server, and does not
occur on Windows (as far as I know, this may occur only on Linux).

The server is free.teranews.com, and I am trying to connect to the
NNTP port (119). To make things even more bizar, I only have trouble
when I try to issue the LIST command. After issuing the command,
Java will not return any more data, and will behave as if EOF was
reached on the socket.

From the failing machine, try connecting to the server with telnet:

$ telnet free.teranews.com 119

Now issue the list command.

Does it work? If not, this isn't a java issue.

If so, there is likely a problem with the code you forgot to post.

I'm guessing there is a problem with the mechanism you use to read
from the server. More specifically I suspect that you are using
InputStream.available() or InputStreamReader.ready() in your read
loop. If that's the case: don't.

/gordon

--
[ do not email me copies of your followups ]
g o r d o n + n e w s @ b a l d e r 1 3 . s e

[Nathan Davis]

Thanks for the response. Yes, telnet'ing to the server and issuing the
list command works. As far as the code goes, no I'm not using
available() or ready(). In a nutshell, here's a version of the code
that fails:

int c = input.read();
while(c != -1){ //read until stream returns "EOF"
c = input.read();
}

Thanks,

--Nathan Davis

Gordon Beaton wrote:

On Fri, 11 Jun 2004 01:03:45 GMT, Nathan Davis wrote:

I am having problems reading data from a socket. The problem only
occurs while trying to read from one particular server, and does not
occur on Windows (as far as I know, this may occur only on Linux).

The server is free.teranews.com, and I am trying to connect to the
NNTP port (119). To make things even more bizar, I only have trouble
when I try to issue the LIST command. After issuing the command,
Java will not return any more data, and will behave as if EOF was
reached on the socket.

From the failing machine, try connecting to the server with telnet:

$ telnet free.teranews.com 119

Now issue the list command.

Does it work? If not, this isn't a java issue.

If so, there is likely a problem with the code you forgot to post.

I'm guessing there is a problem with the mechanism you use to read
from the server. More specifically I suspect that you are using
InputStream.available() or InputStreamReader.ready() in your read
loop. If that's the case: don't.

/gordon

[Nathan Davis]

Ok, I've figured this one out. It turns out that the problem is not
with how I *read* from the server, but how I *write* to it. I was using
PrintStream.println(), which only appends a '\n' character to the
string. Apparently, the server was expecting '\r\n', and was closing
the connection when only a '\n' was recieved! I believe the different
behaviour seen under windows in due to the fact that windows uses '\r\n'
as the newline seperator, whereas unix uses only '\n'. Therefore, I
believe you would see the same behaviour with any unix-like system.

Wow! What a lot of head-banging this turned out to be!

--Nathan Davis

Gordon Beaton wrote:

On Fri, 11 Jun 2004 01:03:45 GMT, Nathan Davis wrote:

I am having problems reading data from a socket. The problem only
occurs while trying to read from one particular server, and does not
occur on Windows (as far as I know, this may occur only on Linux).

The server is free.teranews.com, and I am trying to connect to the
NNTP port (119). To make things even more bizar, I only have trouble
when I try to issue the LIST command. After issuing the command,
Java will not return any more data, and will behave as if EOF was
reached on the socket.

From the failing machine, try connecting to the server with telnet:

$ telnet free.teranews.com 119

Now issue the list command.

Does it work? If not, this isn't a java issue.

If so, there is likely a problem with the code you forgot to post.

I'm guessing there is a problem with the mechanism you use to read
from the server. More specifically I suspect that you are using
InputStream.available() or InputStreamReader.ready() in your read
loop. If that's the case: don't.

/gordon

[Dave]

Congrats, you've reimplimented FTP. Yeah, yeah, I just couldn't resist.

Nathan Davis wrote:

Ok, I've figured this one out. It turns out that the problem is not
with how I *read* from the server, but how I *write* to it. I was using
PrintStream.println(), which only appends a '\n' character to the
string. Apparently, the server was expecting '\r\n', and was closing
the connection when only a '\n' was recieved! I believe the different
behaviour seen under windows in due to the fact that windows uses '\r\n'
as the newline seperator, whereas unix uses only '\n'. Therefore, I
believe you would see the same behaviour with any unix-like system.

Wow! What a lot of head-banging this turned out to be!

--Nathan Davis

Gordon Beaton wrote:

On Fri, 11 Jun 2004 01:03:45 GMT, Nathan Davis wrote:

I am having problems reading data from a socket. The problem only
occurs while trying to read from one particular server, and does not
occur on Windows (as far as I know, this may occur only on Linux).

The server is free.teranews.com, and I am trying to connect to the
NNTP port (119). To make things even more bizar, I only have trouble
when I try to issue the LIST command. After issuing the command,
Java will not return any more data, and will behave as if EOF was
reached on the socket.

From the failing machine, try connecting to the server with telnet:

$ telnet free.teranews.com 119

Now issue the list command.

Does it work? If not, this isn't a java issue.

If so, there is likely a problem with the code you forgot to post.

I'm guessing there is a problem with the mechanism you use to read
from the server. More specifically I suspect that you are using
InputStream.available() or InputStreamReader.ready() in your read
loop. If that's the case: don't.

/gordon

[Gordon Beaton]

On Fri, 11 Jun 2004 21:18:09 GMT, Nathan Davis wrote:

Ok, I've figured this one out. It turns out that the problem is not
with how I *read* from the server, but how I *write* to it. I was using
PrintStream.println(), which only appends a '\n' character to the
string. Apparently, the server was expecting '\r\n', and was closing
the connection when only a '\n' was recieved! I believe the different
behaviour seen under windows in due to the fact that windows uses '\r\n'
as the newline seperator, whereas unix uses only '\n'. Therefore, I
believe you would see the same behaviour with any unix-like system.

Of course if you're implementing a standard protocol it's always a
good idea to read the relevant RFCs first. Most (possible all) text
based protocols define EOL as a CR LF pair. From rfc977 (NNTP):

2.3. Commands

[...]

Each command line must be terminated by a CR-LF (Carriage Return -
Line Feed) pair.

/gordon

--
[ do not email me copies of your followups ]
g o r d o n + n e w s @ b a l d e r 1 3 . s e

[perry]

yes, i have seen this type of thing before. the problem is not with
linux, it has to do with a more efficient socket implementation on linux.

i strongly suspect your EOF is as a result of your connection is being
made too fast and either the client or the server is either not ready or
has given up.

to resolve this problem i had to implement a more intelligent connection
scheme where i would try 3 times, pausing 250 ms before each attempt...

it always seemed to get it either on the 1st or 2nd attempt, i never had
any problems after that.

i'll try to dig up the code and post it for you later

- perry

Nathan Davis wrote:

Hi,

I am having problems reading data from a socket. The problem only
occurs while trying to read from one particular server, and does not
occur on Windows (as far as I know, this may occur only on Linux).

The server is free.teranews.com, and I am trying to connect to the NNTP
port (119). To make things even more bizar, I only have trouble when I
try to issue the LIST command. After issuing the command, Java will not
return any more data, and will behave as if EOF was reached on the socket.

The server *is* returning data. On windows (reading with Java), the
data is returned, but there are a bunch of NULs in the output. I have
not been able to find any strange characters returned by the server
using a logging tcp proxy called ProxyView, however, so that might be a
bug in the Windows version, or possibly a bug on the part of the test
program I wrote.

At any rate, the server interoperates as expected with every news reader
I've tried, including Netscape and KNode, so I am a bit perplexed why
Java on Linux is have trouble reading from the socket. Reproduction
should be easy: just open a socket to free.teranews.net on port 119,
write "list", and read all data until EOF. You should get a list of
newsgroups if it work, or, if it doesn't, you'll get the Welcome message
and then EOF.

Has anyone seen this behaviour before. Any workarounds? I've tried
this with both JDK1.4 and JDK1.5 (beta), and neither work. If you do
get it to work, would you please post the details of your setup --
version of Linux, JDK, etc.?

Thanks,

--Nathan Davis

[perry]

i did that one before, most aggravating... in my case i was sending a
"\n\r" rather than a "\r\n", the "\n\r" seemed more logical to me...

took all day and someone looking over my shoulder at the last minute to
get it

- perry

Nathan Davis wrote:

Ok, I've figured this one out. It turns out that the problem is not
with how I *read* from the server, but how I *write* to it. I was using
PrintStream.println(), which only appends a '\n' character to the
string. Apparently, the server was expecting '\r\n', and was closing
the connection when only a '\n' was recieved! I believe the different
behaviour seen under windows in due to the fact that windows uses '\r\n'
as the newline seperator, whereas unix uses only '\n'. Therefore, I
believe you would see the same behaviour with any unix-like system.

Wow! What a lot of head-banging this turned out to be!

--Nathan Davis

Gordon Beaton wrote:

On Fri, 11 Jun 2004 01:03:45 GMT, Nathan Davis wrote:

I am having problems reading data from a socket. The problem only
occurs while trying to read from one particular server, and does not
occur on Windows (as far as I know, this may occur only on Linux).

The server is free.teranews.com, and I am trying to connect to the
NNTP port (119). To make things even more bizar, I only have trouble
when I try to issue the LIST command. After issuing the command,
Java will not return any more data, and will behave as if EOF was
reached on the socket.

From the failing machine, try connecting to the server with telnet:

$ telnet free.teranews.com 119

Now issue the list command.

Does it work? If not, this isn't a java issue.

If so, there is likely a problem with the code you forgot to post.

I'm guessing there is a problem with the mechanism you use to read
from the server. More specifically I suspect that you are using
InputStream.available() or InputStreamReader.ready() in your read
loop. If that's the case: don't.

/gordon