How to avoid EOFException while using readFully() in java

[VeeraLakshmi]

I am doing a project for internet control using Java,PHP and MySql.All sites
should go through the proxy server only.We are giving access rights as allow or
deny to the sites.If we type the url,first it will ask for authentication.After
giving username and password,the authentication will be confirmed and if the
site has access right as allow,the website will open else a page will be
displayed.There we can send request mail to the administrator to give access
rights to open the site.This is the brief description of my project.
Here java is used as server(back-end) and php is used as front-end.The php files are used to store the values in the database.In the java proxy,i have specified the port,server-ip and apache-port.I am using j2sdk1.4.2_13 and Apache2 versions and working in Windows2000.

The sites which have no access rights are restricted properly.
But there is some problem in accessing sites which have access rights.
For example,rediff.com,sify.com are loading completely and msn.com,microsoft.com are loading partially and yahoo.com,gmail.com are not loading.

Refer the code mentioned below:

 import java.net.*; 
import java.io.*;
import java.util.*;
 
/* Stream reader for HTTP requests.The start line contains the whole URL (http://host/file) to retrieve. */
 
public class HTTPRequestReader extends HTTPMessageReader {
 
/* The method, eg. GET, POST, etc. */
String command;
 
/* The host concerned in the request */
String host;
 
/* The file (with full path) concerned in the request */
String file;
 
/* The whole start line */
String startLine;
 
/* The Basic authorization parameter */
String auth;
 
/* Retrieves a HTTP request message from the connection stream.*/
public HTTPRequestReader(InputStream istream) 
     throws IOException, NoSuchElementException {
    DataInputStream distream = new DataInputStream(istream);
    this.contentLength = 0;
    retrieveHeader(distream);
 
    // We've got the header, let's parse it. 
    StringTokenizer st = 
     new StringTokenizer(new String(HTTPMessageReader.toArray(header)));
    command = st.nextToken();
    URL url = new URL(st.nextToken());
    host = url.getHost();
    file = url.getFile();
    versionProtocol = st.nextToken();
 
    startLine = new String(command + " " + url + " " + versionProtocol);
 
    // Parses, within the remaining header lines, the fields we want control:
    String s;
    while (st.hasMoreTokens()) 
    {
     s = st.nextToken();
 
     if (s.equals("Content-Length:"))
         contentLength = Integer.parseInt(st.nextToken());
 
     if (s.equals("Connection:"))
     {
     connection = new String(st.nextToken());
if (connection.equals("keep-alive")) mustCloseConnection = false;
     }
 
     if (s.equals("Authorization:")) 
     {
         s = st.nextToken();
         if (s.equals("Basic")) 
         auth = new String(st.nextToken());
     }
    }
 
    retrieveBody(distream);
}
}
 
 
--------------------------------------------------------------------------------
 
 
import java.net.*;
import java.io.*;
import java.util.*;
 
 
/**
* Stream reader for HTTP responses.
*/
public class HTTPResponseReader extends HTTPMessageReader {
 
/** Status line: The status code, eg. 200, 404, etc. */
String statusCode;
 
 
/**
* Retrieves a HTTP response message from the connection stream.
* @param istream the stream
*/
public HTTPResponseReader(InputStream istream) 
     throws IOException, NoSuchElementException {
    DataInputStream distream = new DataInputStream(istream);
 
    retrieveHeader(distream);
 
    // We've got the header, let's parse it
 
    // Parses the status line
    StringTokenizer st = 
     new StringTokenizer(new String(HTTPMessageReader.toArray(header)));
 
    versionProtocol = st.nextToken();
    statusCode = st.nextToken();
 
    String s;
    while (st.hasMoreTokens()) 
    {
     s = st.nextToken();
 
if (s.equals("Content-Length:"))
     contentLength = Integer.parseInt(st.nextToken());
 
 
     if (s.equals("Connection:")) 
     {
     connection = new String(st.nextToken());
     if (connection.equals("keep-alive")) mustCloseConnection = false;
     }
    }
 
    retrieveBody(distream);    
}
 
}
 
 
 
--------------------------------------------------------------------------------
 
 
 
import java.net.*;
import java.io.*;
import java.util.*;
 
 
/**
* Stream reader for HTTP messages (requests and responses).
*/
public abstract class HTTPMessageReader {
 
/** The header of the message (blank line included) */
protected ArrayList header;
 
/** The body of the message */
protected ArrayList body;
 
/** The whole message obtained from the stream */
protected byte[] message;
 
/** Start/Status line: The version of HTTP protocol, eg. HTTP/1.1 */
String versionProtocol;
 
/** Header field: The length of message body */
protected int contentLength = -1;
 
/** Position of body in the message */
protected int offset;
 
/** Header field: Connection */
protected String connection;
 
/** Flag that informs whether the connection must be closed or not */
protected boolean mustCloseConnection = true;
 
 
/**
* Reads inside the stream until the "\r\n" couple that
* separes header from body message.
* @param distream the stream of connection
*/
protected void retrieveHeader(DataInputStream distream) throws IOException 
{
    header = new ArrayList();
    byte data;
    offset = 0;
    while (true) {
     data = (byte) distream.read();
     if (data == '\r') {
        header.add(new Byte(data));
        data = (byte) distream.read();
        if (data == '\n') {
         header.add(new Byte(data));
         data = (byte) distream.read();
         if (data == '\r') {
            header.add(new Byte(data));
            data = (byte) distream.read();
            if (data == '\n') {
             header.add(new Byte(data));
             break;
            }
         }
        }
     }
     header.add(new Byte(data));
    }
 
    if (ProxyMain.TRACEON_MSGS)
     System.out.print(new String(toArray(header)));
}
 
 
 
/**
* Receives the body of the message (if exists) from the stream.
* @param distream the stream of connection 
*/
 
protected void retrieveBody(DataInputStream distream) throws IOException {
    /* HTTP/1.0 compliant: sets a buffer enough large for the data.
     HTTP/1.0 marks the end of a message by closing the connection
     (no keep-alive), thus does not need a Content-Length header
     which is obligatory in HTTP/1.1
    */ 
 
 
    if (contentLength == -1) contentLength = 1024 *1024 ;
 
byte[] bbody = new byte[contentLength];
 
distream.readFully(bbody, 0, contentLength);
    //distream.skipBytes(contentLength);
 
body = new ArrayList();
    for (int i = 0; i < contentLength; i++) 
 
     body.add(new Byte(bbody[i]));
 
    if (ProxyMain.TRACEON_FULLMSGS)
     System.out.println(new String(toArray(body)) + "\n");
    else if (ProxyMain.TRACEON_MSGS)
     System.out.println("[" + body.size() + " bytes of data body]\n");
 
    concatenateHeaderBody();
}
 
 
/**
* Concatenates the header and the body in a whole message.
*/
private void concatenateHeaderBody() {
    message = new byte[header.size() + body.size()];
    int c = 0;
    for (int c1 = 0; c1 < header.size(); c1++)
     message[c++] = ((Byte) header.get(c1)).byteValue();
    for (int c2 = 0; c2 < body.size(); c2++)
     message[c++] = ((Byte) body.get(c2)).byteValue();
}
 
 
/**
* Converts an ArrayList of Bytes into an array of bytes.
* @param arraylist an ArrayList
* @return an array of bytes
*/ 
public static byte[] toArray(ArrayList arraylist) {
    byte[] array = new byte[arraylist.size()];
    for (int i = 0; i < arraylist.size(); i++) 
     array[i] = ((Byte) arraylist.get(i)).byteValue();
    return array;
}
 
/**
* Erases from the header the line containing a command.
* @param cmd the command that identifies the line
* @param nbrParam the number of parameter to erase after the command cmd
*/
public void eraseCommand(String cmd, int nbrParam) {
    String string; 
    byte[] bytes;
    ArrayList sal = new ArrayList();
    StringTokenizer st = 
     new StringTokenizer(new String(toArray(header)), "\n", true);
    while (st.hasMoreTokens()) {
     string = new String(st.nextToken());
     if (string.equals(cmd)) {
        for (int i=0; i<nbrParam; i++) st.nextToken();
     }
     else {
        bytes = string.getBytes();
        for (int i = 0; i < bytes.length; i++) 
         sal.add(new Byte(bytes[i]));
     }
    }
    header = sal;
 
    concatenateHeaderBody();
} 
}
 
 
--------------------------------------------------------------------------------
 
 

If the sites have Content-Length,there is no problem in accessing the sites.ie.,the readFully method reads from the file completely upto the
Content-Length.

But if the sites don't have Content-Length,it wil become -1 andwe set some default value as 1024*1024 for that.At that time,the readFully method cannot read from the file upto 1024*1024 bytes.It reaches the end of the file before reading completely,ie,an EOFException is thrown.So the sites are not loading or partially loading.

To avoid this problem,we used a similar method called skipBytes.If we use
that,the sites which were accessed for example,rediff.com,sify.com itselves are
not loading.

Then we made some changes in the code to read until EOF on an InputStream.But the response was the site was on loading and the page was not opened.

I tried my level best to solve this problem.But i cant find a solution.

Can anybody help me to find a solution.

I will be thankful to you.

By,
T.VeeraLakshmi