Buffered Reader

[jyohere]

I am using buffered Reader.....some problem arises ...dnt know why....am using the following code to read a big file about 8MB.

path = "/home/t.txt"

File file=new File(path);
FileReader freader=new FileReader(file);
BufferedReader reader=new BufferedReader(freader);

System.out.println(reader.ready());
while((line = reader.readLine()) != null)
{
//some code
}

in the above case it sometimes gives reader.ready() is false....and so it does not enter the while loop wat could be the fix....and why its giving reader.ready() is false
reason and wat is the fix for it...

[JosAH]

I am using buffered Reader.....some problem arises ...dnt know why....am using the following code to read a big file about 8MB.

path = "/home/t.txt"

File file=new File(path);
FileReader freader=new FileReader(file);
BufferedReader reader=new BufferedReader(freader);

System.out.println(reader.ready());
while((line = reader.readLine()) != null)
{
//some code
}

in the above case it sometimes gives reader.ready() is false....and so it does not enter the while loop wat could be the fix....and why its giving reader.ready() is false
reason and wat is the fix for it...

No matter the value of ready(), your code enters that while loop as far as I can see.
Did you show us the code you were running?

I ripped the following straight from the API docs for the BufferedReader class:

Tells whether this stream is ready to be read. A buffered character stream is ready if the buffer is not empty, or if the underlying character stream is ready.

When the reader is freshly opened, it may not have filled its buffer yet; that could
be a reason for ready() to return false. Don't pay any attention to it and just
read away.

kind regards,

Jos

[jyohere]

No matter the value of ready(), your code enters that while loop as far as I can see.
Did you show us the code you were running?

I ripped the following straight from the API docs for the BufferedReader class:



When the reader is freshly opened, it may not have filled its buffer yet; that could
be a reason for ready() to return false. Don't pay any attention to it and just
read away.

kind regards,

Jos

This is the same code am using.....i printed reader.ready() only after it was not entering the loop... i am using it in my project now....i got the error when it was
tested for stress test....it was working fine when the file size was small...

[jyohere]

This is the same code am using.....i printed reader.ready() only after it was not entering the loop... i am using it in my project now....i got the error when it was
tested for stress test....it was working fine when the file size was small...

The code i said above is in a function....it will get called and then the while loop has to execute immediately the first time itself....

[JosAH]

This is the same code am using.....i printed reader.ready() only after it was not entering the loop... i am using it in my project now....i got the error when it was
tested for stress test....it was working fine when the file size was small...

I'm afraid the error is somewhere else then; check if the file exists; if so read it
and see if the while loop *terminates* properly. Maybe an unchecked exception
what thrown somewhere in the body of that while loop.

If you wrap a BufferedReader around a FileReader, the ready() method doesn't
tell you much.

kind regards,

Jos

[jyohere]

I'm afraid the error is somewhere else then; check if the file exists; if so read it
and see if the while loop *terminates* properly. Maybe an unchecked exception
what thrown somewhere in the body of that while loop.

If you wrap a BufferedReader around a FileReader, the ready() method doesn't
tell you much.

kind regards,

Jos

then why is it not entering the loop only when reader.ready() is false?

[JosAH]

then why is it not entering the loop only when reader.ready() is false?

I don't know because you didn't show us any relevant code that showed the
symptoms you keep on mentioning.

kind regards,

Jos

[jyohere]

I don't know because you didn't show us any relevant code that showed the
symptoms you keep on mentioning.

kind regards,

Jos

this is the method.....sry its too big Jos

public static ArrayList get(int ne_id,int ne_type,

int shelf_id, int card_id, int port_id,

int index1_id, int index2_id,

int index3_id,int index4_id, int[] location,

NameAndStringValue_T[] name,String[] granularity,

String[] pmLocation, short[] layerRate,

String startTime, String endTime){



String[] pmParameters={"PMP_BBE", "PMP_SES", "PMP_ES", "PMP_UAS"};

int index=0;

logger.info("Inside get()");



BufferedReader reader=null;

String line=" ";

ArrayList keyList=new ArrayList();

ArrayList valueList=new ArrayList();

String str[]=null;

String syearStr=" ", eyearStr=" ";

String smonthStr=" ", emonthStr=" ";

String sdateStr=" ", edateStr=" ";

int startDate=0, endDate=0;

StringBuffer fileName=new StringBuffer();

StringBuffer endfileName=new StringBuffer();

String path=" ";

Calendar start_cal,end_cal;

int start_week_of_year=0,end_week_of_year=0;

boolean near=false;



SimpleDateFormat dateFormat=new SimpleDateFormat("yyyyMMddHHmmss");



String sTime=startTime.substring(0,14);

String eTime=endTime.substring(0,14);



ParsePosition p1=new ParsePosition(0);

Date start=dateFormat.parse(sTime,p1);

ParsePosition p2=new ParsePosition(0);

Date end=dateFormat.parse(eTime,p2);



logger.info(HDR + " START TIME : " + start);

logger.info(HDR + " END TIME : " + end);

try{

Matcher m;

if(granularity[0].equalsIgnoreCase("15min")){

syearStr=startTime.substring(0,4);



fileName.append(syearStr);

fileName.append("_");

smonthStr=startTime.substring(4,6);

fileName.append(smonthStr);

fileName.append("_");

sdateStr=startTime.substring(6,8);

fileName.append(sdateStr);

fileName.append("_");

fileName.append("15M");

fileName.append("_");

if(ne_type<=9){

fileName.append("0");

fileName.append(ne_type);

}else{

fileName.append(ne_type);

}

fileName.append(".ppm");



eyearStr=endTime.substring(0,4);

emonthStr=endTime.substring(4,6);

edateStr=endTime.substring(6,8);



startDate=Integer.parseInt(sdateStr);

endDate=Integer.parseInt(edateStr);



}else if(granularity[0].equalsIgnoreCase("24H")){



syearStr=startTime.substring(0,4);

int start_year=Integer.parseInt(syearStr);



fileName.append(syearStr);

fileName.append("_");



smonthStr=startTime.substring(4,6);

int start_month=Integer.parseInt(smonthStr)-1;

sdateStr=startTime.substring(6,8);

int start_date=Integer.parseInt(sdateStr);



start_cal=new GregorianCalendar(start_year,start_month,start_dat e);

start_cal.setMinimalDaysInFirstWeek(7);

start_cal.setFirstDayOfWeek(Calendar.MONDAY);



if ((start_cal.get(Calendar.MONTH) == Calendar.JANUARY) && (start_cal.get(Calendar.WEEK_OF_YEAR) >= 52))

start_week_of_year=0;

else

start_week_of_year=start_cal.get(Calendar.WEEK_OF_ YEAR);



logger.info(HDR+"start_week_of_year " +start_week_of_year);



if(start_week_of_year <=9){

fileName.append("0");

fileName.append(start_week_of_year);

}else{

fileName.append(start_week_of_year);

}

fileName.append("_");

fileName.append("24H");

fileName.append("_");

if(ne_type<=9){

fileName.append("0");

fileName.append(ne_type);

}else{

fileName.append(ne_type);

}

fileName.append(".ppm");



eyearStr=endTime.substring(0,4);

int end_year=Integer.parseInt(eyearStr);



emonthStr=endTime.substring(4,6);

int end_month=Integer.parseInt(emonthStr)-1;

edateStr=endTime.substring(6,8);

int end_date=Integer.parseInt(edateStr);



end_cal=new GregorianCalendar(end_year,end_month,end_date);

end_cal.setMinimalDaysInFirstWeek(7);

end_cal.setFirstDayOfWeek(Calendar.MONDAY);



if ((end_cal.get(Calendar.MONTH) == Calendar.JANUARY) && (end_cal.get(Calendar.WEEK_OF_YEAR) >= 52))

end_week_of_year=0;

else

end_week_of_year=end_cal.get(Calendar.WEEK_OF_YEAR );



logger.info(HDR+"end_week_of_year " +end_week_of_year);



}







path = "/var/ftp/PMFiles/marconi/" + fileName;



logger.info(HDR+"path"+path);



File file=new File(path);

reader=new BufferedReader(new FileReader(file));

logger.info(HDR+"Before while loop");

while((line = reader.readLine()) != null)

{



logger.info(HDR+"Inside while loop");

String[] testStr=line.split(REGEX1);



try{

int test=Integer.parseInt(testStr[testStr.length-1]);

}catch(Exception e){





String newLine= reader.readLine();

StringBuffer buffer=new StringBuffer(line);

buffer.append(newLine);

line=buffer.toString();



}



m=pattern.matcher(line);



boolean matchFound = m.find();

if (matchFound){

str=new String[m.groupCount()+1];



for (int i=0; i<=m.groupCount(); i++)

str[i]=m.group(i);

}



int neId=Integer.parseInt(str[2]);

//logger.info(HDR+"neId " +neId);

int neType=Integer.parseInt(str[3]);

//logger.info(HDR+"neType " +neType);

int shelfId=Integer.parseInt(str[4]);

int cardId=Integer.parseInt(str[6]);

int portId=Integer.parseInt(str[9]);

//logger.info(HDR+"ptp" + shelfId+ " " +cardId +" "+ portId);

int indexId1=Integer.parseInt(str[10]);

int indexId2=Integer.parseInt(str[11]);

int indexId3=Integer.parseInt(str[12]);

int indexId4=Integer.parseInt(str[13]);

//logger.info(HDR+"ctp" + indexId1+ " " +indexId2 +" "+ indexId3+" "+indexId4);

int entityType=Integer.parseInt(str[14]);

//logger.info(HDR+"entity " +entityType);



ParsePosition p=new ParsePosition(0);

Date time=dateFormat.parse(str[18],p);

//logger.info(HDR+"time " +time);



if(((entityType==location[0]||entityType==location[1]))

&& (time.after(start) ||time.equals(start))

&&(time.before(end) ||time.equals(end))

&& neId==ne_id && neType==ne_type && shelfId==shelf_id

&& indexId1==index1_id && indexId2==index2_id && indexId3==index3_id && indexId4==index4_id

&& cardId==card_id && portId ==port_id){





logger.info("Matched so inside data collection...");

StringBuffer key=new StringBuffer();



for(int l=1;l<13;l++){



key.append(str[l]);

key.append(":");

}



key.append(str[18]);

key.append(".0");

String keyStr=key.toString();



StringBuffer value=new StringBuffer();



for(int k=0;k<4;k++){

value.append(name[k].value);

value.append(",");

}



value.append(granularity[0]);

value.append(",");



value.append(layerRate[0]);

value.append(",");



value.append(str[18]);

value.append(".0");

value.append(",");



value.append(str[14]);

value.append(":");

if (entityType==location[0]){

value.append(pmLocation[0]);

near=true;

}else if (entityType==location[1]){

value.append(pmLocation[1]);

near=false;

}

value.append(",");



String pmMeasurement_1=null;

String pmMeasurement_2=null;

String result=null;



if(!keyList.contains(keyStr)){

pmMeasurement_1=getPMMeasurement(pmParameters,str, true,near,pmLocation,location);

value.append(pmMeasurement_1);



result=value.toString();



}else{

pmMeasurement_2=getPMMeasurement(pmParameters,str, false,near,pmLocation,location);

}



if(!keyList.contains(keyStr)){



keyList.add(index,keyStr);

valueList.add(index,result);

index++;

}else{

int position=keyList.indexOf(keyStr);

String temp=(String)valueList.get(position);

String[] tempArray=temp.split(",");

valueList.remove(position);

StringBuffer tempBuffer=new StringBuffer();

for(int i=0;i<(tempArray.length-4);i++){

tempBuffer.append(tempArray[i]);

tempBuffer.append(",");

}



tempBuffer.append(pmMeasurement_2);

valueList.add(position, tempBuffer.toString());



}







}//if

}//end while





if((startDate != endDate) || (start_week_of_year != end_week_of_year)){



logger.info(HDR +"Start and End Not Equal");



if(granularity[0].equalsIgnoreCase("15min")){



endfileName.append(eyearStr);

endfileName.append("_");

endfileName.append(emonthStr);

endfileName.append("_");

endfileName.append(edateStr);

endfileName.append("_");

endfileName.append("15M");

endfileName.append("_");

if(ne_type<=9){

endfileName.append("0");

endfileName.append(ne_type);

}else{

endfileName.append(ne_type);

}

endfileName.append(".ppm");



}else if(granularity[0].equalsIgnoreCase("24H")){



endfileName.append(eyearStr);

endfileName.append("_");

if(end_week_of_year <=9){

endfileName.append("0");

endfileName.append(end_week_of_year);

}else{

endfileName.append(end_week_of_year);

}



endfileName.append("_");

endfileName.append("24H");

endfileName.append("_");

if(ne_type<=9){

endfileName.append("0");

endfileName.append(ne_type);

}else{

endfileName.append(ne_type);

}

endfileName.append(".ppm");

}



logger.info(HDR+"endfileName "+endfileName);



path = "/var/ftp/PMFiles/marconi/" + endfileName;



logger.info(HDR+"path "+path);



File endfile=new File(path);

reader=new BufferedReader(new FileReader(endfile));

while((line = reader.readLine()) != null)

{



String[] testStr=line.split(REGEX1);



try{

int test=Integer.parseInt(testStr[testStr.length-1]);

}catch(Exception e){



String newLine= reader.readLine();



StringBuffer buffer=new StringBuffer(line);

buffer.append(newLine);

line=buffer.toString();



}





m=pattern.matcher(line);



boolean matchFound = m.find();

if (matchFound){

str=new String[m.groupCount()+1];

for (int i=0; i<=m.groupCount(); i++)

str[i]=m.group(i);

}



int neId=Integer.parseInt(str[2]);

int neType=Integer.parseInt(str[3]);

int shelfId=Integer.parseInt(str[4]);

int cardId=Integer.parseInt(str[6]);

int portId=Integer.parseInt(str[9]);

int indexId1=Integer.parseInt(str[10]);

int indexId2=Integer.parseInt(str[11]);

int indexId3=Integer.parseInt(str[12]);

int indexId4=Integer.parseInt(str[13]);

int entityType=Integer.parseInt(str[14]);

ParsePosition p=new ParsePosition(0);

Date time=dateFormat.parse(str[18],p);



if(((entityType==location[0]||entityType==location[1]))

&&(time.before(end) ||time.equals(end))

&& neId==ne_id && neType==ne_type && shelfId==shelf_id

&& indexId1==index1_id && indexId2==index2_id && indexId3==index3_id && indexId4==index4_id

&& cardId==card_id && portId ==port_id){





StringBuffer key=new StringBuffer();



for(int l=1;l<13;l++){



key.append(str[l]);

key.append(":");

}



key.append(str[18]);

key.append(".0");



String keyStr=key.toString();



StringBuffer value=new StringBuffer();



for(int k=0;k<4;k++){

value.append(name[k].value);

value.append(",");

}



value.append(granularity[0]);

value.append(",");



value.append(layerRate[0]);

value.append(",");



value.append(str[18]);

value.append(".0");

value.append(",");



value.append(str[14]);

value.append(":");

if (entityType==location[0]){

value.append(pmLocation[0]);

near=true;

}else if (entityType==location[1]){

value.append(pmLocation[1]);

near=false;

}

value.append(",");



String pmMeasurement_1=null;

String pmMeasurement_2=null;

String result=null;



if(!keyList.contains(keyStr)){

pmMeasurement_1=getPMMeasurement(pmParameters,str, true,near,pmLocation,location);

value.append(pmMeasurement_1);



result=value.toString();



}else{

pmMeasurement_2=getPMMeasurement(pmParameters,str, false,near,pmLocation,location);

}



if(!keyList.contains(keyStr)){



keyList.add(index,keyStr);

valueList.add(index,result);

index++;

}else{

int position=keyList.indexOf(keyStr);

String temp=(String)valueList.get(position);

String[] tempArray=temp.split(",");

valueList.remove(position);

StringBuffer tempBuffer=new StringBuffer();

for(int i=0;i<(tempArray.length-4);i++){

tempBuffer.append(tempArray[i]);

tempBuffer.append(",");

}

tempBuffer.append(pmMeasurement_2);

valueList.add(position, tempBuffer.toString());



}



}

}



}



}catch(FileNotFoundException e){

logger.warn(HDR+e.getMessage());

logger.warn("", e);

}catch(IndexOutOfBoundsException e){

logger.warn(HDR+e.getMessage());

logger.warn("", e);

}catch(PatternSyntaxException e){

logger.warn(HDR+e.getMessage());

logger.warn("", e);

}catch(IOException e){

logger.warn(HDR+e.getMessage());

logger.warn("", e);

}catch(NullPointerException e){

logger.warn(HDR+e.getMessage());

logger.warn("", e);

}catch(Exception e){

logger.warn(HDR+e.getMessage());

logger.warn("", e);

}finally{

try{

keyList=null;

if (reader!=null)

reader.close();

}catch(IOException e){

logger.warn(HDR+e.getMessage());

logger.warn("", e);

}

}



for(int k=0;k<valueList.size();k++)

logger.info("Final valueList Returned is " +valueList.get(k));

logger.info(HDR+"Returned valueList ");

return valueList;



}

[JosAH]

this is the method.....sry its too big Jos

It sure is ;-) What ends up in the log output? Does the log output flush itself
after a single line is written to it?

kind regards,

Jos

[jyohere]

It sure is ;-) What ends up in the log output? Does the log output flush itself
after a single line is written to it?

kind regards,

Jos

The while loop is not executing only sometimes....it does not enter inside....when i
type reader.ready()....it gives false before the loop....wat could the problem ....can it be because of too much load in the server.....one more thing is that for every 15 minutes the file gets overwritten.....but if the file is not there it should have thrown file not found exception isn't it....pls help....have been spending time on this for quite a long time....give me the possible chances of error...

[JosAH]

The while loop is not executing only sometimes....it does not enter inside....when i
type reader.ready()....it gives false before the loop....wat could the problem ....can it be because of too much load in the server.....one more thing is that for every 15 minutes the file gets overwritten.....but if the file is not there it should have thrown file not found exception isn't it....pls help....have been spending time on this for quite a long time....give me the possible chances of error...

So another (external) process is fiddling with that file. I'm beginning to understand
while that entire while loop is skipped: suppose you open the file successfully,
then the file content is changed and your process can't read the (previous)
content anymore because it hadn't started reading yet.

Either make that other process lock the overwriting process; by using an (empty)
lock file on which both processes negoatiate or (this is hacking) make your
process try to reopen the entire thing if opening the file succeeded but the first
read did not.

Your two processes should use a locking mechanism though otherwise you're
asking for trouble (which you already experienced ;-)

kind regards,

Jos

[jyohere]

So another (external) process is fiddling with that file. I'm beginning to understand
while that entire while loop is skipped: suppose you open the file successfully,
then the file content is changed and your process can't read the (previous)
content anymore because it hadn't started reading yet.

Either make that other process lock the overwriting process; by using an (empty)
lock file on which both processes negoatiate or (this is hacking) make your
process try to reopen the entire thing if opening the file succeeded but the first
read did not.

Your two processes should use a locking mechanism though otherwise you're
asking for trouble (which you already experienced ;-)

kind regards,

Jos

the other process is a crontab entry in linux which will fetch the file from an other system and overwrite it.....i am asking about the load because when the file is of small size in the beginning it is working fine...but when the file size increases to say 3mb it is behaving differently....hope its clear for u now..

[JosAH]

the other process is a crontab entry in linux which will fetch the file from an other system and overwrite it.....i am asking about the load because when the file is of small size in the beginning it is working fine...but when the file size increases to say 3mb it is behaving differently....hope its clear for u now..

If the other process doesn't want to play that locking game all bets are off. You
could check (before you start reading) whether or not the file size is changing.
If so, don't read because the other process is writing. If it doesn't change, hope
for the best and start reading.

kind regards,

Jos

[jyohere]

If the other process doesn't want to play that locking game all bets are off. You
could check (before you start reading) whether or not the file size is changing.
If so, don't read because the other process is writing. If it doesn't change, hope
for the best and start reading.

kind regards,

Jos

I am testing the file putting it in a separate place.....let me try my luck

[JosAH]

I am testing the file putting it in a separate place.....let me try my luck

I wish you all the best of luck.

kind regards,

Jos

[jyohere]

thanks jos....i think u are right.....need to test more....wl let u know