Handling Blocks of Bytes

[KWSW]

Just a question on handling "Byte[]" in java.

I have to read in a file of any type and chop it up into blocks for processing later on.

Was wondering if I got do a Byte[][] to keep the Byte[] in an array or would something else be a better way of doing it as I do now know how many elements the array of Byte[] can hold as the file might be huge and the blocks might be small.

Thanks. :)

[JosAH]

Just a question on handling "Byte[]" in java.

I have to read in a file of any type and chop it up into blocks for processing later on.

Was wondering if I got do a Byte[][] to keep the Byte[] in an array or would something else be a better way of doing it as I do now know how many elements the array of Byte[] can hold as the file might be huge and the blocks might be small.

Thanks. :)

Why don't you use an ArrayList<byte[]> for that? Simply stack up all the byte[]s
you have read in the ArrayList.

kind regards,

Jos

[KWSW]

Why don't you use an ArrayList<byte[]> for that? Simply stack up all the byte[]s
you have read in the ArrayList.

kind regards,

Jos

ok thanks will give it a try later :)

[KWSW]

oh btw... just to confirm, when i read in the whole file at the first go before chopping them up into blocks, Byte[19] would refer to the 20th byte in the file right?

[JosAH]

oh btw... just to confirm, when i read in the whole file at the first go before chopping them up into blocks, Byte[19] would refer to the 20th byte in the file right?

Yup, Java (just like C and C++) starts counting from 0, so byte[0] is the first
byte in the array, byte[1] is the second etc. etc. ArrayLists count the same.

kind regards,

Jos

[KWSW]

got another qn though... i would need an integer to define the size of the byte[] array and am worried that if the file is too big, the integer data type cant fit it.

Wanted to use long but the byte[] can only take in integer. Is there a way around it or the integer data type should be big enough.

I would assume a 200~300 mb file would be the biggest I ever have to handle.

[JosAH]

got another qn though... i would need an integer to define the size of the byte[] array and am worried that if the file is too big, the integer data type cant fit it.

Wanted to use long but the byte[] can only take in integer. Is there a way around it or the integer data type should be big enough.

I would assume a 200~300 mb file would be the biggest I ever have to handle.

I thought you were going to read that file in chunks? Each chunk in a byte[].

kind regards,

Jos

[r035198x]

got another qn though... i would need an integer to define the size of the byte[] array and am worried that if the file is too big, the integer data type cant fit it.

Wanted to use long but the byte[] can only take in integer. Is there a way around it or the integer data type should be big enough.

I would assume a 200~300 mb file would be the biggest I ever have to handle.

Any reason why you are using an array instead of an ArrayList?

[KWSW]

Any reason why you are using an array instead of an ArrayList?

oh ok... haha... got a bit confused just now...

ok so lets say i have a file size of 10000 bytes and a block size of 100 bytes.

so i would have 100 byte[100] in the arraylist.

[r035198x]

oh ok... haha... got a bit confused just now...

ok so lets say i have a file size of 10000 bytes and a block size of 100 bytes.

so i would have 100 byte[100] in the arraylist.

Now you've got it.
Refer to Jos' first reply.

[KWSW]

Now you've got it.
Refer to Jos' first reply.

but now i cant seem to write properly... lol... i check the output file and its the same size as the input file...

my codes:

ArrayList<byte[]> al = new ArrayList<byte[]>();
byte[] b = new byte[bSize]; 
 
FileInputStream fIn = new FileInputStream(inFile);
BufferedInputStream in = new BufferedInputStream(fIn);                                                                   
 
 
while(in.read(b) != -1)
{
    al.add(b);
}            
 
in.close();
 
FileOutputStream fOut = new FileOutputStream(outFile);
BufferedOutputStream out = new BufferedOutputStream(fOut);                                                                   
 
 
for(int i = 0; i < al.size(); i++)               
{
    out.write(al.get(i));
}
 
out.close();
 

[JosAH]

You're appending the same byte[] b to the array list over and over again. Of course
the content of that array changes after every read.

kind regards,

Jos

[KWSW]

You're appending the same byte[] b to the array list over and over again. Of course
the content of that array changes after every read.

kind regards,

Jos

it doesn't read the next set of bytes automatically?

[r035198x]

but now i cant seem to write properly... lol... i check the output file and its the same size as the input file...

my codes:

ArrayList<byte[]> al = new ArrayList<byte[]>();
byte[] b = new byte[bSize]; 
 
FileInputStream fIn = new FileInputStream(inFile);
BufferedInputStream in = new BufferedInputStream(fIn);                                                                   
 
 
while(in.read(b) != -1)
{
    al.add(b);
}            
 
in.close();
 
FileOutputStream fOut = new FileOutputStream(outFile);
BufferedOutputStream out = new BufferedOutputStream(fOut);                                                                   
 
 
for(int i = 0; i < al.size(); i++)               
{
    out.write(al.get(i));
}
 
out.close();
 

Why do you expect the input file to have a different size from the source file? You seem to be reading one file into an arraylist and writing the same list's contents to the output file.

[KWSW]

Why do you expect the input file to have a different size from the source file? You seem to be reading one file into an arraylist and writing the same list's contents to the output file.

what i need to do is to read in a file, chop it up into blocks of a certain size, do some checking to the blocks and write them out if they pass the checks.

What I am trying to do now is to make sure that what i read in as blocks are written out properly so the file is still the same.

[r035198x]

but now i cant seem to write properly... lol... i check the output file and its the same size as the input file...

.....
[/code]

I thought you were saying that there's a problem because the files are the same.
So you don't have a problem right now?

[KWSW]

I thought you were saying that there's a problem because the files are the same.
So you don't have a problem right now?

i have a problem becos while the file size is the same, the file isnt...

I solved the part about the last byte[] being smaller than the block size so my files are the same size now. But they are not the same file. As in if its a picture file, the output will be corrupted. But it works fine with text

[KWSW]

ok one of those weird days... now its works... -_-'''

anyway thanks again for the help :)

[r035198x]

i have a problem becos while the file size is the same, the file isnt...

I solved the part about the last byte[] being smaller than the block size so my files are the same size now. But they are not the same file. As in if its a picture file, the output will be corrupted. But it works fine with text

See the specs of the read method here.