467,118 Members | 993 Online
Bytes | Developer Community
Ask Question

Home New Posts Topics Members FAQ

Post your question to a community of 467,118 developers. It's quick & easy.

Reading Binary Image Data

Hey Guys.
I'm looking for some info on how to read binary image data. Basicly RAW image files are strings of bits with say 2 bytes per pixel. What i need to do is read the value of each pixel in turn and store then in another variable, px1, pix2 etc. I'm running an iterative algorithm on 2 pixels at a time then moving to the next two pixels.
I am having alot of problems however reading the data. from the net and various books i see the readline method however this looks for new line charicters. I need something that will read the value of a certain number of bits from the file.

for example to read the first pixel data which is 16bits long i need something that will just read those 16bits and return its value.
Thanks.
Ed
Jun 26 '07 #1
  • viewed: 15586
Share:
11 Replies
bvdet
Expert Mod 2GB
This will read two bytes at a time:
Expand|Select|Wrap|Line Numbers
  1. >>> print [f.read(2) for _ in range(12)]
  2. ['96', '80', '07', '91', '37', '84', '41', '71', '15', '25', '18', '99']
  3. >>> 
This was done on a text file. You should look at PIL link
Jun 26 '07 #2
bartonc
Expert 4TB
This will read two bytes at a time:
Expand|Select|Wrap|Line Numbers
  1. >>> print [f.read(2) for _ in range(12)]
  2. ['96', '80', '07', '91', '37', '84', '41', '71', '15', '25', '18', '99']
  3. >>> 
This was done on a text file. You should look at PIL link
I imagine that you opened that text file in binary mode...
Expand|Select|Wrap|Line Numbers
  1. f = open('somefile.txt', 'b')
first.
Jun 26 '07 #3
bartonc
Expert 4TB
Hey Guys.
I'm looking for some info on how to read binary image data. Basicly RAW image files are strings of bits with say 2 bytes per pixel. What i need to do is read the value of each pixel in turn and store then in another variable, px1, pix2 etc. I'm running an iterative algorithm on 2 pixels at a time then moving to the next two pixels.
I am having alot of problems however reading the data. from the net and various books i see the readline method however this looks for new line charicters. I need something that will read the value of a certain number of bits from the file.

for example to read the first pixel data which is 16bits long i need something that will just read those 16bits and return its value.
Thanks.
Ed
I'm guessing that you know the inner workings of the image format that you are working on and can separate the format information (header) from the actual pixels of the image (data).

For working on large data sets, SciPy arrays have some features that native python lists lack.
Jun 26 '07 #4
Thanks guys.
Will investigate those options once i have time.
The RAW format i am working with is literally just raw data, it doesnt have a header file or start and stop bytes. The pixels have a bitdepth of 2bytes however the actual image data is only 10bits out of those 16, the rest are parity and padding bits.

Edit:
Ok just looking at the f.read(2) code above. The method returns the value of those two bytes as a string. The returned value is a hex value but is a string representation. How do i convert this string value into the actual hex value.
For example:

>>> f = open(r'C:\EFisher_ST_project07\RAW test images\724_day_mac_AV_on.raw', 'rb')
>>> pixel = f.read(2)
>>> pixel
'\xff\x00'
>>> type(pixel)
<type 'str'>

Cheers
Ed
Jun 26 '07 #5
bvdet
Expert Mod 2GB
Thanks guys.
Will investigate those options once i have time.
The RAW format i am working with is literally just raw data, it doesnt have a header file or start and stop bytes. The pixels have a bitdepth of 2bytes however the actual image data is only 10bits out of those 16, the rest are parity and padding bits.

Edit:
Ok just looking at the f.read(2) code above. The method returns the value of those two bytes as a string. The returned value is a hex value but is a string representation. How do i convert this string value into the actual hex value.
For example:

>>> f = open(r'C:\EFisher_ST_project07\RAW test images\724_day_mac_AV_on.raw', 'rb')
>>> pixel = f.read(2)
>>> pixel
'\xff\x00'
>>> type(pixel)
<type 'str'>

Cheers
Ed
Maybe this will help:
Expand|Select|Wrap|Line Numbers
  1. import re
  2. patt = re.compile(r'(?<=\\x)[a-f0-9]+')
  3. s = repr('\xff\x00')
  4.  
  5. for hexNum in patt.findall(s):
  6.     print int(hexNum, 16)
Output:
>>> 255
0
>>>
>>> patt.findall(s)
['ff', '00']
>>>
Jun 26 '07 #6
bartonc
Expert 4TB
Thanks guys.
Will investigate those options once i have time.
The RAW format i am working with is literally just raw data, it doesnt have a header file or start and stop bytes. The pixels have a bitdepth of 2bytes however the actual image data is only 10bits out of those 16, the rest are parity and padding bits.

Edit:
Ok just looking at the f.read(2) code above. The method returns the value of those two bytes as a string. The returned value is a hex value but is a string representation. How do i convert this string value into the actual hex value.
For example:

>>> f = open(r'C:\EFisher_ST_project07\RAW test images\724_day_mac_AV_on.raw', 'rb')
>>> pixel = f.read(2)
>>> pixel
'\xff\x00'
>>> type(pixel)
<type 'str'>

Cheers
Ed
I've never played with the struct module before, but here's what I found:
>>> import struct
>>> a = '\xff\xff'
>>> b = struct.unpack('BB', a)
>>> b
(255, 255)
>>> b = struct.unpack('H', a)
>>> b
(65535,)
>>>
Jun 27 '07 #7
bartonc
Expert 4TB
I've never played with the struct module before, but here's what I found:
>>> import struct
>>> a = '\xff\xff'
>>> b = struct.unpack('BB', a)
>>> b
(255, 255)
>>> b = struct.unpack('H', a)
>>> b
(65535,)
>>>
How about this:

>>> a = a * 8
>>> a
'\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\ xff\xff\xff\xff'
>>> import array
>>> b = array.array('H',a)
>>> b
array('H', [65535, 65535, 65535, 65535, 65535, 65535, 65535, 65535])
>>>
Jun 27 '07 #8
bartonc
Expert 4TB
How about this:

>>> a = a * 8
>>> a
'\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\ xff\xff\xff\xff'
>>> import array
>>> b = array.array('H',a)
>>> b
array('H', [65535, 65535, 65535, 65535, 65535, 65535, 65535, 65535])
>>>
Or better yet, use the array module to read the data directly:
fromfile( f, n)

Read n items (as machine values) from the file object f and append them to the end of the array. If less than n items are available, EOFError is raised, but the items that were available are still inserted into the array. f must be a real built-in file object; something else with a read() method won't do.
Jun 27 '07 #9
Cheers guys, will give these ideas a try and see how they turn out.
Thanks.
Ed
Jun 27 '07 #10
Hey Guys. I managed to do it, thats for your help.
Here is what i came up with:
Expand|Select|Wrap|Line Numbers
  1. def GetPxValue(openfile, bitdepth): #can iterate this part to get the next pixel
  2.     read_bytes = operator.div(bitdepth, 8)  
  3.     px_value = openfile.read(read_bytes)    
  4.     if (bitdepth == 8):
  5.         unpack_code = 'B'
  6.     if (bitdepth == 16):
  7.         unpack_code = 'BB'          
  8.     if (bitdepth == 32):
  9.         unpack_code = 'BBBB'
  10.     hex_value = struct.unpack(unpack_code, px_value)  
  11.     msbyte = Dec2Bin(hex_value[0])
  12.     if (bitdepth == 16):
  13.         lsbyte = Dec2Bin(hex_value[1])      
  14.         for i in range(8):          
  15.             msbyte.append(lsbyte[i])
  16.     if (bitdepth == 32):
  17.         byte2 = Dec2Bin(hex_value[1])
  18.         byte3 = Dec2Bin(hex_value[2])       
  19.         lsbyte = Dec2Bin(hex_value[3])
  20.         for i in range(8):
  21.             msbyte.append(byte2[i])
  22.         for i in range(8):
  23.             msbyte.append(byte3[i])
  24.         for i in range(8):
  25.             msbyte.append(lsbyte[i]) 
  26.     return msbyte 
  27. #return the whole bitdepth long pixel value as a list ready for filtering
  28.  
Note: Dec2Bin is a decimal to binary converter that returns a binary value as a list. the list is necessary for filtering in the next stage.
Cheers.
Ed
Jun 27 '07 #11
bartonc
Expert 4TB
Hey Guys. I managed to do it, thats for your help.
Here is what i came up with:
Expand|Select|Wrap|Line Numbers
  1. def GetPxValue(openfile, bitdepth): #can iterate this part to get the next pixel
  2.     read_bytes = operator.div(bitdepth, 8)  
  3.     px_value = openfile.read(read_bytes)    
  4.     if (bitdepth == 8):
  5.         unpack_code = 'B'
  6.     if (bitdepth == 16):
  7.         unpack_code = 'BB'          
  8.     if (bitdepth == 32):
  9.         unpack_code = 'BBBB'
  10.     hex_value = struct.unpack(unpack_code, px_value)  
  11.     msbyte = Dec2Bin(hex_value[0])
  12.     if (bitdepth == 16):
  13.         lsbyte = Dec2Bin(hex_value[1])      
  14.         for i in range(8):          
  15.             msbyte.append(lsbyte[i])
  16.     if (bitdepth == 32):
  17.         byte2 = Dec2Bin(hex_value[1])
  18.         byte3 = Dec2Bin(hex_value[2])       
  19.         lsbyte = Dec2Bin(hex_value[3])
  20.         for i in range(8):
  21.             msbyte.append(byte2[i])
  22.         for i in range(8):
  23.             msbyte.append(byte3[i])
  24.         for i in range(8):
  25.             msbyte.append(lsbyte[i]) 
  26.     return msbyte 
  27. #return the whole bitdepth long pixel value as a list ready for filtering
  28.  
Note: Dec2Bin is a decimal to binary converter that returns a binary value as a list. the list is necessary for filtering in the next stage.
Cheers.
Ed
Hi Ed. It's great when members post the solutions that they have come up with. I've added CODE tags to your post. Instructions on how to do this are on the right hand side of the page while you are posting or replying.

I felt sure that you would go with the array thing when I discovered it. I think that you will get a performance boost be reading larger chunks and working on slices of the array. The cool thing about python is that you may use one approach to get things working, then, with a little tweaking, make it work really well.

Have fun,
Barton
Jun 27 '07 #12

Post your reply

Sign in to post your reply or Sign up for a free account.

Similar topics

7 posts views Thread by John | last post: by
2 posts views Thread by Ed | last post: by
6 posts views Thread by jcasique.torres@gmail.com | last post: by
By using this site, you agree to our Privacy Policy and Terms of Use.