472,805 Members | 1,035 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 472,805 software developers and data experts.

Manipulating binary data

Hi guys,

I need to manipulate binary data (8 bit) stored in a 2 dimensional
array.

I've tried various methods (arrays, using a string filled with chr(0),
using gd lib) and so far the fastest way I've found is to actually
create an 8-bit image in GD and use imagecolorat and imagesetpixel to
read and write the data.

Are there actually any proper binary extensions or is using gd lib the
way to go (as I guess it's just dealing with binary data as a 2d array
anyway)

Andrew

Apr 29 '07 #1
8 13003
On Sun, 29 Apr 2007 09:36:38 -0700, brainflakes.org wrote:
Hi guys,

I need to manipulate binary data (8 bit) stored in a 2 dimensional
array.

I've tried various methods (arrays, using a string filled with chr(0),
using gd lib) and so far the fastest way I've found is to actually
create an 8-bit image in GD and use imagecolorat and imagesetpixel to
read and write the data.

Are there actually any proper binary extensions or is using gd lib the
way to go (as I guess it's just dealing with binary data as a 2d array
anyway)

Andrew
What kind if wizardry are you trying to achieve? Maybe a bit of your code
and an explanation of what you are after makes helping easier.

You are aware of the bitwise operators: (are you?)
$a & $b And Bits that are set in both $a and $b are set.
$a | $b Or Bits that are set in either $a or $b are set.
$a ^ $b Xor Bits that are set in $a or $b but not both are set.
~ $a Not Bits that are set in $a are not set, and vice versa.
$a << $b Shift left Shift the bits of $a $b steps to the left
(each step means "multiply by two")
$a >$b Shift right Shift the bits of $a $b steps to the right
(each step means "divide by two")
Sh.
Apr 29 '07 #2
On Apr 29, 6:36 pm, brainflakes....@googlemail.com wrote:
Hi guys,

I need to manipulate binary data (8 bit) stored in a 2 dimensional
array.

I've tried various methods (arrays, using a string filled with chr(0),
using gd lib) and so far the fastest way I've found is to actually
create an 8-bit image in GD and use imagecolorat and imagesetpixel to
read and write the data.

Are there actually any proper binary extensions or is using gd lib the
way to go (as I guess it's just dealing with binary data as a 2d array
anyway)

Andrew
I doubt that's the optimal solution. Function calls are relatively
expensive in PHP. The difference in speed you observed between the
different methods can probably be attributed to the number of calls
made per element.

Explain exactly what you're trying to do. There probably is a better
approach.

Apr 30 '07 #3
I doubt that's the optimal solution. Function calls are relatively
expensive in PHP. The difference in speed you observed between the
different methods can probably be attributed to the number of calls
made per element.

Explain exactly what you're trying to do. There probably is a better
approach.
I'm creating some custom image processing filters on 8bit greyscale
images

for($x=0; $x<$width; $x++){
for($y=0; $x<$height; $y++){
check value of pixels (0-255) in a 10x10 area around given pixel
calculate new value
plot value at x,y (0-255)
}
}

I figured the best way to do it would be to just "allocate" a memory
space of $width * $height bytes and copy the image into that, which I
guess is done by creating a string of that length. However even just
plotting to $string[$n++]=chr($val) (no co-ord calculations) is slower
then imagesetpixel() on an 8-bit image.

For some reason tho I can't actually render directly from the 8-bit
image (I use imagesetcolor() to set colours in a greyscale gradient
but imagepng complains there's the wrong number of colours) so I have
to copy the image in and out of a truecolour image anyway.

I have done some further optimisations on the actual processing code
and improved the speed about 10-fold, but if there's a faster way to
read/write 8bit data I'm still interested as every little helps when
you're doing batch processing. (tho I guess I should probably be doing
it in C or .net anyway ;)

Andrew

Apr 30 '07 #4
brainflakes.org wrote:
Are there actually any proper binary extensions or is using gd lib the
way to go (as I guess it's just dealing with binary data as a 2d array
anyway)
Assuming that each "row" of data is of the same length, and each "column"
has the same height (which I think is a fairly safe assumption if you're
modelling the data as a rectangular image) then, why not just store the
data as a string. That is, instead of storing a 3x3 array of data as this:

$array = array(
array(0x61, 0x62, 0x63),
array(0x31, 0x32, 0x33),
array(0x78, 0x79, 0x7A)
);

you store it as:

$string = 'abc123xyz';

Which is roughly how a 2-dimensional array is internally stored by a C
program.

Instead of this:

$cell = $array[$x][$y];

You use:

define(ROW_LENGTH, 3);
$cell = $string[(ROW_LENGTH*$x) + $y];

And instead of this:

$array[$x][$y] = 0x64;

this:

$string[(ROW_LENGTH*$x) + $y] = chr(0x64);
--
Toby A Inkster BSc (Hons) ARCS
http://tobyinkster.co.uk/
Geek of ~ HTML/SQL/Perl/PHP/Python*/Apache/Linux

* = I'm getting there!
Apr 30 '07 #5
On Apr 30, 11:39 am, brainflakes....@googlemail.com wrote:
I doubt that's the optimal solution. Function calls are relatively
expensive in PHP. The difference in speed you observed between the
different methods can probably be attributed to the number of calls
made per element.
Explain exactly what you're trying to do. There probably is a better
approach.

I'm creating some custom image processing filters on 8bit greyscale
images

for($x=0; $x<$width; $x++){
for($y=0; $x<$height; $y++){
check value of pixels (0-255) in a 10x10 area around given pixel
calculate new value
plot value at x,y (0-255)
}

}

I figured the best way to do it would be to just "allocate" a memory
space of $width * $height bytes and copy the image into that, which I
guess is done by creating a string of that length. However even just
plotting to $string[$n++]=chr($val) (no co-ord calculations) is slower
then imagesetpixel() on an 8-bit image.

For some reason tho I can't actually render directly from the 8-bit
image (I use imagesetcolor() to set colours in a greyscale gradient
but imagepng complains there's the wrong number of colours) so I have
to copy the image in and out of a truecolour image anyway.

I have done some further optimisations on the actual processing code
and improved the speed about 10-fold, but if there's a faster way to
read/write 8bit data I'm still interested as every little helps when
you're doing batch processing. (tho I guess I should probably be doing
it in C or .net anyway ;)

Andrew
Hmmm, that sort of processing will likely be slow in any event. Here
are some tips:

A hash look up is faster than calling chr() or ord(). So you can speed
things up by employing two tables:

$chr = array( 0 ="\x00", 1 ="\x01", 2 ="\x02" ..., 0xFF =>
"\xFF" );
$ord = array_flip($chr);
Unpacking the binary data into an array of ints with unpack() and then
repacking the result with pack() could be faster than repeated binary-
to-numeric conversions.
for() loops are slower than foreach(). Using the range() function, you
can create two arrays holding the indexes then go through them with
foreach() instead of incrementing them.
Loops have overhead. You can squeeze out some of that by unrolling
tight loops. What you would do is dynamically write out the PHP code
that'd execute by a full iteration of the loop, then create a lambda
function using create_function(). I used that technique in my
imagecreatefrombmp() routine(http://www.conradish.net/bobo/
show_source.php?filename=bmp.php).
When possible, use array_map() or array_walk() instead of looping.

Apr 30 '07 #6
>
you store it as:

$string = 'abc123xyz';

Which is roughly how a 2-dimensional array is internally stored by a C
program.

As I said I've tried that, however using GD lib is actually 30-40%
faster then manipulating a string

"I figured the best way to do it would be to just "allocate" a memory
space of $width * $height bytes and copy the image into that, which I
guess is done by creating a string of that length. However even just
plotting to $string[$n++]=chr($val) (no co-ord calculations) is slower
then imagesetpixel() on an 8-bit image"

Apr 30 '07 #7
On Apr 30, 11:49 am, Toby A Inkster <usenet200...@tobyinkster.co.uk>
wrote:
brainflakes.org wrote:
Are there actually any proper binary extensions or is using gd lib the
way to go (as I guess it's just dealing with binary data as a 2d array
anyway)

Assuming that each "row" of data is of the same length, and each "column"
has the same height (which I think is a fairly safe assumption if you're
modelling the data as a rectangular image) then, why not just store the
data as a string. That is, instead of storing a 3x3 array of data as this:

$array = array(
array(0x61, 0x62, 0x63),
array(0x31, 0x32, 0x33),
array(0x78, 0x79, 0x7A)
);

you store it as:

$string = 'abc123xyz';

Which is roughly how a 2-dimensional array is internally stored by a C
program.

Instead of this:

$cell = $array[$x][$y];

You use:

define(ROW_LENGTH, 3);
$cell = $string[(ROW_LENGTH*$x) + $y];

And instead of this:

$array[$x][$y] = 0x64;

this:

$string[(ROW_LENGTH*$x) + $y] = chr(0x64);

--
Toby A Inkster BSc (Hons) ARCShttp://tobyinkster.co.uk/
Geek of ~ HTML/SQL/Perl/PHP/Python*/Apache/Linux

* = I'm getting there!
Accessing a character within a string is going to be slower than
accessing an element in an array. It's true that PHP can very quickly
find the value given the index, but it has to create a new string
object to hold that character ($string[$i] is a one character string).
Memory allocation takes more time than a hash look-up, hence negative
saving.

The key to performance here is to reduce the amount of work in the
inner loop, as it's going to be run tens of thousands of times.
Converting the binary data to numbers and breaking them into an array
per line ahead of time will be likely be faster.

Apr 30 '07 #8
I'd love to hear how you end up doing this, if you don't mind.

Aerik

May 2 '07 #9

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

13
by: yaipa | last post by:
What would be the common sense way of finding a binary pattern in a ..bin file, say some 200 bytes, and replacing it with an updated pattern of the same length at the same offset? Also, the...
2
by: Benji99 | last post by:
Hi guys, I'm starting to learn Python and so far am very impressed with it's possibilities. I do however need some help with certain things I'm trying to do which as of yet haven't managed to find...
10
by: Kristian Nybo | last post by:
Hi, I'm writing a simple image file exporter as part of a school project. To implement my image format of choice I need to work with big-endian bytes, where 'byte' of course means '8 bits', not...
103
by: Steven T. Hatton | last post by:
27.4.2.1.4 Type ios_base::openmode Says this about the std::ios::binary openmode flag: *binary*: perform input and output in binary mode (as opposed to text mode) And that is basically _all_ it...
2
by: Lisa Pearlson | last post by:
Hi, My php application (on Apache/Linux) needs to do the following: The PHP script receives a request from a client (binary), asking for certain records of data. My PHP script loops through...
28
by: wwj | last post by:
void main() { char* p="Hello"; printf("%s",p); *p='w'; printf("%s",p); }
2
by: Ido Flatow | last post by:
Hi all, I've been exploring the way I can manipulate WSDL.exe using SchemaImporterExtension in order to create a proxy to my liking. My situation is as follows - I have a web site that has...
1
by: Stephen Cattaneo | last post by:
Hi all, I am relatively new to socket programming. I am attempting to use raw sockets to spoof my IP address. From what I can tell I will have to build from the Ethernet layer on up. This is...
0
by: erikbower65 | last post by:
Using CodiumAI's pr-agent is simple and powerful. Follow these steps: 1. Install CodiumAI CLI: Ensure Node.js is installed, then run 'npm install -g codiumai' in the terminal. 2. Connect to...
0
linyimin
by: linyimin | last post by:
Spring Startup Analyzer generates an interactive Spring application startup report that lets you understand what contributes to the application startup time and helps to optimize it. Support for...
0
by: erikbower65 | last post by:
Here's a concise step-by-step guide for manually installing IntelliJ IDEA: 1. Download: Visit the official JetBrains website and download the IntelliJ IDEA Community or Ultimate edition based on...
0
by: kcodez | last post by:
As a H5 game development enthusiast, I recently wrote a very interesting little game - Toy Claw ((http://claw.kjeek.com/))。Here I will summarize and share the development experience here, and hope it...
0
by: Taofi | last post by:
I try to insert a new record but the error message says the number of query names and destination fields are not the same This are my field names ID, Budgeted, Actual, Status and Differences ...
0
by: Rina0 | last post by:
I am looking for a Python code to find the longest common subsequence of two strings. I found this blog post that describes the length of longest common subsequence problem and provides a solution in...
0
by: lllomh | last post by:
Define the method first this.state = { buttonBackgroundColor: 'green', isBlinking: false, // A new status is added to identify whether the button is blinking or not } autoStart=()=>{
0
by: Mushico | last post by:
How to calculate date of retirement from date of birth
2
by: DJRhino | last post by:
Was curious if anyone else was having this same issue or not.... I was just Up/Down graded to windows 11 and now my access combo boxes are not acting right. With win 10 I could start typing...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.