binutils "strings" like functionality?

[cjl]

Hey all:

I am working on a little script that needs to pull the strings out of a
binary file, and then manipulate them with python.

The command line utility "strings" (part of binutils) has exactly the
functionality I need, but I was thinking about trying to implement this
in pure python.

I did some reading on opening and reading binary files, etc., and was
just wondering if people think this is possible, or worth my time (as a
learning exercise), or if something like this already exists.

-cjl

[Fuzzyman]

I don't know anything about binutils strings, other than that a quick
google reveals that you can search binary files for printable strings
with it.

In python the following code will read a binary file into a single
string :

long_string = open(filename, 'rb').read()

You can then slice long_string, iterate over it... or do whatever you
like with it.

Now you *ought* to be creating a proepr file handle and closing it. You
probably don't want to read the whole file into memory at once, etc....
But implementing these thigns are trivial.

What particular aspect of 'strings' did you want to implement ?

Regards,

Fuzzy
http://www.voidspace.org.uk/python/index.shtml

[Larry Bates]

Take a look at python's struct module in the standard library.
Many people use it to manipulate binary objects at the byte
level with great success.

Larry Bates
cjl wrote:

Hey all:

I am working on a little script that needs to pull the strings out of a
binary file, and then manipulate them with python.

The command line utility "strings" (part of binutils) has exactly the
functionality I need, but I was thinking about trying to implement this
in pure python.

I did some reading on opening and reading binary files, etc., and was
just wondering if people think this is possible, or worth my time (as a
learning exercise), or if something like this already exists.

-cjl

[Kent Johnson]

cjl wrote:

Hey all:

I am working on a little script that needs to pull the strings out of a
binary file, and then manipulate them with python.

The command line utility "strings" (part of binutils) has exactly the
functionality I need, but I was thinking about trying to implement this
in pure python.

I did some reading on opening and reading binary files, etc., and was
just wondering if people think this is possible, or worth my time (as a
learning exercise), or if something like this already exists.

I think a bare-bones version is pretty simple:
- read the file
- use re.split() to split on non-printable characters (string.printable could be handy here)
- print anything in the resulting list that is at least 4 characters

Since you mention this as a learning exercise I'll refrain from posting code but it's only a handful
of lines...of course implementing all of 'strings' is a bit more work.

Kent

[Steve Holden]

Kent Johnson wrote:

cjl wrote:

Hey all:

I am working on a little script that needs to pull the strings out of a
binary file, and then manipulate them with python.

The command line utility "strings" (part of binutils) has exactly the
functionality I need, but I was thinking about trying to implement this
in pure python.

I did some reading on opening and reading binary files, etc., and was
just wondering if people think this is possible, or worth my time (as a
learning exercise), or if something like this already exists.

I think a bare-bones version is pretty simple:
- read the file
- use re.split() to split on non-printable characters (string.printable
could be handy here)
- print anything in the resulting list that is at least 4 characters

The binutils version may well use a zero terminator character to
differentiate between random chunks of printable characters and those
explicitly genreated as strings from C. Or it may not ...
Since you mention this as a learning exercise I'll refrain from posting
code but it's only a handful of lines...of course implementing all of
'strings' is a bit more work.

regards
Steve
--
Meet the Python developers and your c.l.py favorites March 23-25
Come to PyCon DC 2005 http://www.pycon.org/
Steve Holden http://www.holdenweb.com/

[Fredrik Lundh]

"cjl" wrote:

I am working on a little script that needs to pull the strings out of a
binary file, and then manipulate them with python.

The command line utility "strings" (part of binutils) has exactly the
functionality I need, but I was thinking about trying to implement this
in pure python.

something like this could work:

import re

text = open(file, "rb").read()

for m in re.finditer("([\x20-\x7f]{4,})[\n\0]", text):
print m.start(), repr(m.group(1))

you may wish to modify the "[\x20-\x7f]" part to match your definition of
"printable characters". "[-,.!?\w ]" is a reasonable choice in many cases...

if the files can be huge, use the mmap module to map the file into memory,
and run the RE on the mapped view.

</F>

[cjl]

Fredrik Lundh wrote:

something like this could work:

import re

text = open(file, "rb").read()

for m in re.finditer("([\x20-\x7f]{4,})[\n\0]", text):
print m.start(), repr(m.group(1))

Hey...that worked. I actually modified:

for m in re.finditer("([\x20-\x7f]{4,})[\n\0]", text):

to

for m in re.finditer("([\x20-\x7f]{4,})", text):

and now the output is nearly identical to 'strings'. One problem
exists, in that if the binary file contains a string
"monkey/chicken/dog/cat" it is printed as "mokey//chicken//dog//cat",
and I don't know enough to figure out where the extra "/" is coming
from.

Help?

-CJL

[David M. Cooke]

"cjl" <cj****@gmail.com> writes:

Fredrik Lundh wrote:

something like this could work:

import re

text = open(file, "rb").read()

for m in re.finditer("([\x20-\x7f]{4,})[\n\0]", text):
print m.start(), repr(m.group(1))

Hey...that worked. I actually modified:

for m in re.finditer("([\x20-\x7f]{4,})[\n\0]", text):

to

for m in re.finditer("([\x20-\x7f]{4,})", text):

and now the output is nearly identical to 'strings'. One problem
exists, in that if the binary file contains a string
"monkey/chicken/dog/cat" it is printed as "mokey//chicken//dog//cat",
and I don't know enough to figure out where the extra "/" is coming
from.

Are you sure it's monkey/chicken/dog/cat, and not
monkey\chicken\dog\cat? The later one will print monkey\\chicken...
because of the repr() call.

Also, you probably want it as [\x20-\x7e] (the DEL character \x7f
isn't printable). You're also missing tabs (\t).

The GNU binutils string utility looks for \t or [\x20-\x7e].

--
|>|\/|<
/--------------------------------------------------------------------------\
|David M. Cooke
|cookedm(at)physics(dot)mcmaster(dot)ca

[cjl]

David M. Cooke wrote:

Are you sure it's monkey/chicken/dog/cat, and not
monkey\chicken\dog\cat? The later one will print monkey\\chicken...
because of the repr() call.

Also, you probably want it as [\x20-\x7e] (the DEL character \x7f
isn't printable). You're also missing tabs (\t).

The GNU binutils string utility looks for \t or [\x20-\x7e].

Yeah, it is "monkey\chicken\dog\cat", thank you for pointing that out.

I started snooping throught the sourcecode for 'strings' in binutils to
see what it matches against, but I don't really know how to program, so
you just saved me even more time. Thanks again!

-CJL

[Marc 'BlackJack' Rintsch]

In <11**********************@l41g2000cwc.googlegroups .com>, cjl wrote:

I am working on a little script that needs to pull the strings out of a
binary file, and then manipulate them with python.

The command line utility "strings" (part of binutils) has exactly the
functionality I need, but I was thinking about trying to implement this
in pure python.

I did some reading on opening and reading binary files, etc., and was
just wondering if people think this is possible, or worth my time (as a
learning exercise), or if something like this already exists.

If you find it interesting and challenging go ahead and try it. It is
possible and with the help of the 're' and 'mmap' modules it should be
quite easy to filter strings out of files without the need to load them
into memory at once.

Ciao,
Marc 'BlackJack' Rintsch