Hex editor display - can this be more pythonic?

[CC]

Hi:

I'm building a hex line editor as a first real Python programming exercise.

Yesterday I posted about how to print the hex bytes of a string. There
are two decent options:

ln = '\x00\x01\xFF 456\x0889abcde~'
import sys
for c in ln:
sys.stdout.write( '%.2X ' % ord(c) )

or this:

sys.stdout.write( ' '.join( ['%.2X' % ord(c) for c in ln] ) + ' ' )

Either of these produces the desired output:

00 01 FF 20 34 35 36 08 38 39 61 62 63 64 65 7E

I find the former more readable and simpler. The latter however has a
slight advantage in not putting a space at the end unless I really want
it. But which is more pythonic?

The next step consists of printing out the ASCII printable characters.
I have devised the following silliness:

printable = '
1!2@3#4$5%6^7&8*9(0)aAbBcCdDeEfFgGhHiIjJkKlLmMnNoO pPqQrRsStTuUvVwWxXyYzZ\
`~-_=+\\|[{]};:\'",<.>/?'
for c in ln:
if c in printable: sys.stdout.write(c)
else: sys.stdout.write('.')

print

Which when following the list comprehension based code above, produces
the desired output:

00 01 FF 20 34 35 36 08 38 39 61 62 63 64 65 7E ... 456.89abcde~

I had considered using the .translate() method of strings, however this
would require a larger translation table than my printable string. I
was also using the .find() method of the printable string before
realizing I could use 'in' here as well.

I'd like to display the non-printable characters differently, since they
can't be distinguished from genuine period '.' characters. Thus, I may
use ANSI escape sequences like:

for c in ln:
if c in printable: sys.stdout.write(c)
else:
sys.stdout.write('\x1B[31m.')
sys.stdout.write('\x1B[0m')

print
I'm also toying with the idea of showing hex bytes together with their
ASCII representations, since I've often found it a chore to figure out
which hex byte to change if I wanted to edit a certain ASCII char.
Thus, I might display data something like this:

00(\0) 01() FF() 20( ) 34(4) 35(5) 36(6) 08(\b) 38(8) 39(9) 61(a) 62(b)
63(c) 64(d) 65(e) 7E(~)

Where printing chars are shown in parenthesis, characters with Python
escape sequences will be shown as their escapes in parens., while
non-printing chars with no escapes will be shown with nothing in parens.

Or perhaps a two-line output with offset addresses under the data. So
many possibilities!
Thanks for input!


--
_____________________
Christopher R. Carlen
cr***@bogus-remove-me.sbcglobal.net
SuSE 9.1 Linux 2.6.5

[Marc 'BlackJack' Rintsch]

On Sun, 29 Jul 2007 12:24:56 -0700, CC wrote:

ln = '\x00\x01\xFF 456\x0889abcde~'
import sys
for c in ln:
sys.stdout.write( '%.2X ' % ord(c) )

or this:

sys.stdout.write( ' '.join( ['%.2X' % ord(c) for c in ln] ) + ' ' )

Either of these produces the desired output:

00 01 FF 20 34 35 36 08 38 39 61 62 63 64 65 7E

I find the former more readable and simpler. The latter however has a
slight advantage in not putting a space at the end unless I really want
it. But which is more pythonic?

I would use the second with fewer spaces, a longer name for `ln` and in
recent Python versions with a generator expression instead of the list
comprehension:

sys.stdout.write(' '.join('%0X' % ord(c) for c in line))

The next step consists of printing out the ASCII printable characters.
I have devised the following silliness:

printable = '
1!2@3#4$5%6^7&8*9(0)aAbBcCdDeEfFgGhHiIjJkKlLmMnNoO pPqQrRsStTuUvVwWxXyYzZ\
`~-_=+\\|[{]};:\'",<.>/?'

I'd use `string.printable` and remove the "invisible" characters like '\n'
or '\t'.

for c in ln:
if c in printable: sys.stdout.write(c)
else: sys.stdout.write('.')

print

Which when following the list comprehension based code above, produces
the desired output:

00 01 FF 20 34 35 36 08 38 39 61 62 63 64 65 7E ... 456.89abcde~

I had considered using the .translate() method of strings, however this
would require a larger translation table than my printable string.

The translation table can be created once and should be faster.

I'd like to display the non-printable characters differently, since they
can't be distinguished from genuine period '.' characters. Thus, I may
use ANSI escape sequences like:

for c in ln:
if c in printable: sys.stdout.write(c)
else:
sys.stdout.write('\x1B[31m.')
sys.stdout.write('\x1B[0m')

print

`re.sub()` might be an option here.

I'm also toying with the idea of showing hex bytes together with their
ASCII representations, since I've often found it a chore to figure out
which hex byte to change if I wanted to edit a certain ASCII char. Thus,
I might display data something like this:

00(\0) 01() FF() 20( ) 34(4) 35(5) 36(6) 08(\b) 38(8) 39(9) 61(a) 62(b)
63(c) 64(d) 65(e) 7E(~)

Where printing chars are shown in parenthesis, characters with Python
escape sequences will be shown as their escapes in parens., while
non-printing chars with no escapes will be shown with nothing in parens.

For escaping:

In [90]: '\n'.encode('string-escape')
Out[90]: '\\n'

Ciao,
Marc 'BlackJack' Rintsch

[CC]

Marc 'BlackJack' Rintsch wrote:

On Sun, 29 Jul 2007 12:24:56 -0700, CC wrote:

>>The next step consists of printing out the ASCII printable characters.
I have devised the following silliness:

printable = '
1!2@3#4$5%6^7&8*9(0)aAbBcCdDeEfFgGhHiIjJkKlLmMnN oOpPqQrRsStTuUvVwWxXyYzZ\
`~-_=+\\|[{]};:\'",<.>/?'

I'd use `string.printable` and remove the "invisible" characters like '\n'
or '\t'.

What is `string.printable` ? There is no printable method to strings,
though I had hoped there would be. I don't yet know how to make one.

>>for c in ln:
if c in printable: sys.stdout.write(c)
else: sys.stdout.write('.')

The translation table can be created once and should be faster.

I suppose the way I'm doing it requires a search through `printable` for
each c, right? Whereas the translation would just be a lookup
operation? If so then perhaps the translation would be better.

>>I'd like to display the non-printable characters differently, since they
can't be distinguished from genuine period '.' characters. Thus, I may
use ANSI escape sequences like:

for c in ln:
if c in printable: sys.stdout.write(c)
else:
sys.stdout.write('\x1B[31m.')
sys.stdout.write('\x1B[0m')

print

`re.sub()` might be an option here.

Yeah, that is an interesting option. Since I don't wish to modify the
block of data unless the user specifically edits it, so I might prefer
the simple display operation.

For escaping:

In [90]: '\n'.encode('string-escape')
Out[90]: '\\n'

Hmm, I see there's an encoder that can do my hex display too.

Thanks for the input!

--
_____________________
Christopher R. Carlen
cr***@bogus-remove-me.sbcglobal.net
SuSE 9.1 Linux 2.6.5

[CC]

Dennis Lee Bieber wrote:

On Sun, 29 Jul 2007 12:24:56 -0700, CC <cr***@BOGUS.sbcglobal.net>
declaimed the following in comp.lang.python:

>>for c in ln:
if c in printable: sys.stdout.write(c)
else:
sys.stdout.write('\x1B[31m.')
sys.stdout.write('\x1B[0m')

Be aware that this does require having a terminal that understands
the escape sequences (which, to my understanding, means unusable on a
WinXP console window)

Yeah, with this I'm not that concerned about Windows. Though, can WinXP
still load the ansi.sys driver?

>>Thus, I might display data something like this:

00(\0) 01() FF() 20( ) 34(4) 35(5) 36(6) 08(\b) 38(8) 39(9) 61(a) 62(b)
63(c) 64(d) 65(e) 7E(~)

UGH!

:-D Lovely isn't it?

If the original "hex bytes dotted ASCII" side by side isn't
workable, I'd suggest going double line...

00 01 FF 20 34 35 36 08 38 39 61 62 63 64 65 7E
nul soh xFF sp 4 5 6 bs 8 9 a b c d e ~

Yeah, something like that is probably nicer.

Use the standard "name" for the control codes (though I shortened
"space" to "sp", and maybe just duplicate the hex for non-named,
non-printable, codes (mostly those in the x80-xFF range, unless you are
NOT using ASCII but something like ISO-Latin-1

I've got a lot to learn about this encoding business.

To allow for the names, means using a field width of four. Using a
line width of 16-data bytes makes for an edit window width of 64, and
you could fit a hex offset at the left of each line to indicate what
part of the file is being worked.

Right.
Thanks for the reply!
--
_____________________
Christopher R. Carlen
cr***@bogus-remove-me.sbcglobal.net
SuSE 9.1 Linux 2.6.5

[Marc 'BlackJack' Rintsch]

On Sun, 29 Jul 2007 18:27:25 -0700, CC wrote:

Marc 'BlackJack' Rintsch wrote:

>I'd use `string.printable` and remove the "invisible" characters like '\n'
or '\t'.

What is `string.printable` ? There is no printable method to strings,
though I had hoped there would be. I don't yet know how to make one.

In [8]: import string

In [9]: string.printable
Out[9]: '0123456789abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLM NOPQRSTUVWXYZ!"#$%&\'(
)*+,-./:;<=>?@[\\]^_`{|}~\t\n\r\x0b\x0c'

>>>for c in ln:
if c in printable: sys.stdout.write(c)
else: sys.stdout.write('.')

>The translation table can be created once and should be faster.

I suppose the way I'm doing it requires a search through `printable` for
each c, right? Whereas the translation would just be a lookup
operation?

Correct. And it is written in C.

Ciao,
Marc 'BlackJack' Rintsch

[Neil Cerutti]

On 2007-07-30, Dennis Lee Bieber <wl*****@ix.netcom.comwrote:

On Sun, 29 Jul 2007 18:30:22 -0700, CC <cr***@BOGUS.sbcglobal.net>
declaimed the following in comp.lang.python:

>>
Yeah, with this I'm not that concerned about Windows. Though, can WinXP
still load the ansi.sys driver?

I'm actually not sure...

I think if one uses the 16-bit command parser it is available, but
not the 32-bit parser...

command.com vs cmd.exe

Yes. You can load the ansi.sys driver in command.com on Windows
2000 and XP, and it will work with simply batch files. But it
doesn't work with Python, for reasons I don't know enough about
Windows console programs to understand.

--
Neil Cerutti
The audience is asked to remain seated until the end of the recession.
--Church Bulletin Blooper