Encryption with Python

[Kamilche]

I've looked at a few alternatives for encryption with Python, and
didn't come up anything very speedy.

I've written an encryption algorithm in pure Python that can process
22 megs of data a second. I know it's not secure, but it should be
enough to ward off casual hacking. Does someone know of something
speedier?

--Kamilche

[Erik Max Francis]

Kamilche wrote:

I've looked at a few alternatives for encryption with Python, and
didn't come up anything very speedy.

I've written an encryption algorithm in pure Python that can process
22 megs of data a second. I know it's not secure, but it should be
enough to ward off casual hacking. Does someone know of something
speedier?

If you're only concerned with warding off casual hacking, and speed is
your maximal concern, why not just XOR each byte with 0xFF or something
equally brainless?

--
__ Erik Max Francis && ma*@alcyone.com && http://www.alcyone.com/max/
/ \ San Jose, CA, USA && 37 20 N 121 53 W && AIM erikmaxfrancis
\__/ Freedom is never voluntarily given by the oppressor.
-- Dr. Martin Luther King, Jr.

[Paul Rubin]

kl*******@home.com (Kamilche) writes:

I've written an encryption algorithm in pure Python that can process
22 megs of data a second. I know it's not secure, but it should be
enough to ward off casual hacking. Does someone know of something
speedier?

Yes, you can just copy the input to the output. That's quite fast,
and it's at least as insecure as your method.

[Peter Hansen]

Kamilche wrote:

I've looked at a few alternatives for encryption with Python, and
didn't come up anything very speedy.

I've written an encryption algorithm in pure Python that can process
22 megs of data a second. I know it's not secure, but it should be
enough to ward off casual hacking. Does someone know of something
speedier?

In addition to Erik and Paul's comments: if you don't specify
what machine you ran your benchmark on, the number "22MB/s" is
completely meaningless...

Besides, what you say is not possible. On my machine,
which is about a P4 2500MHz, scanning an array.array('c') with
22MB of data in it, doing nothing but reading each byte and
ignoring it, takes about 8 seconds. So does converting the
array to a list, which is pretty much all C code. Strings
are immutable, so you can't be working with one of them...

Doing anything meaningful with real data, no matter how trivial
the algorithm, would definitely take longer.

-Peter

[Paul Rubin]

Peter Hansen <pe***@engcorp.com> writes:

Besides, what you say is not possible. On my machine,
which is about a P4 2500MHz, scanning an array.array('c') with
22MB of data in it, doing nothing but reading each byte and
ignoring it, takes about 8 seconds. So does converting the
array to a list, which is pretty much all C code. Strings
are immutable, so you can't be working with one of them...

Well, you could do 32-bit operations which are maybe 4x faster.
You could maybe get even more speedup using built-in C functions.
For example, you could do rot13 encryption with the string.translate
method which should be quite fast. FWIW,

http://www.nightsong.com/phr/crypto/p3.py

should be quite secure, and should run at least 1 mbyte/sec on your
p4/2500, maybe quite a bit more.

[Peter Hansen]

Paul Rubin wrote:

Peter Hansen <pe***@engcorp.com> writes:

Besides, what you say is not possible. On my machine,
which is about a P4 2500MHz, scanning an array.array('c') with
22MB of data in it, doing nothing but reading each byte and
ignoring it, takes about 8 seconds. So does converting the
array to a list, which is pretty much all C code. Strings
are immutable, so you can't be working with one of them...

Well, you could do 32-bit operations which are maybe 4x faster.
You could maybe get even more speedup using built-in C functions.
For example, you could do rot13 encryption with the string.translate
method which should be quite fast.

Both are good ideas, but I made an assumption that I was starting
with a simple 22MB string, and the cost of converting that to 32-bit
(presumably using array.array still?) would have to be taken into
account, for fairness. I also assume Kamilche wasn't talking
about using the builtin rot13, since he claimed to have written
this himself...

-Peter

[Larry Bates]

Take a look at pyCrypto. My understanding is that
it provides python wrapper around C crypto functions.
Can't get much faster than that.

http://www.amk.ca/python/code/crypto.html

HTH,
Larry Bates

"Kamilche" <kl*******@home.com> wrote in message
news:88**************************@posting.google.c om...

I've looked at a few alternatives for encryption with Python, and
didn't come up anything very speedy.

I've written an encryption algorithm in pure Python that can process
22 megs of data a second. I know it's not secure, but it should be
enough to ward off casual hacking. Does someone know of something
speedier?

--Kamilche

[Paul Rubin]

Peter Hansen <pe***@engcorp.com> writes:

Both are good ideas, but I made an assumption that I was starting
with a simple 22MB string, and the cost of converting that to 32-bit
(presumably using array.array still?) would have to be taken into
account, for fairness. I also assume Kamilche wasn't talking
about using the builtin rot13, since he claimed to have written
this himself...

There's no special conversion needed, just use array.array('L', whatever)
instead of array.array('B', whatever), if I remember right.

[Peter Hansen]

Paul Rubin wrote:

Peter Hansen <pe***@engcorp.com> writes:

the cost of converting that to 32-bit (presumably using array.array still?) ....

There's no special conversion needed, just use array.array('L', whatever)
instead of array.array('B', whatever), if I remember right.

Cool! I didn't realize you could initialize effectively any
array object from a string as well as a list. Never too late
to read the docs... :-)

-Peter

[Kamilche]

Peter Hansen <pe***@engcorp.com> wrote in message news:<Td********************@powergate.ca>...

Besides, what you say is not possible. On my machine,
which is about a P4 2500MHz, scanning an array.array('c') with
22MB of data in it, doing nothing but reading each byte and
ignoring it, takes about 8 seconds. So does converting the
array to a list, which is pretty much all C code. Strings
are immutable, so you can't be working with one of them...

Doing anything meaningful with real data, no matter how trivial
the algorithm, would definitely take longer.

But I DON'T manipulate the data byte by byte, only the encryption
tables.
Ah, the power of algorithms! ;-)
Take a look at this:

def encrypt(s, offset = 0, _tables = []):
' Encrypt or decrypt a string'
cnt = len(_tables)
if cnt == 0:
# Initialize static variable _tables
# Increase 'cnt' to prevent recurring patterns.
import random
cnt = 10
random.seed(257)
for i in range(cnt):
marked, table, max = [], [], 256
for j in range(max):
table.append(chr(j))
marked.append(False)
marked[0] = True
marked[255] = True
for j in range(max):
if not marked[j]:
while 1:
c = random.randint(0, 255)
if marked[c] == False:
table[j], table[c] = table[c], table[j]
marked[c] = True
marked[j] = True
break
_tables.append(''.join(table))

# Tables initialized - encrypt the data.
return s.translate(_tables[offset % cnt])

s = "This is a standard length string to encrypt."
print "\nYou need to specify an offset to avoid patterns."
print "The string is:", s
print "\nEncrypted with offsets, you get:"
for i in range(5):
print "\t", encrypt(s, i)
print "\nEncrypted without offsets, you get:"
for i in range(5):
print "\t", encrypt(s)
Most XOR algorithms could benefit from using this technique. There's
never a need to XOR a byte more than once, while building the table.
From then on, it's a simple substitution problem, which the
'translate' function accomplishes quite nicely.

[Lev Elblert]

Kamilche!

I'm also puzzeld with your results. Here is a little test script,
which generates 16mb array and scans it.

import time
s = 'C'
print time.clock()
for i in range(24): s += s
print len(s)
print time.clock()
for c in s: pass
print time.clock()

When I run it on 500mg P3 the output is:

8.38095110386e-006
16777216
0.275403081843
10.9032218052

on 1.6 P4 it runs 4 seconds.

I'm new to Python and learnig it. Can't you publish the code? If the
crypto method you are using is a secret, please give the basic idea,
how did you achived such a speed.

[Peter Hansen]

Kamilche wrote:

Peter Hansen <pe***@engcorp.com> wrote in message news:<Td********************@powergate.ca>...

Besides, what you say is not possible.

But I DON'T manipulate the data byte by byte, only the encryption
tables.

[snip code]

Very interesting use of translate. I doubt anyone, certainly
not I, guessed that's what you were using. Kudos! (though it
certainly fits your description of "not secure", but it's
admittedly a step up from XOR).

-Peter

[Kamilche]

Peter Hansen <pe***@engcorp.com> wrote in message news:<gp********************@powergate.ca>...

Very interesting use of translate. I doubt anyone, certainly
not I, guessed that's what you were using. Kudos!

I'm glad you enjoyed it!
{unwraps and eats her first Python kudos bar.}

[Till Plewe]

On Wed, Jun 23, 2004 at 12:50:39AM -0400, Peter Hansen wrote:

Kamilche wrote:

....

I've written an encryption algorithm in pure Python that can process
22 megs of data a second. I know it's not secure, but it should be
enough to ward off casual hacking. Does someone know of something
speedier?

In addition to Erik and Paul's comments: if you don't specify
what machine you ran your benchmark on, the number "22MB/s" is
completely meaningless...

Besides, what you say is not possible. On my machine,
which is about a P4 2500MHz, scanning an array.array('c') with
22MB of data in it, doing nothing but reading each byte and
ignoring it, takes about 8 seconds.
So does converting the
array to a list, which is pretty much all C code.

are you sure it takes 8s?

from array import array
from time import time
f=file("xxx").read()
len(f) 22000000 s=time();A=array("c",f);t=time(); print t-s 0.0371170043945 s=time();l=list(A);t=time(); print t-s 0.681946992874

That is over ten times faster than your machine. I cannot
believe that. I use
Python 2.3.4 (#2, Jun 24 2004, 13:32:58)
[GCC 3.3.3 [FreeBSD] 20031106] on freebsd5
CPU: AMD Opteron(tm) Processor 248 (2205.01-MHz K8-class CPU)
s=time();A=array("L",f);t=time(); print t-s 0.0372500419617 r=range(22000000/8)
if 1: .... s=time()
.... for i in r:
.... A[i]^=7877854
.... t=time()
.... print t-s
....
1.80496907234

So with a 4Ghz Opteron (or better) he should be able to xor
a 22MB array in one second with some constant.

- Till

[Peter Hansen]

Till Plewe wrote:

On Wed, Jun 23, 2004 at 12:50:39AM -0400, Peter Hansen wrote:

Besides, what you say is not possible. On my machine,
which is about a P4 2500MHz, scanning an array.array('c') with
22MB of data in it, doing nothing but reading each byte and
ignoring it, takes about 8 seconds.
So does converting the
array to a list, which is pretty much all C code.

are you sure it takes 8s?

What I meant (but didn't state explicitly) was that even if you
convert the string to a list of 23068672 individual characters,
scanning the list takes about as long as scanning the array
did.

And not only did I not say all that, but what I did say I
even said wrong, since I notice I said "array to list"
instead of "string to list".

Basically, I've been totally wrong in everything I said
in this thread. ;-)

-except-this-post-ly y'rs,
Peter

[Pierre-Frédéric Caillaud]

Blowfish encryption is very secure (128 bits) and very fast (10
megabytes/s on a Celeron 400).
I used that, you could try it.

On Thu, 24 Jun 2004 14:53:14 -0400, Peter Hansen <pe***@engcorp.com> wrote:

Till Plewe wrote:

On Wed, Jun 23, 2004 at 12:50:39AM -0400, Peter Hansen wrote:

Besides, what you say is not possible. On my machine,
which is about a P4 2500MHz, scanning an array.array('c') with
22MB of data in it, doing nothing but reading each byte and
ignoring it, takes about 8 seconds. So does converting the
array to a list, which is pretty much all C code.

are you sure it takes 8s?

What I meant (but didn't state explicitly) was that even if you
convert the string to a list of 23068672 individual characters,
scanning the list takes about as long as scanning the array
did.

And not only did I not say all that, but what I did say I
even said wrong, since I notice I said "array to list"
instead of "string to list".

Basically, I've been totally wrong in everything I said
in this thread. ;-)

-except-this-post-ly y'rs,
Peter

--
Using Opera's revolutionary e-mail client: http://www.opera.com/m2/

[Ivan Voras]

Pierre-Frédéric Caillaud wrote:

Blowfish encryption is very secure (128 bits) and very fast (10
megabytes/s on a Celeron 400).
I used that, you could try it.

Is that for the python implementation or a C one? :))
--
What part of "Ph'nglui mglw'nath Cthulhu R'lyeh wgah'nagl fhtagn" don't
you understand?

[Till Plewe]

On Wed, Jun 23, 2004 at 12:56:49PM -0700, Kamilche wrote:

Peter Hansen <pe***@engcorp.com> wrote in message news:<Td********************@powergate.ca>...

Besides, what you say is not possible. On my machine,
which is about a P4 2500MHz, scanning an array.array('c') with
22MB of data in it, doing nothing but reading each byte and
ignoring it, takes about 8 seconds. So does converting the
array to a list, which is pretty much all C code. Strings
are immutable, so you can't be working with one of them...

Doing anything meaningful with real data, no matter how trivial
the algorithm, would definitely take longer.
But I DON'T manipulate the data byte by byte, only the encryption
tables.
Ah, the power of algorithms! ;-)
Take a look at this:

def encrypt(s, offset = 0, _tables = []):

.... # Tables initialized - encrypt the data.
return s.translate(_tables[offset % cnt])

I apologize for my previous post (of course I don't know when it will
actually show up). The only rather weak excuse I have is that I have
spent my entire last month using nothing but C. (Of course, the real
culprit is www.python.org for not sending your message above earlier).

In any case I still have one or two comments.

I think it is not quite "the power of algorithms" but rather making
good use of the speed of string.translate which makes your encryption
function fast. string.translate allows you to use a loop programmed in
C rather than a loop you have to program yourself in python.

If you are happy xoring then there is an even faster method: use
numarray. For me xoring a numarray.array is twice as fast as
string.translate is on the corresponding string for UInt8 arrays and
four times as fast for UInt64 arrays. (But adding the two conversions
between strings and numarrays makes the two methods comparable).

import numarray
Nin=numarray.fromstring(mystring,type=numarray.num erictypes.UInt64)
Nout=xormask^N
mysecretstrin=N.out.tostring()

- Till

[Dennis Lee Bieber]

On Fri, 25 Jun 2004 11:46:44 +0900, Till Plewe
<ti**@score.is.tsukuba.ac.jp> declaimed the following in
comp.lang.python:

On Wed, Jun 23, 2004 at 12:56:49PM -0700, Kamilche wrote:
<snip>

But I DON'T manipulate the data byte by byte, only the encryption
tables.
Ah, the power of algorithms! ;-)
Take a look at this:

def encrypt(s, offset = 0, _tables = []):

...

# Tables initialized - encrypt the data.
return s.translate(_tables[offset % cnt])

Pardon the hijacking off one response to back up to the quoted
persons comments.

I wouldn't call the above an "encryption" function -- it looks
to be nothing more than a substitution cipher. (I don't know how "cnt"
is determined, but translate will, for any given invocation, return the
same output character for any repeat occurrences of the input. A good
encryption algorithm should not do that -- ideally, "bookkeeper" will
encrypt with no repeated characters in the oo, kk, ee*e positions)

-- ================================================== ============ <
wl*****@ix.netcom.com | Wulfraed Dennis Lee Bieber KD6MOG <
wu******@dm.net | Bestiaria Support Staff <
================================================== ============ <
Home Page: <http://www.dm.net/~wulfraed/> <
Overflow Page: <http://wlfraed.home.netcom.com/> <

[Pierre-Frédéric Caillaud]

It was in C but I suppose if you pass a Pyhon string to the Blowfish
function which is in written in C, you'll get the same speed.

It's implemented in OpenSSL. This is probably accessible from Python.
Dunno which module.

Pierre-Frédéric Caillaud wrote:

Blowfish encryption is very secure (128 bits) and very fast (10
megabytes/s on a Celeron 400).
I used that, you could try it.

Is that for the python implementation or a C one? :))

--
Using Opera's revolutionary e-mail client: http://www.opera.com/m2/

[Peter Hansen]

Dennis Lee Bieber wrote:

I wouldn't call the above an "encryption" function -- it looks
to be nothing more than a substitution cipher. (I don't know how "cnt"
is determined, but translate will, for any given invocation, return the
same output character for any repeat occurrences of the input. A good
encryption algorithm should not do that -- ideally, "bookkeeper" will
encrypt with no repeated characters in the oo, kk, ee*e positions)

Kamilche *did* state clearly that it was "not secure, but it should
be enough to ward off casual hacking".

Also, if we're both using the same version of English, a substitution
cipher is still encryption, just a weak form.

See http://en.wikipedia.org/wiki/Cipher and related links, which point
out that even DES can be considered "from a sufficiently abstract
perspective", a substitution cipher on an enormously large binary
alphabet. ;-)

-Peter

[Dennis Lee Bieber]

On Fri, 25 Jun 2004 08:45:43 -0400, Peter Hansen <pe***@engcorp.com>
declaimed the following in comp.lang.python:

Kamilche *did* state clearly that it was "not secure, but it should
be enough to ward off casual hacking".
About to the level of the old "CryptoGrams" puzzles some
newspapers run (though they usually give you one starting substitution
-- but in a long missive, that type of hint may not be needed).

It would be a slow implementation in Python, but one
"improvement" I'd consider (since it looks like there are already
multiple tables): Using just a..z0..9 (ignoring case and punctuation
characters) would be to create 36 substitution tables, then use the
output of one character substitution as the index to the table for the
next character's translation.

This would avoid the "bookkeeper" type input from giving hints
in the output. The reverse (decryption) gets a tad more difficult.

-- ================================================== ============ <
wl*****@ix.netcom.com | Wulfraed Dennis Lee Bieber KD6MOG <
wu******@dm.net | Bestiaria Support Staff <
================================================== ============ <
Home Page: <http://www.dm.net/~wulfraed/> <
Overflow Page: <http://wlfraed.home.netcom.com/> <

[Peter Hansen]

Dennis Lee Bieber wrote:

On Fri, 25 Jun 2004 08:45:43 -0400, Peter Hansen <pe***@engcorp.com>
declaimed the following in comp.lang.python:

Kamilche *did* state clearly that it was "not secure, but it should
be enough to ward off casual hacking".

About to the level of the old "CryptoGrams" puzzles some
newspapers run (though they usually give you one starting substitution
-- but in a long missive, that type of hint may not be needed).

It would be a slow implementation in Python, but one
"improvement" I'd consider (since it looks like there are already
multiple tables): Using just a..z0..9 (ignoring case and punctuation
characters) would be to create 36 substitution tables, then use the
output of one character substitution as the index to the table for the
next character's translation.

This would avoid the "bookkeeper" type input from giving hints
in the output. The reverse (decryption) gets a tad more difficult.

But why would anyone want to improve the security of this approach?
If one wants casual obscurification(tm), it does just fine as-is.
If one wants real security, one would be insane to use anything
less than Blowfish, 3DES, AES and their ilk...

-Peter