473,396 Members | 2,034 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,396 software developers and data experts.

Bottleneck: easy obscurity "encryption" via xor

Hi!

I identified a bottleneck in my programs.

I just want to "encrypt" data by easy xoring. Ok - that's no
encryption at all - I know. But it's hardly readable - and that's
enough :-) Just some quick obscurity.

It turns out not to be quick at all. I really didn't expect this to be
a bottleneck, but it takes quite some time.

Here's the code:
$ cat python/EasyCrypt.py
#! /usr/bin/env python
import operator
def xorcrypt(str, salt = 255):
if salt > 255:
raise "Invalid salt! Must be < 255!"
return reduce(lambda x,y: operator.add(x, chr(y)), map(lambda char, _salt = salt: operator.xor(ord(char), _salt), str), "")
xor'ing medium sized-files takes long time. For example a 360
kByte-File takes:
$ time ./just_crypt.py Userdatan/ScanImage01.jpg > bert
real 1m52.138s
user 0m40.320s
sys 1m6.030s


on my 2.66 GHz P4 machine!

Hmmm, do you have some better implementation ideas? Some optimizing
tricks? (Besides coding in C to avoid immutable string problems)
I already took the operator module to speed up a bit - but it seems
that's not enough...

Thanks

Tino

Jul 18 '05 #1
7 2727
Tino Lange wrote:
It turns out not to be quick at all. I really didn't expect this to be
a bottleneck, but it takes quite some time.

return reduce(lambda x,y: operator.add(x, chr(y)), map(lambda char, _salt = salt: operator.xor(ord(char), _salt), str), "")


Running this on a large string builds up a huge list of ints,
that you are converting to chars and then concatenating them
together using +... this creates a HUGE number of temporary
string objects.
The usual pattern of fast string joining is:

''.join(list-of-fragments)

So first try:

return ''.join(map(lambda char, _salt = salt: chr(operator.xor(ord(char), _salt)), string))

This runs MUCH faster already.

But the version I'd recommend is:

def xorcrypt(string, salt = 255):
if salt <0 or salt> 255:
raise "Invalid salt! Must be 0<=salt<=255!"
return ''.join( [ chr(ord(c) ^ salt) for c in string ] )

because
1) salt must be 0..255 not only <=255
2) forget about map & lambda, use a list comprehension.

That implementation runs about 20 times faster than your original one;
0.11 seconds for 100 Kb source data. (python 2.3)

HTH,
--Irmen de Jong
Jul 18 '05 #2
On Wed, 30 Jul 2003 00:25:59 +0200, Irmen de Jong <irmen@-NOSPAM-REMOVETHIS-xs4all.nl> wrote:
Tino Lange wrote:
It turns out not to be quick at all. I really didn't expect this to be
a bottleneck, but it takes quite some time.
return reduce(lambda x,y: operator.add(x, chr(y)), map(lambda char, _salt = salt: operator.xor(ord(char), _salt), str), "")


Running this on a large string builds up a huge list of ints,
that you are converting to chars and then concatenating them
together using +... this creates a HUGE number of temporary
string objects.
The usual pattern of fast string joining is:

''.join(list-of-fragments)

So first try:

return ''.join(map(lambda char, _salt = salt: chr(operator.xor(ord(char), _salt)), string))

This runs MUCH faster already.

But the version I'd recommend is:

def xorcrypt(string, salt = 255):

def xorcrypt(s, salt = 255): # better name choice, even though string module may not be used if salt <0 or salt> 255:
raise "Invalid salt! Must be 0<=salt<=255!"
return ''.join( [ chr(ord(c) ^ salt) for c in string ] ) return s.translate(''.join([chr(ic^salt) for ic in xrange(256)]))
because
1) salt must be 0..255 not only <=255
2) forget about map & lambda, use a list comprehension. forget about list comprehension, use str.translate ;-)
That implementation runs about 20 times faster than your original one;
0.11 seconds for 100 Kb source data. (python 2.3)

s.translate ought to a good deal faster yet ;-)

Regards,
Bengt Richter
Jul 18 '05 #3
Tino Lange <tl*****@nexgo.de> writes:
Hmmm, do you have some better implementation ideas? Some optimizing
tricks? (Besides coding in C to avoid immutable string problems)
I already took the operator module to speed up a bit - but it seems
that's not enough...


Use the array module. See <http://www.nightsong.com/phr/crypto/p2.py>.
Jul 18 '05 #4
On Wed, Jul 30, 2003 at 12:03:06AM +0200, Tino Lange wrote:
Hi!

I identified a bottleneck in my programs.

I just want to "encrypt" data by easy xoring. Ok - that's no
encryption at all - I know. But it's hardly readable - and that's
enough :-) Just some quick obscurity.

It turns out not to be quick at all. I really didn't expect this to be
a bottleneck, but it takes quite some time.


If you want higher performance always try to use things that operate
on larger chunks. When you do things byte-by-byte you start to notice the
fact that Python is really an interpreter.

As noted by Bengt Richter xoring with a constant value can be done by
str.translate. It doesn't work for variable values, though.

This code does around 250kb/second on a Pentium 800. XORing is done 32
bits at a time. Conversion to and from character strings is done in even
larger chunks using the array module instead of using ord() and chr().

Oren
from __future__ import generators

import sha

def xor_stream_to_arrays(fin, seed, hashfunc=sha):
""" fin is a file-like object.
yields arrays that may be written to a stream """
from array import array

h = hashfunc.new(seed)
maskchunk = h.digest()
chunksize = len(maskchunk)

while True:
datachunk = fin.read(chunksize)
if len(datachunk) < chunksize:
break
yield array('l', [x^y for (x,y) in zip(
array('l', maskchunk),
array('l', datachunk))])

h.update('x')
maskchunk = h.digest()

maskchunk = maskchunk[:len(datachunk)] # trim to length of remainder

# do the rest by bytes:
yield array('b', [x^y for (x,y) in zip(
array('b', maskchunk),
array('b', datachunk))])

def xor_stream_to_stream(fin, fout, seed):
""" fin, fout are file-like objects """
for a in xor_stream_to_arrays(fin, seed):
fout.write(buffer(a))

def xor_string_to_string(s, seed):
""" gets a string, returns a string """
from cStringIO import StringIO
fin = StringIO(s)
fout = StringIO()
xor_stream_to_stream(fin, fout, seed)
return fout.getvalue()
Jul 18 '05 #5
Tino Lange <tl*****@nexgo.de> writes:
And it seems that Bengt's reciepe is the fastest. For very small strings
(<255 chars) the method irmen2 should be the best choice - it doesn' have
to pre-create the translation-table and does everything on-the-fly.


You should be able to use the array module to do the xor's 4 bytes at
a time and get a speedup over the 1-byte version. The
string.translate version is the fastest, of course, but depends on
using the same translation table for every char in the string.

If you want to encrypt in python, try the p2.py that I posted; it's
been carefully designed with good algorithms and fairly well optimized
and should give much better security than some roll-your-own method.
Jul 18 '05 #6
At 12:25 AM 7/30/2003 +0200, Irmen de Jong wrote:
Tino Lange wrote:
It turns out not to be quick at all. I really didn't expect this to be
a bottleneck, but it takes quite some time.
return reduce(lambda x,y: operator.add(x, chr(y)), map(lambda char,
_salt = salt: operator.xor(ord(char), _salt), str), "")


Running this on a large string builds up a huge list of ints,
that you are converting to chars and then concatenating them
together using +... this creates a HUGE number of temporary
string objects.
The usual pattern of fast string joining is:

''.join(list-of-fragments)

So first try:

return ''.join(map(lambda char, _salt = salt:
chr(operator.xor(ord(char), _salt)), string))

This runs MUCH faster already.

But the version I'd recommend is:

def xorcrypt(string, salt = 255):
if salt <0 or salt> 255:
raise "Invalid salt! Must be 0<=salt<=255!"
return ''.join( [ chr(ord(c) ^ salt) for c in string ] )


Great minds think alike? I came up with (independently!):
return ''.join([chr(ord(char) ^ salt) for char in txt])
I also favor comprehension because it is more readable.
because
1) salt must be 0..255 not only <=255
2) forget about map & lambda, use a list comprehension.

That implementation runs about 20 times faster than your original one;
0.11 seconds for 100 Kb source data. (python 2.3)

HTH,
--Irmen de Jong
--
http://mail.python.org/mailman/listinfo/python-list


---
Incoming mail is certified Virus Free.
Checked by AVG anti-virus system (http://www.grisoft.com).
Version: 6.0.500 / Virus Database: 298 - Release Date: 7/10/2003


Bob Gailer
bg*****@alum.rpi.edu
303 442 2625
---
Outgoing mail is certified Virus Free.
Checked by AVG anti-virus system (http://www.grisoft.com).
Version: 6.0.500 / Virus Database: 298 - Release Date: 7/10/2003

Jul 18 '05 #7
Tino Lange <tl*****@nexgo.de> writes:
Thanks! But BTW your "time-bomb" and your comments in the file tell me that
this script must not be used anymore...


Oh yeah. The code is ok, I just want to rename the function and
release it as p3.py. I haven't gotten around to that because nobody
seems to be using it. I keep forgetting. Anyway I'd appreciate it if
you don't distribute p2.py to other people with the time bomb removed,
but feel free to remove it for your own use.
Jul 18 '05 #8

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

5
by: Peter Clark | last post by:
Think of something like MyYahoo: a personalized portal with news aggregator, weather forecast, comics, etc. Now instead of visiting a web site, think of all of it being sent daily as an email. It...
10
by: joshsackett | last post by:
I am starting an encryption project for my database and I'm performing some tests on decryption speed. A lot of my application queries use a LIKE parameter in the WHERE clause. To keep from...
0
by: pigeon | last post by:
SSL only works when ms sql server has "force encryption option" turned on... If I turn that off, and try to have the client software request encryption.. I get a ssl security error msg. The...
2
by: Brian Genisio | last post by:
Hi all, So, IE has a code obfuscation method that is specific to IE. I have read that it has been easily decrypted. Has the decryption for this method been published anywhere? Is it legal to...
7
by: Dica | last post by:
i've used the sample code from msdn to create an encyption/decryption assembly as found here: http://msdn.microsoft.com/library/default.asp?url=/library/en-us/dnnetsec/html/SecNetHT10.asp i'm...
5
by: Leon | last post by:
How can I encrypted data sent across my website from web forms without using SSL? Such as on Login the user enter "EmailAddress" & "Password" and Simply Registration Form in which the user...
12
by: Paul Tillotson | last post by:
At my company we are looking at deploying clients for our client/server app outside our firewall, which will then require our postgres box to be internet-accessible. Does anyone out there have...
0
by: Ismail Fatih Yıldırım | last post by:
I modified the RSACSPSample from MSDN to try out a simple commutative encryption model using RSA encryption but when i run the progrem the first encryption command works but during the second...
2
by: Dan2kx | last post by:
Hello to all, im bored and like to waste time doing silly things that are not really neccessary... i am however tying to complete a Holiday database for my Boss... looking for sum brownie...
0
by: ryjfgjl | last post by:
In our work, we often receive Excel tables with data in the same format. If we want to analyze these data, it can be difficult to analyze them because the data is spread across multiple Excel files...
0
BarryA
by: BarryA | last post by:
What are the essential steps and strategies outlined in the Data Structures and Algorithms (DSA) roadmap for aspiring data scientists? How can individuals effectively utilize this roadmap to progress...
1
by: nemocccc | last post by:
hello, everyone, I want to develop a software for my android phone for daily needs, any suggestions?
0
by: Hystou | last post by:
There are some requirements for setting up RAID: 1. The motherboard and BIOS support RAID configuration. 2. The motherboard has 2 or more available SATA protocol SSD/HDD slots (including MSATA, M.2...
0
marktang
by: marktang | last post by:
ONU (Optical Network Unit) is one of the key components for providing high-speed Internet services. Its primary function is to act as an endpoint device located at the user's premises. However,...
0
by: Hystou | last post by:
Most computers default to English, but sometimes we require a different language, especially when relocating. Forgot to request a specific language before your computer shipped? No problem! You can...
0
jinu1996
by: jinu1996 | last post by:
In today's digital age, having a compelling online presence is paramount for businesses aiming to thrive in a competitive landscape. At the heart of this digital strategy lies an intricately woven...
0
tracyyun
by: tracyyun | last post by:
Dear forum friends, With the development of smart home technology, a variety of wireless communication protocols have appeared on the market, such as Zigbee, Z-Wave, Wi-Fi, Bluetooth, etc. Each...
0
agi2029
by: agi2029 | last post by:
Let's talk about the concept of autonomous AI software engineers and no-code agents. These AIs are designed to manage the entire lifecycle of a software development project—planning, coding, testing,...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.