I see rotor was removed for 2.4 and the docs say use an AES module
provided separately... Is there a standard module that works alike or
an AES module that works alike but with better encryption?
cheers,
reed
92  6102 
"Reed L. O'Brien" <re**@intersiege.com> writes: I see rotor was removed for 2.4 and the docs say use an AES module
provided separately... Is there a standard module that works alike or
an AES module that works alike but with better encryption?
If you mean a module in the distribution, the answer is no, for
political reasons.
There are a number of AES modules available on the net. Most are C
extension modules which means you need to compile them, and if you
want to deploy them widely, you need binaries for every target platform.
There's a few pure-Python AES implementations but they are verrry slow.
Here's something written in Python that uses the built-in sha1 module
as a crypto primitive. Its security should be better than rotor and
performance is reasonable for most applications: http://www.nightsong.com/phr/crypto/p3.py
Paul Rubin wrote: "Reed L. O'Brien" <re**@intersiege.com> writes:
I see rotor was removed for 2.4 and the docs say use an AES module
provided separately... Is there a standard module that works alike or
an AES module that works alike but with better encryption?
If you mean a module in the distribution, the answer is no, for
political reasons.
......I'm also missing the rotor module and regret that something useful
was warned about and now removed with no plugin replacement.
I had understood that this was because rotor was insecure, but you
mention politics. Are other useful modules to suffer from politics?
What exactly are/were the political reasons for rotor removal?
I might add that the source for rotormodule is still easily obtainable
and can be compiled trivially as an extension for Python-2.4. Does the
Python community take a position on the sources of removed modules?
--
Robin Becker
Robin Becker <ro***@SPAMREMOVEjessikat.fsnet.co.uk> writes: What exactly are/were the political reasons for rotor removal?
Some countries have laws about cryptography software (against some
combination of export, import, or use). The Python maintainers didn't
want to deal with imagined legal hassles that might develop from
including good crypto functions in the distribution. Then it became
obvious that the same imagined hassles could also befall the rotor
module, so that was removed.
I might add that the source for rotormodule is still easily obtainable
and can be compiled trivially as an extension for Python-2.4. Does the
Python community take a position on the sources of removed modules?
Those are still free to distribute, but I'd advise against doing so
with the rotor module unless you absolutely need it for some
interoperability purpose. Otherwise, it's insecure and should not be
used. The routine I posted was intended as a straightforward
replacement for the rotor module that doesn't depend on C compilers
and is reasonably secure. If you don't mind using C extensions,
there's various AES modules available, plus fancier packages like
mxCrypto.
Robin Becker <ro***@SPAMREMOVEjessikat.fsnet.co.uk> wrote: Paul Rubin wrote: "Reed L. O'Brien" <re**@intersiege.com> writes:
I see rotor was removed for 2.4 and the docs say use an AES module
provided separately... Is there a standard module that works alike or
an AES module that works alike but with better encryption?
If you mean a module in the distribution, the answer is no, for
political reasons.
.....I'm also missing the rotor module and regret that something useful
was warned about and now removed with no plugin replacement.
I had understood that this was because rotor was insecure, but you
mention politics. Are other useful modules to suffer from politics?
What exactly are/were the political reasons for rotor removal?
Presumably he is talking about crypo-export rules. In the past strong
cryptography has been treated as munitions, and as such exporting it
(especially from the USA) could have got you into very serious
trouble.
However I believe those restrictions have been lifted (the cat having
been let out of the bag somewhat ;-), and its easy to do this for open
source encryption software. A wade through http://www.bxa.doc.gov/Encryption/enc.htm
Might be interesting.
A case in point: the linux 2.6 kernel is chock full of crypo and comes
with implementations of AES, ARC4, Blowfish, Cast5+6, DES, Serpent,
Twofish, TEA, etc. The linux kernel+source surely goes everywhere
python does so I don't think adding strong crypto modules to python is
a problem now-a-days.
AES in the core python library would be very useful and it would
discourage people from writing their own crypto routines (looks easy
but isn't!)
--
Nick Craig-Wood <ni**@craig-wood.com> -- http://www.craig-wood.com/nick
Robin Becker <ro***@SPAMREMOVEjessikat.fsnet.co.uk> writes: Paul Rubin wrote:
.....I'm also missing the rotor module and regret that something useful
was warned about and now removed with no plugin replacement.
Hm, yes. Here is a (rather slow) replacement:
"""This module is derived from Modules/rotormodule.c and translated
into Python. I have appended the Copyright by Lance Ellinghouse
below. The rotor module has been removed from the Python 2.4
distribution because
the rotor module uses an insecure algorithm and is deprecated.
================================================== ============
Of course, this does still hold. However, I think this module might
be used and adapted for demonstration purposes and might help some
poor users who have encrypted (or obfuscated) some old stuff with
the rotor module and have no access to older Python versions any
more.
Documentation can be found in
Python Library Reference, Guido van Rossum, Fred L. Drake, Jr., editor,
PythonLabs, Release 2.3.4 May 20, 2004
<http://www.python.org/doc/2.3.4/lib/module-rotor.html>
################################################## ###################
Copyright 1994 by Lance Ellinghouse,
Cathedral City, California Republic, United States of America.
All Rights Reserved
Permission to use, copy, modify, and distribute this software and its
documentation for any purpose and without fee is hereby granted,
provided that the above copyright notice appear in all copies and that
both that copyright notice and this permission notice appear in
supporting documentation, and that the name of Lance Ellinghouse
not be used in advertising or publicity pertaining to distribution
of the software without specific, written prior permission.
LANCE ELLINGHOUSE DISCLAIMS ALL WARRANTIES WITH REGARD TO
THIS SOFTWARE, INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY AND
FITNESS, IN NO EVENT SHALL LANCE ELLINGHOUSE BE LIABLE FOR ANY SPECIAL,
INDIRECT OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING
FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT,
NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION
WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
################################################## ###################
"""
class newrotor(object):
def __init__(self, key, n_rotors=6):
self.n_rotors = n_rotors
self.setkey(key)
def setkey(self, key):
self.key = key
self.rotors = None
self.positions = [None, None]
def encrypt(self, buf):
# Reset (encrypt) positions and encryptmore
self.positions[0] = None
return self.cryptmore(buf, 0)
def encryptmore(self, buf):
return self.cryptmore(buf, 0)
def decrypt(self, buf):
# Reset decrypt positions and decryptmore
self.positions[1] = None
return self.cryptmore(buf, 1)
def decryptmore(self, buf):
return self.cryptmore(buf, 1)
def cryptmore(self, buf, do_decrypt):
size, nr, rotors, pos = self.get_rotors(do_decrypt)
outbuf = []
append = outbuf.append
for c in map(ord, buf):
if do_decrypt:
# Apply decrypt rotors and xor in reverse order
for i in xrange(nr-1,-1,-1):
c = pos[i] ^ rotors[i][c]
else:
# Apply xor and ecrypt rotors
for i in xrange(nr):
c = rotors[i][c ^ pos[i]]
append(c)
# Advance rotors, i.e. add the (fixed) rotor increments to the
# variable rotor positions with carry.
# Note: In the C-implementation, the result of the carry addition
# was stored to an "unsigned char". Hence the next carry
# is lost if pos[i] == size-1 and pnew >= size.
# Masking with 0xff simulates this behavior.
#
pnew = 0 # (pnew >= size) works as "carry bit"
for i in xrange(nr):
pnew = ((pos[i] + (pnew >= size)) & 0xff) + rotors[i][size]
pos[i] = pnew % size
return ''.join(map(chr, outbuf))
def get_rotors(self, do_decrypt):
# Return a tuple (size, nr, rotors, positions) where
# - size is the rotor size (== 256, because of 8-bit bytes)
# - nr is the number of rotors.
# - rotors is a tuple of nr encrypt or nr decrypt rotors
# for do_decrypt == 0 or do_decrypt == 1 respectively.
# - postions is a list of nr "rotor positions".
#
# The rotors represent the static aspect of the rotor machine which
# is initially computed from key and fixed during en/decryption.
# A rotor is a random permutation of range(size) extended
# by an "increment value" in range(size).
#
# The followng statements hold for a tuple of encrypt rotors E and
# and the corresponding tuple of decrypt rotors D.
#
# D[i][E[i][j]] == j for i in range(nr) for j in range(size)
#
# E[i][D[i][j]] == j for i in range(nr) for j in range(size)
#
# This means that the corresponding rotors E[i] and D[i] are
# inverse permutations.
# The increments are equal for the corresponding encrypt and
# decrypt rotors and have an odd value:
#
# D[i][size] == E[i][size] and E[i][size] == 1 mod 2 and
# 0 < E[i][size] < size for i in range(nr)
#
# The position vector represents the dynamic aspect.
# It changes after each en/decrypted character (the rotors
# are "advanced"). The initial position vector is also computed
# from the key
#
nr = self.n_rotors
rotors = self.rotors
positions = self.positions[do_decrypt]
if positions is None:
if rotors:
positions = list(rotors[3])
else:
# Generate identity permutation for 8-bit bytes plus an
# (unused) increment value
self.size = size = 256
id_rotor = range(size+1)
# Generate nr "random" initial positions and "random"
# en/decrypt rotors from id_rotor.
#
rand = random_func(self.key)
E = []
D = []
positions = []
for i in xrange(nr):
i = size
positions.append(rand(i))
erotor = id_rotor[:]
drotor = id_rotor[:]
drotor[i] = erotor[i] = 1 + 2*rand(i/2) # increment
while i > 1:
r = rand(i)
i -= 1
er = erotor[r]
erotor[r] = erotor[i]
erotor[i] = er
drotor[er] = i
drotor[erotor[0]] = 0
E.append(tuple(erotor))
D.append(tuple(drotor))
self.rotors = rotors = (
tuple(E), tuple(D), size, tuple(positions))
self.positions[do_decrypt] = positions
return rotors[2], nr, rotors[do_decrypt], positions
def random_func(key):
# Generate a random number generator that is "seeded" from key.
# This algorithm is copied from Python2.3 randommodule.c.
#
mask = 0xffff
x=995
y=576
z=767
for c in map(ord, key):
x = (((x<<3 | x>>13) + c) & mask)
y = (((y<<3 | y>>13) ^ c) & mask)
z = (((z<<3 | z>>13) - c) & mask)
# Emulate (unintended?) cast to short
maxpos = mask >> 1
mask += 1
if x > maxpos: x -= mask
if y > maxpos: y -= mask
if z > maxpos: z -= mask
y |= 1 # avoid very bad seed, why not for x and z too?
# Oh, dear, for compatibility, we must evaluate the first seed transition
# the hard way, later it becomes much simpler
x = 171 * (x % 177) - 2 * (x/177)
y = 172 * (y % 176) - 35 * (y/176)
z = 170 * (z % 178) - 63 * (z/178)
if x < 0: x += 30269
if y < 0: y += 30307
if z < 0: z += 30323
# At least all values are > 0 by now but may be greater than expected ..
def rand(n, seed=[(x, y, z)]):
# Return a random number 0 <= r < n
#
x, y, z = seed[0]
seed[0] = ((171*x) % 30269, (172*y) % 30307, (170*z) % 30323)
return int((x/30269.0 + y/30307.0 + z/30323.0) * n) % n
# Original code was like this:
# from math import floor
# val = x/30269.0 + y/30307.0 + z/30323.0
# val = val - floor(val)
# if val >= 1.0:
# val = 0.0
# n = int(val*n) % n
return rand
Thanks very much to all the folks who replied to this. I've been loking
for pure python encryption routines for a while and found this thread
very useful.
There is also a pure python des implementation in 'aps' an NTLM proxy
server.
Regards,
Fuzzy http://www.voidspace.org.uk/python/index.shtml
Paul Rubin wrote: Robin Becker <ro***@SPAMREMOVEjessikat.fsnet.co.uk> writes:What exactly are/were the political reasons for rotor removal? Some countries have laws about cryptography software (against some
combination of export, import, or use).
I understand this to be true. Since I am trying to address encryption
in the zipfile module, and I know you actually follow a bit of the
encryption stuff, can you answer a question or two for me?
The Python maintainers didn't want to deal with imagined legal hassles
that might develop from including good crypto functions in the
distribution. Then it became obvious that the same imagined hassles
could also befall the rotor module, so that was removed.
Are you saying these hassles are, in fact, imaginary rather than real?
Is this because you feel python is over-cautious about the USA, or is
this an opinion on "essentially all countries?" This is not a quibble
or a kvetch; I would like your understanding about the world legal
state of dealing with encryption (which, I assure you, I won't take as
definitive). I would hate to think someone in, for example, the UAE,
was busted for downloading or republishing python "out-of-the-box."
Don't get me wrong, I'd love the answer to be "sure its fine," but my
current plans are to provide a way to connect a crypto package to
zipfile without providing any such package myself.
--Scott David Daniels Sc***********@Acm.Org
Scott David Daniels <Sc***********@Acm.Org> writes: I understand this to be true. Since I am trying to address encryption
in the zipfile module, and I know you actually follow a bit of the
encryption stuff, can you answer a question or two for me?
Sure, I can try, so go ahead. There's more crypto expertise in
sci.crypt though.
Zipfile encryption is totally incompatible with the rotor module, by
the way, and traditionally it didn't use AES. There are a couple of
replacements for the traditional method that do use AES but that I
think are somewhat incompatible with each other.
> The Python maintainers didn't want to deal with imagined legal hassles
> that might develop from including good crypto functions in the
> distribution. Then it became obvious that the same imagined hassles
could also befall the rotor module, so that was removed.
Are you saying these hassles are, in fact, imaginary rather than real?
Well, I didn't want to say that the hassles were real, but I wasn't
trying to insinuate quite as much as it may have sounded. Like, I
don't drive my car at 100 mph on Main Street because I can imagine
what would happen and it's not pretty. The imagined carnage is a good
enough reason not to drive that way. However, I do feel that the
Python distributors are being over-cautious, see below.
Is this because you feel python is over-cautious about the USA, or is
this an opinion on "essentially all countries?" This is not a quibble
or a kvetch; I would like your understanding about the world legal
state of dealing with encryption (which, I assure you, I won't take as
definitive). I would hate to think someone in, for example, the UAE,
was busted for downloading or republishing python "out-of-the-box."
I think the Python maintainers were more concerned about that UAE
situation. However, the most widely deployed encryption software is
the SSL stack in just about every web browser (MSIE, Firefox, etc.)
and I'm sure lots of people are using those browsers in the UAE. The
Mozilla foundation isn't hestitating to ship the encryption as far as
I can tell.
See http://www.bxa.doc.gov/Encryption for the USA rules. Basically
for a free encryption program on the web, you're supposed to notify
the Dept. of Commerce by sending them an email when you publish it,
telling them where they can get it (address is on that site). As far
as anyone can tell, the DOC never does anything with those emails.
The rules are more complicated for nonpublished commercial programs,
crypto hardware, etc.
Don't get me wrong, I'd love the answer to be "sure its fine," but my
current plans are to provide a way to connect a crypto package to
zipfile without providing any such package myself.
I'd say provide a package if you can, unless you have realistic
concern about getting in trouble.
Nick Craig-Wood wrote: Robin Becker <ro***@SPAMREMOVEjessikat.fsnet.co.uk> wrote:
Paul Rubin wrote:
"Reed L. O'Brien" <re**@intersiege.com> writes:
I see rotor was removed for 2.4 and the docs say use an AES module
provided separately... Is there a standard module that works alike or
an AES module that works alike but with better encryption?
If you mean a module in the distribution, the answer is no, for
political reasons.
.....I'm also missing the rotor module and regret that something useful
was warned about and now removed with no plugin replacement.
I had understood that this was because rotor was insecure, but you
mention politics. Are other useful modules to suffer from politics?
.... Presumably he is talking about crypo-export rules. In the past strong
cryptography has been treated as munitions, and as such exporting it
(especially from the USA) could have got you into very serious
trouble.
well since rotor is a german (1930's) invention it is a bit late for
Amricans (Hollywood notwithstanding) to be worried about its export
However I believe those restrictions have been lifted (the cat having
been let out of the bag somewhat ;-), and its easy to do this for open
source encryption software. A wade through
--
Robin Becker
Robin Becker wrote: Presumably he is talking about crypo-export rules. In the past strong
cryptography has been treated as munitions, and as such exporting it
(especially from the USA) could have got you into very serious
trouble.
So Python is an American Language and must obey American Law. Luckily I
seem to have escaped that fate.
--
Robin Becker
Robin Becker <ro***@SPAMREMOVEjessikat.fsnet.co.uk> writes: Presumably he is talking about crypo-export rules. In the past strong
cryptography has been treated as munitions, and as such exporting it
(especially from the USA) could have got you into very serious
trouble.
well since rotor is a german (1930's) invention it is a bit late for
Amricans (Hollywood notwithstanding) to be worried about its export
1. I think the concern was not about exporting from the US, but rather
importing into some countries that restrict the use of crypto. But
the cat is out of the bag on that one too. Just about every web
browser includes an SSL stack and those browsers are in use
everywhere.
2. It's irrelevant for the purpose of export rules how old an
invention is or where it was invented. I don't know where machine
guns were invented, but they're at least 100 years old and you can't
export those without a license either. My gripe with the crypto rules
are not about the age or nationality of crypto rotor machines (rotor
is not a clone of the Enigma by the way; it just operates on related
principles) but rather on the control of information in general.
Exporting a machine gun is much different from publishing a
description of one. Software is just a precise type of description.
On 19 Jan 2005 17:09:19 -0800, Paul Rubin <http://ph****@NOSPAM.invalid> wrote: Robin Becker <ro***@SPAMREMOVEjessikat.fsnet.co.uk> writes: > Presumably he is talking about crypo-export rules. In the past strong
> cryptography has been treated as munitions, and as such exporting it
> (especially from the USA) could have got you into very serious
> trouble.
well since rotor is a german (1930's) invention it is a bit late for
Amricans (Hollywood notwithstanding) to be worried about its export
1. I think the concern was not about exporting from the US, but rather
importing into some countries that restrict the use of crypto. But
the cat is out of the bag on that one too. Just about every web
browser includes an SSL stack and those browsers are in use
everywhere.
Isn't the SSL dependent on OS or at least shared lib support?
Wasn't there a default 40-bit version that was ok (weak), but you had
to declare yourself US resident to download 128-bit support?
I dimly recall encountering this sort of thing installing Netscape
a long time ago, I think. Is 128 just standard now? And now that 128
is wobbly(?), will the same thing be replayed with the ante upped?
2. It's irrelevant for the purpose of export rules how old an
invention is or where it was invented. I don't know where machine
guns were invented, but they're at least 100 years old and you can't
export those without a license either. My gripe with the crypto rules
are not about the age or nationality of crypto rotor machines (rotor
is not a clone of the Enigma by the way; it just operates on related
principles) but rather on the control of information in general.
I can easily conceive of information that I'd rather not see publicized
without severe access controls. But in general I do believe in open sharing
of free information as the most productive for everyone.
Exporting a machine gun is much different from publishing a
description of one. Software is just a precise type of description.
Yeah, but ... ;-)
Regards,
Bengt Richter bo**@oz.net (Bengt Richter) writes: Isn't the SSL dependent on OS or at least shared lib support?
Firefox has its own implementation. IE uses wininet which is built
Windows. I'm not aware of any no-crypto version of Windows but even
if there is one, the US version is running, like, everywhere.
Wasn't there a default 40-bit version that was ok (weak), but you had
to declare yourself US resident to download 128-bit support?
That was years ago. The regulations changed since then, so they all
have 128 bits now.
I dimly recall encountering this sort of thing installing Netscape
a long time ago, I think. Is 128 just standard now? And now that 128
is wobbly(?), will the same thing be replayed with the ante upped?
128 isn't wobbly. It will be a long time before any machine can do
2**128 operations to break a message.
Paul Rubin schrieb: Wasn't there a default 40-bit version that was ok (weak), but you had
to declare yourself US resident to download 128-bit support?
That was years ago. The regulations changed since then, so they all
have 128 bits now.
Perhaps the NSA has found a way to handle 128bit in the meantime.
But this is unlikely because there is no export regulation to ban
512bit as far as I know :)
--
-------------------------------------------------------------------
Peter Maas, M+R Infosysteme, D-52070 Aachen, Tel +49-241-93878-0
E-mail 'cGV0ZXIubWFhc0BtcGx1c3IuZGU=\n'.decode('base64')
-------------------------------------------------------------------
Peter Maas wrote: Paul Rubin schrieb:
Wasn't there a default 40-bit version that was ok (weak), but you had
to declare yourself US resident to download 128-bit support?
That was years ago. The regulations changed since then, so they all
have 128 bits now.
Perhaps the NSA has found a way to handle 128bit in the meantime.
But this is unlikely because there is no export regulation to ban
512bit as far as I know :)
Apparently factorization based crypto is on the way out anyhow (as an
article in Scientific American is reported to claim). http://www.sciam.com/article.cfm?cha...7F0000&ref=rdf
-can't wait to get my quantum computer-ly yrs-
Robin Becker
Robin Becker <ro***@SPAMREMOVEjessikat.fsnet.co.uk> writes: Apparently factorization based crypto is on the way out anyhow (as an
article in Scientific American is reported to claim).
I haven't seen that SA article but I saw the Slashdot blurb. They
have confused "quantum cryptography" with quantum computation, when
they are entirely different things. Quantum cryptography (basically
communicating a message over an optical fiber in such a way that any
attempt to eavesdrop is supposed destroy the readability of the
message) has been done over quite long distances, 10's of km or even
more. Quantum computation is mostly a theoretical speculation. The
largest quantum computer ever built held seven bits, and factored the
number 15 into its factors 3 and 5. Building larger ones seems to
have complexity exponential in the number of bits, which is not too
much better than using an exponential-time algorithm on a conventional
computer. It's not even known in theory whether quantum computing is
possible on a significant scale. There are just some theorems about
what properties such a computer would have, if it can exist. One of
them, however, is being able to factor in P-time, and that caused
lots of excitement.
Paul Rubin wrote: Some countries have laws about cryptography software (against some
combination of export, import, or use). The Python maintainers didn't
want to deal with imagined legal hassles that might develop from
including good crypto functions in the distribution. Then it became
obvious that the same imagined hassles could also befall the rotor
module, so that was removed.
Do you know this for a fact? The PSF does comply with the U.S. American
export procedures for crypto code, and reports the crypto code in
Python appropriately to BXA.
Regards,
Martin
"Martin v. Löwis" <ma****@v.loewis.de> writes: Some countries have laws about cryptography software (against some
combination of export, import, or use). The Python maintainers didn't
want to deal with imagined legal hassles that might develop from
including good crypto functions in the distribution. Then it became
obvious that the same imagined hassles could also befall the rotor
module, so that was removed.
Do you know this for a fact?
I'm going by newsgroup messages from around the time that I was
proposing to put together a standard block cipher module for Python.
The PSF does comply with the U.S. American export procedures for
crypto code, and reports the crypto code in Python appropriately to BXA.
Since rotor was removed, there is no crypto code in Python that needs
reporting. ph*@localhost.localdomain wrote: Do you know this for a fact?
I'm going by newsgroup messages from around the time that I was
proposing to put together a standard block cipher module for Python.
Ah, newsgroup messages. Anybody could respond, whether they have insight
or not. The PSF does comply with the U.S. American export procedures for
crypto code, and reports the crypto code in Python appropriately to BXA.
Since rotor was removed, there is no crypto code in Python that needs
reporting.
We have released different versions of Python in the past. For Python
2.2, a report about the rotor module was sent to BXA.
Regards,
Martin
Robin Becker schreef: well since rotor is a german (1930's) invention
And AES is a Belgian invention... ;-)
it is a bit late for
Amricans (Hollywood notwithstanding) to be worried about its export
--
JanC
"Be strict when sending and tolerant when receiving."
RFC 1958 - Architectural Principles of the Internet - section 3.9
"Martin v. Löwis" <ma****@v.loewis.de> writes: I'm going by newsgroup messages from around the time that I was
proposing to put together a standard block cipher module for Python.
Ah, newsgroup messages. Anybody could respond, whether they have insight
or not.
Here's the message I had in mind: http://groups-beta.google.com/group/...fbec9f4d7300cc
It came from someone who follows Python crypto issues as closely as
anyone, and refers to a consensus on python-dev. I'm not on python-dev
myself but I feel that the author of that message is credible and is
not just "anyone".
Paul Rubin <http> wrote: Here's the message I had in mind:
http://groups-beta.google.com/group/...fbec9f4d7300cc
It came from someone who follows Python crypto issues as closely as
anyone, and refers to a consensus on python-dev. I'm not on python-dev
myself but I feel that the author of that message is credible and is
not just "anyone".
And here is the relevant part...
"A.M. Kuchling" wrote: On Fri, 27 Feb 2004 11:01:08 -0800 Trevor Perrin wrote: Are you and Paul still looking at adding ciphers to stdlib? That would
make me really, really happy :-)....
No, unfortunately; the python-dev consensus was that encryption raised
export control issues, and the existing rotor module is now on its way to
being removed.
I'm sure thats wrong now-a-days. Here are some examples of open
source software with strong crypto
Linux kernel: http://www.kernel.org/
GNU crypto project: http://www.gnu.org/software/gnu-crypto/
TryCrypt: http://truecrypt.sourceforge.net/
OpenSSL: http://www.openssl.org/
AEScrypt: http://aescrypt.sourceforge.net/
<lots more here!>
Note that some of these are being worked on at sourceforge just like
python.
Surely it must be possible to add a few simple crypto modules to
python?
That said
a) IANAL
b) 'apt-get install python-crypto' works for me ;-)
--
Nick Craig-Wood <ni**@craig-wood.com> -- http://www.craig-wood.com/nick
Nick Craig-Wood <ni**@craig-wood.com> writes: No, unfortunately; the python-dev consensus was that encryption raised
export control issues, and the existing rotor module is now on its way to
being removed.
I'm sure thats wrong now-a-days. Here are some examples of open
source software with strong crypto
There's tons of such examples, but python-dev apparently reached
consensus that the Python maintainers were less willing than the
maintainers of those other packages to deal with those issues.
You're right that this specifically says export control. I'm now
thinking I saw some other messages, again from knowledgeable posters,
saying that there was a bigger concern that including crypto in the
distribution could make trouble for users in countries where having
crypto at all was restricted. I'll see if I can find those.
Martin, do you know more about this? I remember being disappointed
about the decisions since I had done some work on a new block cipher
API and I had wanted to submit an implementation to the distro. But
when I heard there was no hope of including it, I stopped working on
it. If there's an interest in it again, I can do some more with it.
On 22 Jan 2005 04:50:30 -0800,
Paul Rubin <http> wrote: Martin, do you know more about this? I remember being disappointed
about the decisions since I had done some work on a new block cipher
It was discussed in this thread: http://mail.python.org/pipermail/pyt...il/034959.html
Guido and M.-A. Lemburg were leaning against including crypto; everyone else
was positive. But Guido's the BDFL, so I interpreted his vote as being the
critical one.
--amk
Paul Rubin wrote: Martin, do you know more about this? I remember being disappointed
about the decisions since I had done some work on a new block cipher
API and I had wanted to submit an implementation to the distro. But
when I heard there was no hope of including it, I stopped working on
it.
"I'll only work on stuff if I'm sure it's going right into the core" isn't exactly
a great way to develop good Python software. I recommend the "would
anyone except me have any use for this?" approach.
</F>
"Fredrik Lundh" <fr*****@pythonware.com> writes: "I'll only work on stuff if I'm sure it's going right into the core"
isn't exactly a great way to develop good Python software. I
recommend the "would anyone except me have any use for this?"
approach.
1. Crypto is an important "battery" for many security applications.
As a crypto activist I like to spread crypto, and I therefore think it
would be useful if crypto were in the core. That is the reason I was
willing to do the work of writing a suitable module. To have it go
into the core and further my goal of spreading crypto. That's as good
a reason as any to write a crypto module.
2. "Would anyone except me have any use for this?" shows a lack of
understanding of how Python is used. Some users (call them
"application users" or AU's) use Python to run Python applications for
whatever purpose. Some other users (call them "developers") use
Python to develop applications that are intended to be run by AU's.
Now we're talking about an extension module written in C. There is no
way to write AES for Python any other way and still have reasonable
perfomance.
Modules written in C and distributed separately from the core are a
pain in the neck to download and install. You need compilers, which
not everyone has access to. AU's often use Windows, which doesn't
come with any compilers, so many AU's have no compilers. Developers
generally have access to compilers for the platforms they develop on,
but usually won't have compilers for every target platform that every
AU in their audience might want to run their app on. Even AU's with
compilers need to be able to install extension modules before they can
run them, which isn't always possible, for example if they're using
Python at a web hosting service.
What I'm getting at here is that C modules are considerably more
useful to AU's if they're in the core than if they're outside it, and
the effect is even larger for developers. For developers, extension
modules are practically useless unless they're in the core. Depending
on extension modules that have to be installed by the AU severely
limits the audience for the developer's app.
The module we're discussing was intended for developers. "Would
anyone except me have any use for this, [even if it doesn't go in the
core]?" is a bizarre question. The whole purpose of the module was to
let developers ship Python crypto apps that don't making the AU load
external C modules. If it's not in the core, it doesn't meet its
usefulness criterion. Your proposed question amounts to asking "is
this worth doing even if its usefulness is severely limited?". I
aleady asked myself that question and the answer was no. I was only
interested in the higher-usefulness case, which means putting the
module in the core. I don't see anything unreasonable about that. I
can only work on a limited number of things, so I pick the most useful
ones.
"A.M. Kuchling" <am*@amk.ca> writes: It was discussed in this thread:
http://mail.python.org/pipermail/pyt...il/034959.html
Guido and M.-A. Lemburg were leaning against including crypto; everyone else
was positive. But Guido's the BDFL, so I interpreted his vote as being the
critical one.
That's interesting, so it's an export issue after all. But export
from the US is handled by sending an email to the DOC, and Martin
mentions that's already been done for some Python modules. I had been
under the impression was that the concern was over causing possible
problems for users in some destination countries, and possibly having
to maintain separate distros for the sake of users like that. But
maybe I was wrong about that.
Paul Rubin wrote: 2. "Would anyone except me have any use for this?" shows a lack of
understanding of how Python is used. Some users (call them
"application users" or AU's) use Python to run Python applications for
whatever purpose. Some other users (call them "developers") use
Python to develop applications that are intended to be run by AU's.
"lack of understanding of how Python is used"
wonderful. I'm going to make a poster of your post, and put it on my
office wall.
</F>
Paul Rubin wrote: There's tons of such examples, but python-dev apparently reached
consensus that the Python maintainers were less willing than the
maintainers of those other packages to deal with those issues.
As Andrew says, it is not apparent that there was consensus.
Martin, do you know more about this?
I'm pretty certain that we (the PSF) sent a message to BXA, reporting
the rotor module. While I can't find out exactly when this happened
right now, the board meeting on 2002-04-09 decided that this should
happen, see http://python.org/psf/records/board/...002-04-09.html
Regards,
Martin
Paul Rubin wrote: If he understood how Python is actually used, he'd understand that any
C module is a lot more useful in the core than out of it.
This is non-sense. I have been distributing C modules outside
the core for quite some time now, and I found that the modules
are quite useful. distutils makes it really easy for anybody
to use them.
There are already tons of
3rd party crypto modules outside the core, and the module I was
writing wouldn't add anything useful to those.
Why do you think these are not part of the core? It's not because
they contain crypto code, or because they had been rejected. They
are primarily not included in Python because they have not been
contributed to Python, yet.
If they were contributed, a number of things still would need to
be considered, e.g. what is the size of the code, including libraries,
is the contributor really the author, is the code likely going
to be maintained, and so on. However, it never got that far.
Where does Frederik get
off lecturing me about wanting to get a module into the core, when
Guido had invited me to do precisely that with that very module?
I know that *I* am very opposed to any contribution of a larger
module that has not seen any real users, yet. So if the module
was primarily written to be included in the core, I would initially
reject it for that very reason. After one year or so in its life,
and a recognizable user base, inclusion can be considered.
I did release a replacement for the rotor module that's written in
Python, which means it's reasonably useable without being in the core.
However, while its security should be ok, to provide reasonable
performance it had to use a nonstandard algorithm and therefore isn't
good for interoperating with anything. To be both acceptably fast and
interoperable with other applications, a C module is needed.
I fail to see the problem you seem to have with C modules. They are
very easy to use if written properly.
Regards,
Martin
I also feel the lack of a standard cryptography module in the core...
even a *basic* one. At least rotor provided that, before it was
deprecated. I (along with many other python users) write CGIs where the
only extension modules that I can use are either pure python, or ones
my web hoster is willing to install. Luckily my hoster will install
modules without too much fuss - others aren't so lucky. Cryptography is
a pretty 'standard' requirement these days.. and IMHO ought to be in
the 'standard library'.
Without commenting (or reading properly) on this exchange... it does
seem that many of '<F>'s contributions have been very bad tempered
recently. Hmmm.....
Regards,
Fuzzy http://www.voidspace.org.uk/python/index.shtml
"Fuzzyman" <fu******@gmail.com> writes: I also feel the lack of a standard cryptography module in the core...
even a *basic* one. At least rotor provided that, before it was deprecated.
Rotor was insecure and really shouldn't have been used. If you need
crypto in pure Python, try this: http://www.nightsong.com/phr/crypto/p3.py
It's a nonstandard algorithm, but so was rotor. Its security should
be much better than rotor's, and its speed should be tolerable for
most purposes.
[Note: this is a 2nd attempt at posting reply to Martin's message,
since the first one didn't reach the server. It's a rewrite from memory
but says about the same thing as the other attempt. --Paul]
"Martin v. Löwis" <ma****@v.loewis.de> writes: Paul Rubin wrote: If he understood how Python is actually used, he'd understand that any
C module is a lot more useful in the core than out of it.
This is non-sense. I have been distributing C modules outside the
core for quite some time now, and I found that the modules are quite
useful. distutils makes it really easy for anybody to use them.
Maybe we're not thinking about the same problems. Say I'm an app
writer and I want to use one of your modules. My development
environment is GNU/Linux, and I want to ship a self-contained app that
anyone can run without having to download additional components. That
includes people wanting to run my app on Windows, Macintosh, Amiga,
and various other Python target platforms that I don't have compilers
for. How do I do it?
I'm sure your modules are excellent but as an app writer, I feel I
must try to avoid needing them, in favor of modules that are already
in the core, even if it means more work for me or worse functionality
in my app, just for the sake of reducing installation headaches for my
target audience. So, if your modules are generally useful, I hope you
will try to get them into the core.
There are already tons of 3rd party crypto modules outside the
core, and the module I was writing wouldn't add anything useful to those.
Why do you think these are not part of the core? It's not because
they contain crypto code, or because they had been rejected. They
are primarily not included in Python because they have not been
contributed to Python, yet.
I can't speak for the authors but I'd personally say that none of
those old modules are really suitable for the core on technical
grounds. They're too fancy and specialized, or they depend on
external libraries like OpenSSL, or they're written mostly to support
some specific application and don't present a simple, general-purpose
interface like a core module should have. Maybe the authors felt the
same way and chose not to submit them. Or maybe the authors had other
reasons, such as licensing priorities that conflict with the Python
license.
The module I'm discussing would simply implement the FIPS standard
block ciphers (AES and DES) and the FIPS operating modes. These are
the basic building blocks that I feel are missing from the core. They
should be enough to fill the crypto needs of most applications.
If they were contributed, a number of things still would need to
be considered, e.g. what is the size of the code, including libraries,
is the contributor really the author, is the code likely going
to be maintained, and so on. However, it never got that far.
The module we're talking about wouldn't have had any of those problems.
I know that *I* am very opposed to any contribution of a larger
module that has not seen any real users, yet.
We're not talking about a "larger module", we're talking about a
module that implements the basic AES and DES block ciphers (similar to
how the sha module implements the basic SHA-1 hash algorithm) and
wraps them with some well-defined FIPS operating modes (similar to how
the hmac module wraps the sha module to compute RFC 2104 hmac
authentication codes). This stuff isn't rocket science. It's just
straightforward implementation of modes and algorithms that go back to
the 1970's, are the subject of national and international standards,
and are already in use in thousands of non-Python applications all
over the world.
Also, I've already released a pure-Python implementation of the
interface, suitable for application testing but too slow for real use.
I tried rather hard to design a convenient and general interface that
I feel was an improvement over PEP 272. I don't know if anyone except
me is using it, but several people including Andrew (author of PEP
272) have written favorably about it based on reading the
specification. The spec was designed from the beginning to fill the
needs of the core. If I were writing it as a non-core module I would
have included more stuff.
So if the module was primarily written to be included in the core, I
would initially reject it for that very reason. After one year or so
in its life, and a recognizable user base, inclusion can be
considered.
That makes no sense; improving the core is a perfectly valid reason to
write something. If someone wrote a straightforward patch that made
the Python interpreter 3x faster, would you still want to wait a year
before thinking about including it? I certainly believe that every
contribution needs code review and user testing before entering wide
release, but that doesn't seem to be what you're getting at.
I fail to see the problem you seem to have with C modules. They are
very easy to use if written properly.
Well again, maybe we're not talking about the same thing, or perhaps I
just need more explanation of what you mean. Suppose I want to write
an "application" that tells people the SHA-1 checksum of their name.
I write 3 lines of Python using the built-in sha module:
import sha
name = raw_input('Enter your name: ')
print 'Your hash is', sha.new(name).hexdigest()
I release this "app" and presto, every Python user on every OS
platform can run it with no fuss. That includes CGI authors who
aren't allowed to install C modules at all, without getting their
hosting service to do it. How do I do the same thing if there's no
built-in sha module and all I have is sha.c?
To me, the whole Python notion of "batteries included" expresses
precisely the idea that I'm trying to describe. Commonly-used
functions belong in the core, because making people download or
install them separately is a pain in the neck.
<ph*@localhost.localdomain> wrote: Maybe we're not thinking about the same problems. Say I'm an app
writer and I want to use one of your modules. My development
environment is GNU/Linux, and I want to ship a self-contained app that
anyone can run without having to download additional components. That
includes people wanting to run my app on Windows, Macintosh, Amiga,
and various other Python target platforms that I don't have compilers
for. How do I do it?
have you tried this, or are you just making things up to be able to continue
the argument? (hint: it doesn't work; python portability means that it's fairly
easy to write programs that run on multiple platforms, not that they will run
on all available platforms without porting and testing).
</F>
"Fredrik Lundh" <fr*****@pythonware.com> writes: have you tried this, or are you just making things up to be able to
continue the argument? (hint: it doesn't work; python portability
means that it's fairly easy to write programs that run on multiple
platforms, not that they will run on all available platforms without
porting and testing).
As an app writer I want to publish code that runs on multiple
platforms without needing special attention from the end user, and
preferably without needing special platform-specific attention from
me. If you want to use the words "Python portability" to mean
something weaker than that, that's ok, but I'm trying to accomplish
something better. I've written lots of Python code that uses the sha
module without my needing to worry about the target platform, so that
shows the goal can be met. I'm aiming for an aes/des module that's as
convenient for app writers and end users as the sha module is.
So now I know what you think "Python portability" means. What do you
think "batteries included" means?
<ph*@localhost.localdomain> wrote As an app writer I want to publish code that runs on multiple
platforms without needing special attention from the end user, and
preferably without needing special platform-specific attention from
me.
please answer the question: have you done this? what kind of programs
have you successfully delivered as "self-contained apps" for use on arbi-
trary platforms?
</F>
"Fredrik Lundh" <fr*****@pythonware.com> writes: please answer the question: have you done this? what kind of programs
have you successfully delivered as "self-contained apps" for use on arbi-
trary platforms?
Here's a simple one:
import sha
name = raw_input('Enter your name: ')
print 'Your name hashes to:', sha.new(name).hexdigest()
Maybe the sha module isn't in fact available on all platforms, but
it's definitely available on more platforms than I personally have
compilers for. Maybe some more complicated app will hit some
non-sha-related platform dependency and present a porting problem.
But it's worse if it has additional porting problems caused by the sha
module. Making the sha module available on multiple platforms is a
good thing regardless of what the rest of the app does. Doing that
means one less porting problem to worry about. So your question is an
irrelevant misdirection.
Anyway, I've written Python code that that's run under Linux and
Windows and the Macintosh without needing porting. That's not the
same as testing on every platform Python runs on, but if one of those
platforms fails other than from using a builtin module that's
documented as being platform specific, especially if the module is a
pure mathematically-defined algorithm like sha and not an OS-interfacing
module like threading or Tkinter, then I'd say the portability bug is
in Python or its library, not in my app. ph*@localhost.localdomain wrote: Maybe we're not thinking about the same problems. Say I'm an app
writer and I want to use one of your modules. My development
environment is GNU/Linux, and I want to ship a self-contained app that
anyone can run without having to download additional components. That
includes people wanting to run my app on Windows, Macintosh, Amiga,
and various other Python target platforms that I don't have compilers
for. How do I do it?
This is not possible - whether the module is included in Python or not.
People *will* have to download something besides your application,
namely Python - and in the version your require, too. If they cannot
compile Python themselves, they will have to find somebody who did that
for them.
This is very easy if you are using Windows, Debian, or Mac OS X.
It is very difficult if you are using an Amiga, or any of the other
Python target platforms that you don't have compilers for. HP-UX
users of your application will have a hard time to get it running
even if the module was available as a built-in.
This does not change much if the module is available separately,
and it does not change much whether the separate module is written
in Python or in C: Windows, Debian, and OS X users will still
find precompiled binaries for the module, and all others will still
have to build it themselves. This is very easy, since all they have
to do is to use distutils.
So, if your modules are generally useful, I hope you
will try to get them into the core.
Right - the critical condition is "if the modules are generally
useful". I cannot possibly know whether they are generally useful
until a general public has commented that they are.
I can't speak for the authors but I'd personally say that none of
those old modules are really suitable for the core on technical
grounds.
That is my impression, too.
The module I'm discussing would simply implement the FIPS standard
block ciphers (AES and DES) and the FIPS operating modes. These are
the basic building blocks that I feel are missing from the core. They
should be enough to fill the crypto needs of most applications.
Hmm. Most applications don't have any crypto needs. Those that do
typically need TLS, or secure password hashing (crypt(3) specifically).
So I somehow doubt that an AES module would be generally useful.
So if the module was primarily written to be included in the core, I
would initially reject it for that very reason. After one year or so
in its life, and a recognizable user base, inclusion can be
considered.
That makes no sense; improving the core is a perfectly valid reason to
write something. If someone wrote a straightforward patch that made
the Python interpreter 3x faster, would you still want to wait a year
before thinking about including it?
I would have to apply the "generally useful" test. For an 3x speed
improvement, about 100% of the Python users would find that useful.
For an AES module with an interface nobody has ever used but yourself,
it would be very difficult to argue that the module is generally useful.
If the module does not serve a need that people have, in general,
I don't see why it should be included (and yes, I wish a few of the
modules that were included recently wouldn't have been added so
quickly).
I certainly believe that every
contribution needs code review and user testing before entering wide
release, but that doesn't seem to be what you're getting at.
I'm primarily looking for the user testing. I require that this happens
*outside* the Python distribution. Make a distutils package, and report
the user feedback. Then propose inclusion into the core.
I release this "app" and presto, every Python user on every OS
platform can run it with no fuss. That includes CGI authors who
aren't allowed to install C modules at all, without getting their
hosting service to do it. How do I do the same thing if there's no
built-in sha module and all I have is sha.c?
It very much depends on the details of the target system. Yes,
it is a bit more tricky to install the additional library - but
that is the case with any additional library, whether it is C
or not.
For the CGI user who is not allowed to install binaries into
/usr/local, she can still install the module in her home
directory, no?
To me, the whole Python notion of "batteries included" expresses
precisely the idea that I'm trying to describe. Commonly-used
functions belong in the core, because making people download or
install them separately is a pain in the neck.
See, this is the critical point: "commonly-used functions", not
"functions I believe would be commonly used". You must have
*existing* users for a function to be commonly-used.
Regards,
Martin
Martin v. Löwi said Hmm. Most applications don't have any crypto needs.
Any program where one stores data would have crypto needs.
Here are some examples: Database, wordprocessor, spreadsheet, address
book, mail program, (should I go on?). What would be the alternative to
encryption to satisfy the functionality encryption
provides...memorization? But I believe this may be a case of things
being so obvious people forget their importance (e.g. breathing).
I have written a class that goes something like this (highly simplified)
class encryptFilter:
def __init__(acipher):
self.mycipher = acipher
def open(self, filename):
self.myfile = open(filename,"w")
def write(data):
encrypted_data = self.encrypt(data)
self.myfile.write(encrypted_data)
etc...
Now what if acipher and this class could be made from part of the core
distro? Any application could have the option of encryption with only a
few lines of code:
import std_crypt
# other code here
if get_user_pref("do_encryption"):
password = my_get_password()
acipher = std_crypt.AES(password)
outfile = std_crypt.encryptFilter(acipher)
outfile.open(filename)
else:
outfile = open(filename,"w")
# all other code doesn't change...
outfile.write(buncha_data)
This could be even more simpler, but that's the general idea. The
usefulness would be tremendous.
James Stroud wrote: Martin v. Löwi saidHmm. Most applications don't have any crypto needs.
Any program where one stores data would have crypto needs.
James, you must have mistyped the above, or your logic
is quite flawed.
I have written dozens of programs which store data, and
only a couple have had any crypto needs. This alone
invalidates your statement as it stands.
What did you really mean to say? Maybe "could" instead
of "would"? That would make the statement true, yet
somewhat pointless, and it doesn't contradict what
Martin wrote in either case.
-Peter
On Monday 24 January 2005 03:40 pm, Fredrik Lundh wrote: have you tried this, or are you just making things up to be able to continue
the argument? (hint: it doesn't work; python portability means that it's fairly
easy to write programs that run on multiple platforms, not that they will run
on all available platforms without porting and testing).
I don't know, they come pretty close. Problems with portability in pure-python
modules have been almost non-existent for me. I almost never test on the
Windows platform, but I know some of my code is being used by Windows
users. And I guess I won't start, really, because I haven't heard any complaints
about it breaking.
I *did* have a few problems with Zope packages running on FreeBSD, bizarrely
enough, given that it's in the "POSIX" family of OSs, and therefore seems very
similar on the surface.
And, well, I'm sorry Mr. Lundh, but your PIL module actually is something
of a pain to install still. The OP is right about that. Usually worth it, but I don't
like the fact that that pain is being passed on to the end-user of my software.
The fact that you got all snide and refused to work with me when I asked
for a copy of the autoconf sources to try to fix the problem didn't make me
too happy, either. So, naturally, I just dropped it. I'm in no position to
pick it up now -- just don't need it badly enough now (AFAIK it's unchanged
still -- lately I've been using the Debian package though, so I don't have to
worry about it).
I know that you don't want PIL to go into the core, of course, but I'm pretty
sure that problem would've needed fixing, if it were to be introduced.
Small problem, trivial to fix, but that's what bothers me about it.
Terry
--
Terry Hancock ( hancock at anansispaceworks.com )
Anansi Spaceworks http://www.anansispaceworks.com
I was purposefully making an illogical statement to illustrate the lapse
in reason of Martin's statement. Users have crypto needs, not
applications. Applications are presumably not anthropomorphic enough to
have needs--hence the lack of logic.
However, I am not an application (as far as you or I know) and thus can
not truly speak to the needs of applications in general, if indeed they
have any.
Applications that lack features force users to accept a limited feature
set or they use an alternative program with other limitations. Putting
the possibility for cryptographic storage increases the utility of any
application that stores data, and it could be done without much work if
it were properly included in the core distribution. I have found it
amazing how few programs include encryption as an option. I believe this
is because its hard for programmers to include it and/or they falsely
reason that "if I don't need it, the user doesn't", which is up there
with "if I can't see them, then they can't see me" in terms of bad
logic.
James
On Mon, 2005-01-24 at 17:17, James Stroud wrote: I was purposefully making an illogical statement to illustrate the lapse
in reason of Martin's statement. Users have crypto needs, not
applications. Applications are presumably not anthropomorphic enough to
have needs--hence the lack of logic.
However, I am not an application (as far as you or I know) and thus can
not truly speak to the needs of applications in general, if indeed they
have any.
Applications that lack features force users to accept a limited feature
set or they use an alternative program with other limitations. Putting
the possibility for cryptographic storage increases the utility of any
application that stores data, and it could be done without much work if
it were properly included in the core distribution. I have found it
amazing how few programs include encryption as an option. I believe this
is because its hard for programmers to include it and/or they falsely
reason that "if I don't need it, the user doesn't", which is up there
with "if I can't see them, then they can't see me" in terms of bad
logic.
James Stroud <js*****@mbi.ucla.edu> writes: Applications that lack features force users to accept a limited feature
set or they use an alternative program with other limitations. Putting
the possibility for cryptographic storage increases the utility of any
application that stores data, and it could be done without much work if
it were properly included in the core distribution. I have found it
amazing how few programs include encryption as an option. I believe this
is because its hard for programmers to include it and/or they falsely
reason that "if I don't need it, the user doesn't", which is up there
with "if I can't see them, then they can't see me" in terms of bad logic.
The likely-best-known Python application in the world (probably more
people have heard of it than have heard of Python) originally had
crypto (an AES module distributed as a C extension with the beta test
versions of the app) but the crypto was brutally ripped out of the
application for the final release (you can still see scar tissue in
the code where the crypto was, i.e., there's still a function called
something like "encrypt_packet" which no longer actually encrypts the
packet). I haven't found out exactly why the crypto was removed so I
can't be certain that it's because of cross-platform installation
headaches. I do know that its author wanted an AES module in the core
to use for that application, in order to not have to distribute a C
extension, and he did some of the early work with me on writing such a
module to submit for that purpose. The module I ended up proposing is
an offshoot of that effort.
[Again I'm having news server trouble and made a previous attempt to
post this, so sorry if you see it twice. This version is edited
somewhat from the previous.]
"Martin v. Löwis" <ma****@v.loewis.de> writes: This is not possible - whether the module is included in Python or not.
People *will* have to download something besides your application,
namely Python - and in the version your require, too. If they cannot
compile Python themselves, they will have to find somebody who did that
for them.
Sorry, the presumption is that they already have Python installed.
So, if your modules are generally useful, I hope you
will try to get them into the core.
Right - the critical condition is "if the modules are generally
useful". I cannot possibly know whether they are generally useful
until a general public has commented that they are.
There is considerable demand for a crypto module. Note how this thread
started: someone who had been using the rotor module complained that it's
now missing.
Hmm. Most applications don't have any crypto needs. Those that do
typically need TLS, or secure password hashing (crypt(3) specifically).
So I somehow doubt that an AES module would be generally useful.
I believe it's feasible to do TLS in pure Python with acceptable
performance, given a DES module and the existing sha module. I think
Trevor Perrin has already written something like that. It's feasible
because outside of bulk crypto operations using DES and SHA, TLS's
execution time is dominated by public-key math calculations, which are
acceptably fast using Python's built-in longs. The rest of the stuff
like certificate parsing happens just once per session and doing it in
Python isn't too awful. On the other hand, it's impossible to do a
general purpose TLS stack acceptably without a DES module. According
to RFC 2246 sec. 9, supporting TLS_DHE_DSS_WITH_3DES_EDE_CBC_SHA is
mandatory unless the application profile specifies otherwise. SHA is
already in the core and DHE/DSS can be done with Python longs, so DES
is the missing critical piece.
Also, since DES is in the process of being decertified and AES is its
replacement, supporting DES without supporting AES is silly. GnuPG
for example uses AES by default, so writing an interoperating Python
application (I have some parts of one working) needs AES.
If by crypt(3) you mean the old thing of DES with the modified E
table, I don't think it's used much any more. I don't see any reason
against adding it if people want it, but it's not an encryption
function, it's an authentication function, and shouldn't be subject to
the legal concerns of a general purpose encryption module.
I do think it would be nice to have TLS in the core someday, just like
it's already in the Java distro (JSSE). However, that's much more
ambitious and anyway it would rely on the block cipher module, so the
block cipher module is needed anyway. Meanwhile, a pure-Python TLS
implementation outside the core is ok as a workaround. Getting more
pure Python modules into the core is in general less valuable than
getting more C modules into the core.
For an AES module with an interface nobody has ever used but yourself,
it would be very difficult to argue that the module is generally useful.
Eh? Lots of people have asked for general purpose crypto modules, and
the interface follows the approach used by any number of existing
libraries. It just does exposes the simplest possible interface to
the block cipher and then implements the FIPS operating modes in a
natural and obvious way. I've used a number of other such libraries
and am pretty familiar with what they need to do. The main thing it
tries to do differently from PEP 272 is separate the block ciphers
from the FIPS modes, so that each new cipher module doesn't have to
implement all the different modes.
I hadn't thought there was any controversy over the technical side of
this. There's the legal/political issue which I can gently suggest is
overblown, but since I'm not the one who would have to take the heat
if some government bigwig somewhere flipped their lid over crypto in
the Python distro, I don't feel entitled to be very vociferous about
this aspect.
I'm primarily looking for the user testing. I require that this happens
*outside* the Python distribution. Make a distutils package, and report
the user feedback. Then propose inclusion into the core.
I'm happy to have that kind of testing (and I requested it), given
that the goal is inclusion in the core, and the core developers have
told me (as they did) that the proposal looks good and they'd like to
have the module, so I can reasonably expect it to go into the core if
it meets its technical expectations.
If the developers instead say (as they seemed to somewhat later) that
because of legal/political concerns, there's no way the module can
possibly go into the core no matter how good it is technically, then
my motivation for writing the module dries up quite a bit. If someone
wants to pay me a salary to write it anyway, I might be interested,
but people who work as volunteers get to pick their own reasons for
doing things or not doing them.
It very much depends on the details of the target system. Yes,
it is a bit more tricky to install the additional library - but
that is the case with any additional library, whether it is C or not.
I don't see this. If a library is pure Python, I can always include
it with the application. If it's in C, somebody has to compile and
install it, and I as the app writer may not have access to compilers
for the target platform. Of course a Python library might have
platform-dependent OS calls or other configuration hair, but that
shouldn't apply to a pure mathematical function that's written
portably and makes no OS calls at all.
For the CGI user who is not allowed to install binaries into
/usr/local, she can still install the module in her home
directory, no?
Evidently not always. And how would the CGI user create a binary
anyway, even given a way to install it, if the web hosting service is
using a platform that the CGI user doesn't have a compiler for? Think
of a Mac user whose web host runs Windows, or vice versa.
See, this is the critical point: "commonly-used functions", not
"functions I believe would be commonly used". You must have
*existing* users for a function to be commonly-used.
You're going around in circles. DES and AES have millions of users.
They have few Python users because the functions aren't available in
Python. To fix that, they must be added to Python. How many users
do you think the Python sha module had before it went into Python?
--Paul
Terry Hancock wrote: And, well, I'm sorry Mr. Lundh, but your PIL module actually is something
of a pain to install still. The OP is right about that. Usually worth it, but I don't
like the fact that that pain is being passed on to the end-user of my software.
The fact that you got all snide and refused to work with me when I asked
for a copy of the autoconf sources to try to fix the problem didn't make me
too happy, either.
ah, you're the guy who flamed me and called me names because PIL didn't
fit some hypothetical GNU-inspired definition of "open source software." it's
always sad when people have to attack those who don't share their religion,
but it's not that much I can do about that.
So, naturally, I just dropped it.
which means that you have no idea how easy or hard it is to install PIL today
(hint: things have changed)
</F>
<ph*@localhost.localdomain> wrote: The likely-best-known Python application in the world (probably more
people have heard of it than have heard of Python) originally had
crypto
and what Python application is that? I can think of quite a few applications
written in Python that are widely known, but none of these are distributed as
Python code to end users.
I do know that its author wanted an AES module in the core
to use for that application, in order to not have to distribute a C
extension
</F>
Fredrik Lundh wrote: <ph*@localhost.localdomain> wrote:
The likely-best-known Python application in the world (probably more
people have heard of it than have heard of Python) originally had
crypto
and what Python application is that? I can think of quite a few applications
written in Python that are widely known, but none of these are distributed as
Python code to end users.
BitTorrent, probably.
I believe that a very high percentage of users get the BitTorrent apps
as standalone executables, not as a Python package where one has to go
download Python first. The ones that do get it as Python source probably
get it from their Linux/FreeBSD/other distribution where such
dependencies are also taken care of for them.
--
Robert Kern rk***@ucsd.edu
"In the fields of hell where the grass grows high
Are the graves of dreams allowed to die."
-- Richard Harter ph*@localhost.localdomain wrote: I hadn't thought there was any controversy over the technical side of
this.
There isn't. The interface might be beautifully designed, and you might
claim it is, and I would *still* require that the module gets field
testing before being incorporated into Python. If other people start
attesting that the module is beatifully designed, and should be included
in the Python core - *then* it is worth looking into inclusion.
I'm happy to have that kind of testing (and I requested it), given
that the goal is inclusion in the core, and the core developers have
told me (as they did) that the proposal looks good and they'd like to
have the module, so I can reasonably expect it to go into the core if
it meets its technical expectations.
Not if I have a say in it. *Any* new module should see out-of-the-core
distribution first (unless there is BDFL pronouncement to include it,
of course).
This really is a matter of development process, not of technical
quality.
If the developers instead say (as they seemed to somewhat later) that
because of legal/political concerns, there's no way the module can
possibly go into the core no matter how good it is technically, then
my motivation for writing the module dries up quite a bit.
I personally would not say that, although I can imagine that some people
do say that, and I would also defend an inclusion, and push compliance
the BXA requirements so we can legally export Python out of the
U.S.A.
Evidently not always. And how would the CGI user create a binary
anyway, even given a way to install it, if the web hosting service is
using a platform that the CGI user doesn't have a compiler for? Think
of a Mac user whose web host runs Windows, or vice versa.
In either case, the user would best use the pre-compiled binary that
somebody else provided for the platform. Actually, the Windows user
using an OS X CGI server can probably just invoke the gcc which is
on the target system, anyway.
See, this is the critical point: "commonly-used functions", not
"functions I believe would be commonly used". You must have
*existing* users for a function to be commonly-used.
You're going around in circles.
No, I'm merely repeating myself, and rephrasing each time.
I have to, because apparently you don't see what my requirement
is.
They have few Python users because the functions aren't available in
Python. To fix that, they must be added to Python. How many users
do you think the Python sha module had before it went into Python?
The original source code of the SHA-1 implementation is the NIST code
(Gutmann, then Hollerbach), so I guess that had hundreds of users before
the module was contributed to Python. The module itself (including the
API) was written by Greg Stein and Andrew Kuchling. I believe (without
being able to verify) that they distributed this module for quite some
time, before contributing it to Python. We would have to ask them
how many users they had until they felt confident to contribute the
code.
Regards,
Martin
Andrew Kuchling This discussion thread is closed Replies have been disabled for this discussion. Similar topics
46 posts
views
Thread by Robin Becker |
last post: by
|
5 posts
views
Thread by rony steelandt |
last post: by
|
1 post
views
Thread by Protoman |
last post: by
| | | | | | | | | | | | | | | | | | | |
