Crypto.Cipher.ARC4, bust or me doing something wrong?

[Michael Sparks]

Hi,
I suspect this is a bug with AMK's Crypto package from
http://www.amk.ca/python/code/crypto , but want to
check to see if I'm being dumb before posting a bug
report.

I'm looking at using this library and to familiarise myself writing
small tests with each of the ciphers. When I hit Crypto.Cipher.ARC4 I've
found that I can't get it to decode what it encodes. This might be a
case of PEBKAC, but I'm trying the following:

from Crypto.Cipher import ARC4 as cipher
key = "........"
obj = cipher.new(key)
obj.encrypt("This is some random text") ')f\xd4\xf6\xa6Lm\x9a%}\x8a\x95\x8ef\x00\xd6:\x12\ x00!\xf3k\xafX' X=_
X ')f\xd4\xf6\xa6Lm\x9a%}\x8a\x95\x8ef\x00\xd6:\x12\ x00!\xf3k\xafX' obj.decrypt(X)

'\x87\xe1\x83\xc1\x93\xdb\xed\x93U\xe4_\x92}\x9f\x db\x84Y\xa3\xd4b\x9eHu~'

Clearly this decode doesn't match the encode. Me being dumb or bug?

Any comments welcome :)
Michael.
--
Mi************@rd.bbc.co.uk, http://kamaelia.sourceforge.net/
British Broadcasting Corporation, Research and Development
Kingswood Warren, Surrey KT20 6NP

This message (and any attachments) may contain personal views
which are not the views of the BBC unless specifically stated.

[Michael J. Fromberger]

In article <dg**********@nntp0.reith.bbc.co.uk>,
Michael Sparks <mi******@rd.bbc.co.uk> wrote:

Hi,
I suspect this is a bug with AMK's Crypto package from
http://www.amk.ca/python/code/crypto , but want to
check to see if I'm being dumb before posting a bug
report.

I'm looking at using this library and to familiarise myself writing
small tests with each of the ciphers. When I hit Crypto.Cipher.ARC4 I've
found that I can't get it to decode what it encodes. This might be a
case of PEBKAC, but I'm trying the following:

from Crypto.Cipher import ARC4 as cipher
key = "........"
obj = cipher.new(key)
obj.encrypt("This is some random text") ')f\xd4\xf6\xa6Lm\x9a%}\x8a\x95\x8ef\x00\xd6:\x12\ x00!\xf3k\xafX' X=_
X ')f\xd4\xf6\xa6Lm\x9a%}\x8a\x95\x8ef\x00\xd6:\x12\ x00!\xf3k\xafX' obj.decrypt(X) '\x87\xe1\x83\xc1\x93\xdb\xed\x93U\xe4_\x92}\x9f\x db\x84Y\xa3\xd4b\x9eHu~'

Clearly this decode doesn't match the encode. Me being dumb or bug?

Any comments welcome :)

Michael,

Since ARC4 is a stream cipher, the keystream changes over time -- with
ARC4, after each character enciphered. To decrypt successfully, you
need to make sure the decrypting keystream exactly matches the
encrypting one.

In your example, you used a different keystream to decrypt than you used
to encrypt -- in this case, a little further downstream of the original
encryption key.

Contrast your experience above with the following transcript:

from Crypto.Cipher import ARC4 as cipher
enc = cipher.new("abcdefgh")
dec = cipher.new("abcdefgh")
x = enc.encrypt("This is some random text")
x "\x05o\xd5XH|\xa4\xfc\xf7z\xecd\xe92\xfb\x05rR'\xb f\xc0F\xfc\xde" y = dec.decrypt(x)
y 'This is some random text' enc.decrypt(x)

'M|[bI\x1ciG6A]\x13Hz\xb0\x19\xca\xf1-\x9a\x1a2\x9e%'

I hope this helps clear up your confusion.

Cheers,
-M

--
Michael J. Fromberger | Lecturer, Dept. of Computer Science
http://www.dartmouth.edu/~sting/ | Dartmouth College, Hanover, NH, USA

[Michael Sparks]

Michael J. Fromberger wrote:
....

Since ARC4 is a stream cipher, the keystream changes over time -- with
ARC4, after each character enciphered. To decrypt successfully, you
need to make sure the decrypting keystream exactly matches the
encrypting one.

....

from Crypto.Cipher import ARC4 as cipher
enc = cipher.new("abcdefgh")
dec = cipher.new("abcdefgh")
x = enc.encrypt("This is some random text")
x "\x05o\xd5XH|\xa4\xfc\xf7z\xecd\xe92\xfb\x05rR'\xb f\xc0F\xfc\xde" y = dec.decrypt(x)
y 'This is some random text' enc.decrypt(x)

'M|[bI\x1ciG6A]\x13Hz\xb0\x19\xca\xf1-\x9a\x1a2\x9e%'

I hope this helps clear up your confusion.

Hi Michael,
Thanks for this, much appreciated.
Michael
--
Mi************@rd.bbc.co.uk, http://kamaelia.sourceforge.net/
British Broadcasting Corporation, Research and Development
Kingswood Warren, Surrey KT20 6NP

This message (and any attachments) may contain personal views
which are not the views of the BBC unless specifically stated.

[Paul Rubin]

Michael Sparks <mi******@rd.bbc.co.uk> writes:

I'm looking at using this library and to familiarise myself writing
small tests with each of the ciphers. When I hit Crypto.Cipher.ARC4 I've
found that I can't get it to decode what it encodes. This might be a
case of PEBKAC, but I'm trying the following:

You have to reinitialize the cipher state for decryption, as someone
else explained. You also have to make sure that keys are unique and
independent for separate messages. For most applications you probably
want to think about adding authentication. In general you shouldn't
use arc4 unless you know what you're doing. What's the application?

[Michael Sparks]

Paul Rubin wrote:

Michael Sparks <mi******@rd.bbc.co.uk> writes:

I'm looking at using this library and to familiarise myself writing
small tests with each of the ciphers. When I hit Crypto.Cipher.ARC4
I've found that I can't get it to decode what it encodes. This might
be a case of PEBKAC, but I'm trying the following:
You have to reinitialize the cipher state for decryption, as someone
else explained. You also have to make sure that keys are unique and
independent for separate messages.

Hmm... Thanks for this.
For most applications you probably
want to think about adding authentication.
Indeed - though I'm working bottom up. How a key gets transfered
from a to b safely will be another step. Another one will be how to
trust that exchange, for how long, and how much, etc. I'm well aware
that this is a well trod area though, hence why I'm working bottom up.
In general you shouldn't
use arc4 unless you know what you're doing.
Having looked at how it works from a user perspective, it's fairly
inappropriate anyway, due to wanting to work over unreliable
channels anyway.
What's the application?

Components for secure communications and identity confirmation.
Use cases:
* Signing content multicast by the BBC to indicate that it came from
the BBC. (ohh, so many issues :-)
* Signing content captured on a mobile device such that we know that
it came from that mobile. Specific high level use case of that is
to be able to accept contributions from people from arbitrary
devices and know who they came from. Perhaps embedded in the
essence.
* Related to that would be the ability to tag content with rights [1]
information, and to know it's not been tampered with.

[1] eg Who created it. When? Has it been published/broadcast or not?
When? Have additional rights over and above fair use/dealing
been granted (eg creative commons attribution license).

* Similarly people want the ability to protect data in transit
between trusted end points.

Some people using the system would probably be looking to build
restrictions management as well, but that's largely beyond the scope
of our system. Indeed restrictions management would require breaking
our system.

For these use cases to work, encryption & digest tools are clearly an
option, and hence having components that support encryption and
digest are (to say the least) useful. They're obviously not the only
techniques though.

Rather than re-inventing wheels I thought I'd pick a library sit down
and see how pycrypt's meant to be used before actually going anyway.
(Amongst other reasons, this is why I suspected me, rather than the
library :-)

Given the pycrypt library docs page starts off with:
"This documentation assumes you have some basic knowledge about
the Python language, but not necessarily about cryptography."

I do for example know enough about cryptography to know that me
devising my own approach is foolhardy. Thus the statement above
appealed :)

As a result I decided to sit down and learn how to use this to form some
basic components for encryption/decryption. Given another part early in
the docs was """A central goal of the author's has been to provide a
simple, consistent interface for similar classes of algorithms.""" I
decided to start off with "sketches" implementing minimal examples for
each digest and cipher. I'm now working through the Public Key
examples.

FWIW, I'm well aware how easy it is to get cipher/digest/etc based
security/id systems wrong. I'm really starting with pycrypt because it
looked simple enough, low level enough and self contained enough to
act as a base for working with existing more complex systems.

I supsect we'll end up looking at or wrapping some other library
instead/as well, but it struck me as a nice starting point.

Anyway, once I've gone through all of the existing digests/ciphers/PK
ciphers, I'll post the snippets up on our site as raw examples for
pycrypto, which will hopefully be a) correct usage b) be useful to
others.

Thanks for the comments,
Michael.
--
Mi************@rd.bbc.co.uk, http://kamaelia.sourceforge.net/
British Broadcasting Corporation, Research and Development
Kingswood Warren, Surrey KT20 6NP

This message (and any attachments) may contain personal views
which are not the views of the BBC unless specifically stated.

[Paul Rubin]

Michael Sparks <mi******@rd.bbc.co.uk> writes:

Rather than re-inventing wheels I thought I'd pick a library sit down
and see how pycrypt's meant to be used before actually going anyway.
(Amongst other reasons, this is why I suspected me, rather than the
library :-)
Pycrypt doesn't operate at anything like the level you need. It just
gives you low level cipher primitives. You need higher level protocols.
FWIW, I'm well aware how easy it is to get cipher/digest/etc based
security/id systems wrong. I'm really starting with pycrypt because it
looked simple enough, low level enough and self contained enough to
act as a base for working with existing more complex systems.
Do yourself a favor and stick to something standard like TLS, rather
than cook up your own protocol. There are some Python wrappers for
OpenSSL or GNU TLS, for example.
Anyway, once I've gone through all of the existing digests/ciphers/PK
ciphers, I'll post the snippets up on our site as raw examples for
pycrypto, which will hopefully be a) correct usage b) be useful to
others.

You really need to know a lot more than it sounds like you know, to
have any chance of getting fancy protocol designs correct.

http://www.cs.ucdavis.edu/~rogaway/c.../book/main.pdf

is a textbook that will show you how to do this, or at least give you
an idea of what you're dealing with. Watch out, it is rather theoretical.

[Michael Sparks]

Paul Rubin wrote:

Michael Sparks <mi******@rd.bbc.co.uk> writes:

Rather than re-inventing wheels I thought I'd pick a library sit down
and see how pycrypt's meant to be used before actually going anyway.
(Amongst other reasons, this is why I suspected me, rather than the
library :-)
Pycrypt doesn't operate at anything like the level you need. It just
gives you low level cipher primitives. You need higher level protocols.

Agreed. As I say I'm building up systems slowly on this front, and part of
the aim will be to show how outright dumb it is to build these things
without a decent understanding of the underlying tech.

Sadly there's many people (where I work) who don't understand how easy
it is to get these things wrong, and unfortunately practical demonstration
is one of the few languages people listen to :-/

This is where I border on being unprofessional so I'll be quiet at that
stage :-((

On a more positive note, as I say I picked something simple initially, and
have no intention at this stage of it being the primary approach.

FWIW, I'm well aware how easy it is to get cipher/digest/etc based
security/id systems wrong. I'm really starting with pycrypt because it
looked simple enough, low level enough and self contained enough to
act as a base for working with existing more complex systems.

Do yourself a favor and stick to something standard like TLS, rather
than cook up your own protocol. There are some Python wrappers for
OpenSSL or GNU TLS, for example.

If those operated over multicast then they'd be an option... However...

Incidentally our primary use is to indicate *integrity* of data not to
prevent content being viewed. It's also obviously susceptible to attack
in many ways. Again, I'd say more offline about this - Usenet is just the
wrong place for it.

Anyway, once I've gone through all of the existing digests/ciphers/PK
ciphers, I'll post the snippets up on our site as raw examples for
pycrypto, which will hopefully be a) correct usage b) be useful to
others.

You really need to know a lot more than it sounds like you know,
*shrug* It's not a matter of what I don't know it's a matter of showing how
things can be used, and I want to use it (internally) to demonstrate what
goes wrong when people try to design their own systems naively.

One way to learn is to build.
to have any chance of getting fancy protocol designs correct.
FWIW, I have little intention of inventing something new (in terms of
protocols) /if/ possible. What these components will be used for is to
demonstrate how they work inside, and what goes wrong if people try to
reinvent wheels.

Quite frankly we *should* be using a crypto professional, however that's
currently not a realistic option :-((( As a result currently it's up to me
to demonstrate here the risks involved in creating your own system.
http://www.cs.ucdavis.edu/~rogaway/c.../book/main.pdf

is a textbook that will show you how to do this, or at least give you
an idea of what you're dealing with. Watch out, it is rather theoretical.

That's fine/great :) - thanks for the reference :-)

FWIW, I *do* have a good idea of what I'm dealing with, //which is why// I'm
pessimistic about getting it right. I might have more experience with these
things than those I'm working with, however I am very aware of my
(current :) limits. Hopefully I'll be able to use these tools educate those
around me why rolling your own is an idea fraught with issues.

Best Regards,
Michael.