Digest MD5 authentication over using ZSI

[trapeze.jsg]

Hi.

I am trying to get through to Microsoft MapPoint Services using ZSI for
soap handling. I can generate the service classes and also the
soap-requests generated by the service classes seem to be OK. The
problem I am facing is that I can't seem to authenticate myself. I have
made a small change to ZSI.client so that when I get a "401
Unauthorized" response from the remote server I build up a nice
authorization request:

POST /Find-30/FindService.asm x HTTP/1.1
Host: findv3.staging. mappoint.net
Accept-Encoding: identity
User-Agent: Mozilla/4.0 (compatible; MSIE 6.0; MS Web Services Client
Protocol 1.1.4322.573)
SOAPAction: "http://s.mappoint.net/mappoint-30/FindAddress"
Authorization: Digest username="10628 8", realm="MapPoint ",
nonce="91168da8 e3a097f41264875 211009a194b99a9 4ffe5bc61941582 0880a5b",
uri="/Find-30/FindService.asm x",
response="26aa9 e36f9ff2a830803 0810ab83dad1", qop=auth, nc=00000001,
cnonce="623c12f 33f0eb883"
Content-length: 0
Expect: 100-continue
The problem is that the server won't authorize me. I have a C# .net
program that does exactly the same I'm trying in python, and it is
authorized nicely:

POST /Find-30/FindService.asm x HTTP/1.1
User-Agent: Mozilla/4.0 (compatible; MSIE 6.0; MS Web Services Client
Protocol 1.1.4322.573)
Content-Type: text/xml; charset=utf-8
SOAPAction: "http://s.mappoint.net/mappoint-30/FindAddress"
Authorization: Digest
username="10628 8",realm="MapPo int",nonce="487 911f02ed2ef7063 26675211008a8ec 39cfa4fb0930475 7c8dde417354",u ri="/Find-30/FindService.asm x",cnonce="e1ed 9880c5e3777a4ba 280cec1c9e362", nc=00000001,qop ="auth",respons e="b4119a4db738 14fd09ae5fec11f c9730"
Content-Length: 523
Expect: 100-continue
Host: findv3.staging. mappoint.net

So I guess the problem is in the Digest calculation. Unfortunately I
don't know much about the issue but I have managed to "steel" this from
urllib2 and changed it a bit to fit my usage (see below).

1. Can I do this another way?
2. Has anybody made a digest MD5 authenticator for ZSI?
3. Whats wrong with my calculation or is it the header??
4. Can I use urllib2 to test the authentication part of a Soap Service.

----------------- My authentication calculator ---->

import md5
import sha
import re
import time
import random
import os.path
def randombytes(n):
"""Return n random bytes."""
# Use /dev/urandom if it is available. Fall back to random module
# if not. It might be worthwhile to extend this function to use
# other platform-specific mechanisms for getting random bytes.
if os.path.exists( "/dev/urandom"):
f = open("/dev/urandom")
s = f.read(n)
f.close()
return s
else:
L = [chr(random.rand range(0, 256)) for i in range(n)]
return "".join(L)
class Challenge:
def __init__(self,c hallenge):
self.params = {}
self.no_chal=0
if challenge.upper ().find('WWW-Authenticate:'. upper())==-1:
self.no_chal=1
rx =
re.compile('WWW-Authenticate:.* qop="(\w+)"',re .IGNORECASE|re. MULTILINE)
m = rx.search(chall enge)
if m:
self.params['qop'] = m.group(1)

rx =
re.compile('WWW-Authenticate:.* realm="(\w+)"', re.IGNORECASE|r e.MULTILINE)
m = rx.search(chall enge)
if m:
self.params['realm'] = m.group(1)

rx =
re.compile('WWW-Authenticate:.* algorithm="(\w+ )"',re.IGNORECA SE|re.MULTILINE )
m = rx.search(chall enge)
if m:
self.params['algorithm'] = m.group(1)

rx =
re.compile('WWW-Authenticate:.* nonce="(\w+)"', re.IGNORECASE|r e.MULTILINE)
m = rx.search(chall enge)
if m:
self.params['nonce'] = m.group(1)

rx =
re.compile('WWW-Authenticate:.* opaque="(\w+)"' ,re.IGNORECASE| re.MULTILINE)
m = rx.search(chall enge)
if m:
self.params['opaque'] = m.group(1)

rx =
re.compile('WWW-Authenticate:.* Digest.*"',re.I GNORECASE|re.MU LTILINE)
m = rx.search(chall enge)
if m:
self.params['Digest'] = 1
def get(self,keywor d,default=None) :
if self.params.has _key(keyword):
return self.params[keyword]
else:
return default

def no_challenge(se lf):
return self.no_chal
class AbstractDigestA uthHandler:
# Digest authentication is specified in RFC 2617.

# XXX The client does not inspect the Authentication-Info header
# in a successful response.

# XXX It should be possible to test this implementation against
# a mock server that just generates a static set of challenges.

# XXX qop="auth-int" supports is shaky

def __init__(self, user, passwd):
self.user = user
self.passwd = passwd
self.retried = 0
self.nonce_coun t = 0

def reset_retry_cou nt(self):
self.retried = 0

def retry_http_dige st_auth(self, req, auth):
token, challenge = auth.split(' ', 1)
chal = parse_keqv_list (parse_http_lis t(challenge))
auth = self.get_author ization(req, chal)
if auth:
auth_val = 'Digest %s' % auth
if req.headers.get (self.auth_head er, None) == auth_val:
return None
req.add_header( self.auth_heade r, auth_val)
resp = self.parent.ope n(req)
return resp

def get_cnonce(self , nonce):
# The cnonce-value is an opaque
# quoted string value provided by the client and used by both
client
# and server to avoid chosen plaintext attacks, to provide
mutual
# authentication, and to provide some message integrity
protection.
# This isn't a fabulous effort, but it's probably Good Enough.
dig = sha.new("%s:%s: %s:%s" % (self.nonce_cou nt, nonce,
time.ctime(),
randombytes(8)) ).hexdigest()
return dig[:16]

def get_authorizati on(self, chal, method, selector):
try:
realm = chal.get('realm ')
nonce = chal.get('nonce ')
qop = chal.get('qop')
algorithm = chal.get('algor ithm', 'MD5')
# mod_digest doesn't send an opaque, even though it isn't
# supposed to be optional
opaque = chal.get('opaqu e', None)
except KeyError:
return None

H, KD = self.get_algori thm_impls(algor ithm)
if H is None:
return None

user, pw = self.user, self.passwd
if user is None:
return None

# XXX not implemented yet
entdig = None
#if req.has_data():
# entdig = self.get_entity _digest(req.get _data(), chal)

A1 = "%s:%s:%s" % (user, realm, pw)
A2 = "%s:%s" % (method,
# XXX selector: what about proxies and full
urls
selector)
if qop == 'auth':
self.nonce_coun t += 1
ncvalue = '%08x' % self.nonce_coun t
cnonce = self.get_cnonce (nonce)
noncebit = "%s:%s:%s:%s:%s " % (nonce, ncvalue, cnonce, qop,
H(A2))
respdig = KD(H(A1), noncebit)
else:
pass
# XXX should the partial digests be encoded too?

base = 'username="%s", realm="%s", nonce="%s", uri="%s", ' \
'response="%s"' % (user, realm, nonce, selector,
respdig)
if opaque:
base = base + ', opaque="%s"' % opaque
if entdig:
base = base + ', digest="%s"' % entdig
if algorithm != 'MD5':
base = base + ', algorithm="%s"' % algorithm
if qop:
base = base + ', qop=auth, nc=%s, cnonce="%s"' % (ncvalue,
cnonce)
return base

def get_algorithm_i mpls(self, algorithm):
# lambdas assume digest modules are imported at the top level
if algorithm == 'MD5':
H = lambda x: md5.new(x).hexd igest()
elif algorithm == 'SHA':
H = lambda x: sha.new(x).hexd igest()
# XXX MD5-sess
KD = lambda s, d: H("%s:%s" % (s, d))
return H, KD

def get_entity_dige st(self, data, chal):
# XXX not implemented yet
return None

<---------------
Ethereal request sniffer: (the authorization request and server
response)
No. Time Source Destination
Protocol Info
15 0.757244 192.168.0.106 64.4.58.250 HTTP
POST /Find-30/FindService.asm x HTTP/1.1

Frame 15 (612 bytes on wire, 612 bytes captured)
Arrival Time: Sep 3, 2005 23:57:20.487466 000
Time delta from previous packet: 0.000111000 seconds
Time since reference or first frame: 0.757244000 seconds
Frame Number: 15
Packet Length: 612 bytes
Capture Length: 612 bytes
Protocols in frame: eth:ip:tcp:http
Ethernet II, Src: FirstInt_6c:a0: 01 (00:40:ca:6c:a0 :01), Dst:
D-Link_36:0e:3d (00:0d:88:36:0e :3d)
Destination: D-Link_36:0e:3d (00:0d:88:36:0e :3d)
Source: FirstInt_6c:a0: 01 (00:40:ca:6c:a0 :01)
Type: IP (0x0800)
Internet Protocol, Src: 192.168.0.106 (192.168.0.106) , Dst: 64.4.58.250
(64.4.58.250)
Version: 4
Header length: 20 bytes
Differentiated Services Field: 0x00 (DSCP 0x00: Default; ECN: 0x00)
0000 00.. = Differentiated Services Codepoint: Default (0x00)
.... ..0. = ECN-Capable Transport (ECT): 0
.... ...0 = ECN-CE: 0
Total Length: 598
Identification: 0x7797 (30615)
Flags: 0x04 (Don't Fragment)
0... = Reserved bit: Not set
.1.. = Don't fragment: Set
..0. = More fragments: Not set
Fragment offset: 0
Time to live: 128
Protocol: TCP (0x06)
Header checksum: 0x44fa [correct]
Source: 192.168.0.106 (192.168.0.106)
Destination: 64.4.58.250 (64.4.58.250)
Transmission Control Protocol, Src Port: 3183 (3183), Dst Port: http
(80), Seq: 1, Ack: 1, Len: 558
Source port: 3183 (3183)
Destination port: http (80)
Sequence number: 1 (relative sequence number)
Next sequence number: 559 (relative sequence number)
Acknowledgement number: 1 (relative ack number)
Header length: 20 bytes
Flags: 0x0018 (PSH, ACK)
0... .... = Congestion Window Reduced (CWR): Not set
.0.. .... = ECN-Echo: Not set
..0. .... = Urgent: Not set
...1 .... = Acknowledgment: Set
.... 1... = Push: Set
.... .0.. = Reset: Not set
.... ..0. = Syn: Not set
.... ...0 = Fin: Not set
Window size: 64512
Checksum: 0x9143 [correct]
Hypertext Transfer Protocol
POST /Find-30/FindService.asm x HTTP/1.1\r\n
Request Method: POST
Request URI: /Find-30/FindService.asm x
Request Version: HTTP/1.1
Host: findv3.staging. mappoint.net\r\ n
Accept-Encoding: identity\r\n
User-Agent: Mozilla/4.0 (compatible; MSIE 6.0; MS Web Services
Client Protocol 1.1.4322.573)\r \n
SOAPAction: "http://s.mappoint.net/mappoint-30/FindAddress"\r\ n
Authorization: Digest username="10628 8", realm="MapPoint ",
nonce="8033d257 de12190a0484875 21100021388b534 e89ffd4bad42926 66e620c",
uri="/Find-30/FindService.asm x",
response="0ff36 d6cbaf353f4aba1 83cef52d1de9", qop=auth, nc=00000001,
cnonce="8
Content-length: 0\r\n
Expect: 100-continue\r\n
\r\n

0000 00 0d 88 36 0e 3d 00 40 ca 6c a0 01 08 00 45 00
....6.=.@.l.... E.
0010 02 56 77 97 40 00 80 06 44 fa c0 a8 00 6a 40 04
..Vw.@...D....j @.
0020 3a fa 0c 6f 00 50 88 c9 eb f1 f0 ea 88 6f 50 18
:..o.P.......oP .
0030 fc 00 91 43 00 00 50 4f 53 54 20 2f 46 69 6e 64 ...C..POST
/Find
0040 2d 33 30 2f 46 69 6e 64 53 65 72 76 69 63 65 2e
-30/FindService.
0050 61 73 6d 78 20 48 54 54 50 2f 31 2e 31 0d 0a 48 asmx
HTTP/1.1..H
0060 6f 73 74 3a 20 66 69 6e 64 76 33 2e 73 74 61 67 ost:
findv3.stag
0070 69 6e 67 2e 6d 61 70 70 6f 69 6e 74 2e 6e 65 74
ing.mappoint.ne t
0080 0d 0a 41 63 63 65 70 74 2d 45 6e 63 6f 64 69 6e
...Accept-Encodin
0090 67 3a 20 69 64 65 6e 74 69 74 79 0d 0a 55 73 65 g:
identity..Use
00a0 72 2d 41 67 65 6e 74 3a 20 4d 6f 7a 69 6c 6c 61 r-Agent:
Mozilla
00b0 2f 34 2e 30 20 28 63 6f 6d 70 61 74 69 62 6c 65 /4.0
(compatible
00c0 3b 20 4d 53 49 45 20 36 2e 30 3b 20 4d 53 20 57 ; MSIE 6.0; MS
W
00d0 65 62 20 53 65 72 76 69 63 65 73 20 43 6c 69 65 eb Services
Clie
00e0 6e 74 20 50 72 6f 74 6f 63 6f 6c 20 31 2e 31 2e nt Protocol
1.1.
00f0 34 33 32 32 2e 35 37 33 29 0d 0a 53 4f 41 50 41
4322.573)..SOAP A
0100 63 74 69 6f 6e 3a 20 22 68 74 74 70 3a 2f 2f 73 ction:
"http://s
0110 2e 6d 61 70 70 6f 69 6e 74 2e 6e 65 74 2f 6d 61
..mappoint.net/ma
0120 70 70 6f 69 6e 74 2d 33 30 2f 46 69 6e 64 41 64
ppoint-30/FindAd
0130 64 72 65 73 73 22 0d 0a 41 75 74 68 6f 72 69 7a
dress"..Authori z
0140 61 74 69 6f 6e 3a 20 44 69 67 65 73 74 20 75 73 ation: Digest
us
0150 65 72 6e 61 6d 65 3d 22 31 30 36 32 38 38 22 2c
ername="106288" ,
0160 20 72 65 61 6c 6d 3d 22 4d 61 70 50 6f 69 6e 74
realm="MapPoint
0170 22 2c 20 6e 6f 6e 63 65 3d 22 38 30 33 33 64 32 ",
nonce="8033d2
0180 35 37 64 65 31 32 31 39 30 61 30 34 38 34 38 37
57de12190a04848 7
0190 35 32 31 31 30 30 30 32 31 33 38 38 62 35 33 34
521100021388b53 4
01a0 65 38 39 66 66 64 34 62 61 64 34 32 39 32 36 36
e89ffd4bad42926 6
01b0 36 65 36 32 30 63 22 2c 20 75 72 69 3d 22 2f 46 6e620c",
uri="/F
01c0 69 6e 64 2d 33 30 2f 46 69 6e 64 53 65 72 76 69
ind-30/FindServi
01d0 63 65 2e 61 73 6d 78 22 2c 20 72 65 73 70 6f 6e ce.asmx",
respon
01e0 73 65 3d 22 30 66 66 33 36 64 36 63 62 61 66 33
se="0ff36d6cbaf 3
01f0 35 33 66 34 61 62 61 31 38 33 63 65 66 35 32 64
53f4aba183cef52 d
0200 31 64 65 39 22 2c 20 71 6f 70 3d 61 75 74 68 2c 1de9",
qop=auth,
0210 20 6e 63 3d 30 30 30 30 30 30 30 31 2c 20 63 6e nc=00000001,
cn
0220 6f 6e 63 65 3d 22 38 61 63 33 34 34 37 33 63 33
once="8ac34473c 3
0230 33 32 66 61 38 66 22 0d 0a 43 6f 6e 74 65 6e 74
32fa8f"..Conten t
0240 2d 6c 65 6e 67 74 68 3a 20 30 0d 0a 45 78 70 65 -length:
0..Expe
0250 63 74 3a 20 31 30 30 2d 63 6f 6e 74 69 6e 75 65 ct:
100-continue
0260 0d 0a 0d 0a ....

No. Time Source Destination
Protocol Info
16 0.950469 64.4.58.250 192.168.0.106 HTTP
HTTP/1.1 401 Unauthorized

Frame 16 (387 bytes on wire, 387 bytes captured)
Arrival Time: Sep 3, 2005 23:57:20.680691 000
Time delta from previous packet: 0.193225000 seconds
Time since reference or first frame: 0.950469000 seconds
Frame Number: 16
Packet Length: 387 bytes
Capture Length: 387 bytes
Protocols in frame: eth:ip:tcp:http
Ethernet II, Src: D-Link_36:0e:3d (00:0d:88:36:0e :3d), Dst:
FirstInt_6c:a0: 01 (00:40:ca:6c:a0 :01)
Destination: FirstInt_6c:a0: 01 (00:40:ca:6c:a0 :01)
Source: D-Link_36:0e:3d (00:0d:88:36:0e :3d)
Type: IP (0x0800)
Internet Protocol, Src: 64.4.58.250 (64.4.58.250), Dst: 192.168.0.106
(192.168.0.106)
Version: 4
Header length: 20 bytes
Differentiated Services Field: 0x00 (DSCP 0x00: Default; ECN: 0x00)
0000 00.. = Differentiated Services Codepoint: Default (0x00)
.... ..0. = ECN-Capable Transport (ECT): 0
.... ...0 = ECN-CE: 0
Total Length: 373
Identification: 0x395f (14687)
Flags: 0x04 (Don't Fragment)
0... = Reserved bit: Not set
.1.. = Don't fragment: Set
..0. = More fragments: Not set
Fragment offset: 0
Time to live: 108
Protocol: TCP (0x06)
Header checksum: 0x9813 [correct]
Source: 64.4.58.250 (64.4.58.250)
Destination: 192.168.0.106 (192.168.0.106)
Transmission Control Protocol, Src Port: http (80), Dst Port: 3183
(3183), Seq: 1, Ack: 559, Len: 333
Source port: http (80)
Destination port: 3183 (3183)
Sequence number: 1 (relative sequence number)
Next sequence number: 334 (relative sequence number)
Acknowledgement number: 559 (relative ack number)
Header length: 20 bytes
Flags: 0x0018 (PSH, ACK)
0... .... = Congestion Window Reduced (CWR): Not set
.0.. .... = ECN-Echo: Not set
..0. .... = Urgent: Not set
...1 .... = Acknowledgment: Set
.... 1... = Push: Set
.... .0.. = Reset: Not set
.... ..0. = Syn: Not set
.... ...0 = Fin: Not set
Window size: 17122
Checksum: 0x0f4b [correct]
SEQ/ACK analysis
This is an ACK to the segment in frame: 15
The RTT to ACK the segment was: 0.193225000 seconds
Hypertext Transfer Protocol
HTTP/1.1 401 Unauthorized\r\ n
Request Version: HTTP/1.1
Response Code: 401
Connection: close\r\n
Date: Sat, 03 Sep 2005 22:00:41 GMT\r\n
Server: Microsoft-IIS/6.0\r\n
P3P:CP="BUS CUR CONo FIN IVDo ONL OUR PHY SAMo TELo"\r\n
X-Powered-By: ASP.NET\r\n
WWW-Authenticate: Digest qop="auth", realm="MapPoint ",
nonce="13c38f35 7408a1fc1484875 21100702d0fadd0 5e6b7cf54806565 713c790"\r\n
Content-Length: 0\r\n
\r\n

0000 00 40 ca 6c a0 01 00 0d 88 36 0e 3d 08 00 45 00
..@.l.....6.=.. E.
0010 01 75 39 5f 40 00 6c 06 98 13 40 04 3a fa c0 a8
..u9_@.l...@.:. ..
0020 00 6a 00 50 0c 6f f0 ea 88 6f 88 c9 ee 1f 50 18
..j.P.o...o.... P.
0030 42 e2 0f 4b 00 00 48 54 54 50 2f 31 2e 31 20 34 B..K..HTTP/1.1
4
0040 30 31 20 55 6e 61 75 74 68 6f 72 69 7a 65 64 0d 01
Unauthorized.
0050 0a 43 6f 6e 6e 65 63 74 69 6f 6e 3a 20 63 6c 6f .Connection:
clo
0060 73 65 0d 0a 44 61 74 65 3a 20 53 61 74 2c 20 30 se..Date: Sat,
0
0070 33 20 53 65 70 20 32 30 30 35 20 32 32 3a 30 30 3 Sep 2005
22:00
0080 3a 34 31 20 47 4d 54 0d 0a 53 65 72 76 65 72 3a :41
GMT..Server:
0090 20 4d 69 63 72 6f 73 6f 66 74 2d 49 49 53 2f 36
Microsoft-IIS/6
00a0 2e 30 0d 0a 50 33 50 3a 43 50 3d 22 42 55 53 20 .0..P3P:CP="BUS

00b0 43 55 52 20 43 4f 4e 6f 20 46 49 4e 20 49 56 44 CUR CONo FIN
IVD
00c0 6f 20 4f 4e 4c 20 4f 55 52 20 50 48 59 20 53 41 o ONL OUR PHY
SA
00d0 4d 6f 20 54 45 4c 6f 22 0d 0a 58 2d 50 6f 77 65 Mo
TELo"..X-Powe
00e0 72 65 64 2d 42 79 3a 20 41 53 50 2e 4e 45 54 0d red-By:
ASP.NET.
00f0 0a 57 57 57 2d 41 75 74 68 65 6e 74 69 63 61 74
..WWW-Authenticat
0100 65 3a 20 44 69 67 65 73 74 20 71 6f 70 3d 22 61 e: Digest
qop="a
0110 75 74 68 22 2c 20 72 65 61 6c 6d 3d 22 4d 61 70 uth",
realm="Map
0120 50 6f 69 6e 74 22 2c 20 6e 6f 6e 63 65 3d 22 31 Point",
nonce="1
0130 33 63 33 38 66 33 35 37 34 30 38 61 31 66 63 31
3c38f357408a1fc 1
0140 34 38 34 38 37 35 32 31 31 30 30 37 30 32 64 30
48487521100702d 0
0150 66 61 64 64 30 35 65 36 62 37 63 66 35 34 38 30
fadd05e6b7cf548 0
0160 36 35 36 35 37 31 33 63 37 39 30 22 0d 0a 43 6f
6565713c790"..C o
0170 6e 74 65 6e 74 2d 4c 65 6e 67 74 68 3a 20 30 0d ntent-Length:
0.
0180 0a 0d 0a ...

[Jakob Simon-Gaarde]

Just for the record. After accepting that pythons build-in digest
authentication (HTTPDigestAuth Handler) does *NOT* work, I made my own
digest authentication handler and built it into ZSI, they have recieved
and accepted the patch so it should be part of the next ZSI release
(current 1.7).

I have tested the implementation with Microsoft MapPoint services,
seems OK:

#-------------------------------------
#Usage example (MapPoint SOAP Services):
#-------------------------------------
from CommonService_s ervices import *
loc = FindServiceLoca tor()
import sys
import ZSI
kw={'tracefile' :sys.stdout, 'auth' : (
ZSI.AUTH.httpdi gest, 'username', 'passwd') }
portType = loc.getFindServ iceSoap(**kw)

AddressLine='Le rgravsvej 28'
PostalCode='866 0'
CountryRegion=' DK'
InputAddress = ns1.Address_Def ()
InputAddress._A ddressLine = AddressLine
InputAddress._P ostalCode = PostalCode
InputAddress._C ountryRegion = CountryRegion

specification = ns1.FindAddress Specification_D ef()
specification._ InputAddress = InputAddress
specification._ DataSourceName = 'MapPoint.EU'
request = FindAddressSoap InWrapper()

request._specif ication = specification

res = portType.FindAd dress(request)
#-----------------------------------------

Best regards
Jakob Simon-Gaarde