Very slow opening of client connections to localhost with httplib

[Martin Fuzzey]

I am using xmlrpclib (based on httplib) in Python 2.3 on Mandrake
Linux.

When my client attempts to connect to a server using a
"http://localhost:port" style URL there is a long delay before the
connection is established (it finally works).

This certainly smells a name resolution problem but ping localhost,
telnet localhost etc all work fine.

Tracking down a bit it appears that the problem comes from the
getaddrinfo() call in code in HttpConnection.connect() :
def connect(self):
"""Connect to the host and port specified in __init__."""
msg = "getaddrinfo returns an empty list"
for res in socket.getaddrinfo(self.host, self.port, 0,
socket.SOCK_STREAM):

To test this I did :

import socket
socket.getaddrinfo("localhost", 10000, 0, socket.SOCK_STREAM)

and sure enough it blocked for several seconds before finally
returning 127.0.0.1

However
socket.gethostbyname("localhost")
returns immediately

If I remplace the 0 in the getaddrinfo call above by socket.AF_INET
all is fine as well.

So I did a tcpdump to see what it was sending to the nameserver and
got

18:36:01.790045 X.Y.Z.41.32961 > X.Y.Z.12.53: 42057+ AAAA?
localhost.besancon.parkeon.com. (48) (DF)
18:36:01.790526 X.Y.Z.12.53 > X.Y.Z.41.32961: 42057* 0/1/0 (95)
18:36:01.790652 X.Y.Z.41.32961 > X.Y.Z.12.53: 42058+ AAAA? localhost.
(27) (DF)
18:36:06.792898 X.Y.Z.41.32962 > X.Y.Z.215.53: 42058+ AAAA?
localhost. (27) (DF)
18:36:06.793134 X.Y.Z.215.53 > X.Y.Z.41.32962: 42058 ServFail 0/0/0
(27)
18:36:06.793301 X.Y.Z.41.32962 > X.Y.Z.12.53: 42058+ AAAA? localhost.
(27) (DF)
18:36:11.802998 X.Y.Z.41.32963 > X.Y.Z.215.53: 42058+ AAAA?
localhost. (27) (DF)
18:36:11.803222 X.Y.Z.215.53 > X.Y.Z.41.32963: 42058 ServFail 0/0/0
(27)
18:36:13.356344 X.Y.Z.12.53 > X.Y.Z.41.32959: 42056 ServFail 0/0/0
(27)

(my machine is X.Y.Z.41 and the NS is X.Y.Z.12)

When AF_INET or gethostbyname() is used there is no network
communication.

This appears to be a problem with IPV6 DNS queries?? (the AAAA
records). We don't use IPV6 but the 0 parameter (AF_UNSPEC) used in
httplib seems to cause them to be sent.

Anyone else had this problem or got any ideas?

The problem does NOT occur under windows.

Thanks,

Martin

[Peter Hansen]

Martin Fuzzey wrote:

To test this I did :

import socket
socket.getaddrinfo("localhost", 10000, 0, socket.SOCK_STREAM)

and sure enough it blocked for several seconds before finally
returning 127.0.0.1

However
socket.gethostbyname("localhost")
returns immediately

Any chance your /etc/host.conf, /etc/resolv.conf or /etc/hosts files
are buggering this up somehow? (It doesn't explain gethostbyname()
returning immediately, unless perhaps that function internally handles
localhost "magically".) Just a thought.

In my experience, delays like this generally result from host.conf
having "order bind,hosts" instead of the other way around, plus perhaps
an inappropriate "options timeout:xxx" value in resolv.conf.

I haven't ever seen an issue with localhost, however, but I've never
been on a machine that didn't have the right entry in /etc/hosts.

-Peter

[Jarkko Torppa]

In article <8c**************************@posting.google.com >, Martin Fuzzey wrote:

I am using xmlrpclib (based on httplib) in Python 2.3 on Mandrake
Linux.

When my client attempts to connect to a server using a
"http://localhost:port" style URL there is a long delay before the
connection is established (it finally works). .... import socket
socket.getaddrinfo("localhost", 10000, 0, socket.SOCK_STREAM)

and sure enough it blocked for several seconds before finally
returning 127.0.0.1

If I remplace the 0 in the getaddrinfo call above by socket.AF_INET
all is fine as well.

Try adding
::1 localhost

to your /etc/hosts

Or delete IPv6 from your kernel.

--
Jarkko Torppa, Elisa Internet