473,836 Members | 2,012 Online
Bytes | Software Development & Data Engineering Community
+ Post

Home Posts Topics Members FAQ

xmlrpc, httplib and SSL

I have just spent several weeks mashing xmlrpc, httplib and SSL (from
M2Crypto) to work together. The current standard library has several
problems:

- Builtin SSL is pretty much useless if you actually care
about security
- Poor HTTP authentication support
- No server side stuff (SSL, HTTP authentication etc)
- Pathological coding to ensure that at most one request is
sent on a connection, rather than reusing an already open
connection (http/1.1 keepalives)
- Pathological coding to ensure that connections are closed
from as many different places as possible
- A broken model for dealing with who owns a connection socket.
It all started when someone added a 'makefile' method that
returned a dup'ed file descriptor since all there are
all those close calls everywhere, so the reference counting
sort of works on UNIX (where dup is a normal operation).
This leads to even more heroic coding to work around the
explicitly coded close's everywhere, and implement yet
another layer of reference counting and encapsulation.
- No real possibility of dealing with things like automatically
reopening connections (eg if you have an HTTP/1.1 connection,
make a request, stay idle for so long the other end closes the
connection, and then try to send a new one)

I now have code that works for me and my project. However it
wouldn't really be appropriate for going back into the standard
library because it spends most of its time having to subvert the
design and implementation of the existing classes.

However I was wondering if anyone was working on fixing the
(IMHO horrible) mess and wants any moral support?

Roger

Jul 18 '05 #1
4 3813

[snip summary of changes]

Roger> I now have code that works for me and my project. However it
Roger> wouldn't really be appropriate for going back into the standard
Roger> library because it spends most of its time having to subvert the
Roger> design and implementation of the existing classes.

Roger> However I was wondering if anyone was working on fixing the (IMHO
Roger> horrible) mess and wants any moral support?

Sounds like you've done most of the work already. Why not post a few
patches to SF for the various affected modules?

Skip
Jul 18 '05 #2
Skip Montanaro wrote:
[snip summary of changes]

Roger> I now have code that works for me and my project. However it
Roger> wouldn't really be appropriate for going back into the standard
Roger> library because it spends most of its time having to subvert the
Roger> design and implementation of the existing classes.

Roger> However I was wondering if anyone was working on fixing the (IMHO
Roger> horrible) mess and wants any moral support?

Sounds like you've done most of the work already. Why not post a few
patches to SF for the various affected modules?


The standard modules need to be redesigned! And M2Crypto would have
to be the standard SSL. And as I state in the first paragraph you quote,
it has to subvert the existing classes. Think of my code as the
result of using chainsaws and band-aids all over the existing code.
The end result works fine for me and my project. But I wouldn't
remotely want to change the existing library to anything like it.

Here were the goals I was trying to meet. The current Python library
does not remotely meet them.

XML-RPC client:

- Connections are made over SSL
- A callback to verify the certificate if there is no
CA or other defined mechanism
- HTTP/1.1 is used, and the connection is reused if
still open
- HTTP authentication is used
- Automatic reopen of connection if above one doesn't
work (eg remote end closed it after timeout)
- Be thread safe (eg one connection per thread - don't
reuse across threads)

XML-RPC server:

- Connections are accepted over SSL
- Callbacks to verify incoming connections, certificates and
credentials of the connections
- Verification of HTTP authentication information
- Connections are kept open/keep-alive (HTTP/1.1)
- Use of a bounded thread pool that handles the connections

Roger
Jul 18 '05 #3

On Mar 27, 2004, at 5:03 AM, Roger Binns wrote:
However I was wondering if anyone was working on fixing the
(IMHO horrible) mess and wants any moral support?


I wanted to do keep the connections on which my clients do XMLRPC calls
open, and after staring at the xmlrpclib.py source for a while, came up
with the class at the bottom of the message. Just thought I would post
it here in the spirit of sharing. You would use it like this:

import httplib, xmlrpclib
s = xmlrpclib.Serve rProxy('http://SOMEURL',
transport=Persi stTransport())

And then use as normal. When any error occurs, the connection is
closed, seems a bit pessimistic, but I didn't want to do anything more
fancy.

cheers,

Etienne Posthumus
---
http://www.mnemosyne.org/
Cultural Heritage Research
Python, Zope, XML expertise for hire.
Amsterdam, Nederland
----

class PersistTranspor t(xmlrpclib.Tra nsport):
'''Provides a Transport for the xmlrpclib that uses httplib
supporting persistent connections
Does not close the connection after each request.
'''
connection = None

def request(self, host, handler, request_body, verbose=0):
if not self.connection :
host, extra_headers, x509 = self.get_host_i nfo(host)
self.connection = httplib.HTTPCon nection(host)
self.headers = {"User-Agent" : self.user_agent ,
"Content-Type" : "text/xml",
"Accept": "text/xml"}
if extra_headers:
for key, item in extra_headers:
self.headers[key] = item

self.headers["Content-Length"] = str(len(request _body))
self.connection .request('POST' , handler, request_body,
self.headers)
r = self.connection .getresponse()
if r.status != 200:
self.connection .close()
self.connection = None
raise xmlrpclib.Proto colError( host + handler, r.status,
r.reason, '' )
data = r.read()
p, u = self.getparser( )
p.feed(data)
p.close()
return u.close()
Jul 18 '05 #4
> I wanted to do keep the connections on which my clients do XMLRPC calls
open, and after staring at the xmlrpclib.py source for a while, came up
with the class at the bottom of the message.


Are you sure you are actually getting persistent connections? The
code will auto-close and auto-open the connection even just keeping
the same HTTP connection object.

In order for the connection to not be auto-closed, the remote end
must return a HTTP/1.1 response (see httplib.HTTPRes ponse._check_cl ose).

If your server end is Python, then it will always close the connection
unless the request was HTTP/1.1 *and* a 'Connection: keep-alive' header
was sent, which the Python client does not do.

The above is all true for Python 2.3. For 2.2 it does HTTP/1.0 IIRC
so you have no hope.

Roger
Jul 18 '05 #5

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

0
1840
by: glin | last post by:
Hi I am trying to integrate the xmlrpc server into a class, does anyone know how to get it working? test.html: <html> <head> <title>XMLRPC Test</title> <script src="jsolait/init.js"></script> <script src="jsolait/lib/urllib.js"></script> <script src="jsolait/lib/xml.js"></script>
0
1744
by: Juan Carlos CORUÑA | last post by:
Hello all, I'm trying to create a COM Server with an embedded xmlrpc server. Here is way it must work: - The client application (programmed with a COM capable language) instantiates my COM server (programmed with python). - The COM server must have a connect interface in order to let the client application process the xmlrpc request. - After executing a "serveforever" method on the COM server it begins
1
1960
by: Ben | last post by:
Hi all, I'm trying to connect to URL such as http://betty.userland.com through my LAN proxy. I've read earlier postings on this newsgroups but still i haven't been able to figure out the way to do it. It doesn't look so simple. It's very easy to do it using httplib and urllib but xmlrpc looks a bit complicated. Can someone tell me if there is an easy way around this problem? My code is like this:
0
1341
by: penglish1 | last post by:
Hi, I'm trying to get xmlrpc working with usernames and passwords and having some issues. This is on Linux (WBEL3.0R1). First of all with python 2.2.3 which comes with WBEL the following simple code fails (passwords & server names altered to protect the innocent): #!/usr/bin/python
6
5366
by: Michael Urman | last post by:
Hi. I'm a user of python for about 3 years now. I've written a client-server application that uses SimpleXMLRPCServer and xmlrpclib.ServerProxy to communicate. It's intended to be used by a single-person as a backend and GUI frontend. I've got it running great. Much stabler than my custom RPC I'd tried before. I've used the default support available by these classes. Thus it will run on a potentially public TCP/IP port. As the...
0
1202
by: Laszlo Nagy | last post by:
I'm running a service on a machine. The service is written in Python (of course) and it connects to an XMLRPC server periodically. It recreates the ServerProxy instance each time it needs to connect to the RPC server. The server is created with this code: server = xmlrpclib.ServerProxy(local.SERVER_URL,allow_none=True) # local.SERVER_URL is something like 'https://myserver.com:3421'
1
3260
by: Thomas Liesner | last post by:
Hi all, this may have been asked before, but as a newbie with xmlrpc i can't find any suitable info on that. Sorry. I am trying to write a simple xmlrpc-client in python and the server i am trying to receive data from requires http auth digest. The info on xmlrpclib covers auth basic thrugh url econding such as "user:pass@server", but no auth digest. Is there any other library i could use for that or can i write some sort
0
2754
by: Benjamin Grieshaber | last post by:
Hi, I´m on SuSE 9.3 with xmlrpc-c and xmlrpc-c-devel installed (ver. 0.9.10) I tried to compile php with xmlrpc support and got the following errors: ext/xmlrpc/.libs/xmlrpc-epi-php.o(.text+0x359): In function `set_zval_xmlrpc_type': /php-5.2.5/ext/xmlrpc/xmlrpc-epi-php.c:1313: undefined reference to `XMLRPC_CreateValueDateTime_ISO8601'
0
1827
by: Gustavo Rahal | last post by:
Hi I'm trying to build a xmlrpc client that uses a proxy to connect to a xmlrpc server in https. I've googled and came up with a code snippet that doesn't actually work. What is missing?
0
9812
marktang
by: marktang | last post by:
ONU (Optical Network Unit) is one of the key components for providing high-speed Internet services. Its primary function is to act as an endpoint device located at the user's premises. However, people are often confused as to whether an ONU can Work As a Router. In this blog post, we’ll explore What is ONU, What Is Router, ONU & Router’s main usage, and What is the difference between ONU and Router. Let’s take a closer look ! Part I. Meaning of...
0
9658
by: Hystou | last post by:
Most computers default to English, but sometimes we require a different language, especially when relocating. Forgot to request a specific language before your computer shipped? No problem! You can effortlessly switch the default language on Windows 10 without reinstalling. I'll walk you through it. First, let's disable language synchronization. With a Microsoft account, language settings sync across devices. To prevent any complications,...
0
10824
Oralloy
by: Oralloy | last post by:
Hello folks, I am unable to find appropriate documentation on the type promotion of bit-fields when using the generalised comparison operator "<=>". The problem is that using the GNU compilers, it seems that the internal comparison operator "<=>" tries to promote arguments from unsigned to signed. This is as boiled down as I can make it. Here is my compilation command: g++-12 -std=c++20 -Wnarrowing bit_field.cpp Here is the code in...
0
10533
jinu1996
by: jinu1996 | last post by:
In today's digital age, having a compelling online presence is paramount for businesses aiming to thrive in a competitive landscape. At the heart of this digital strategy lies an intricately woven tapestry of website design and digital marketing. It's not merely about having a website; it's about crafting an immersive digital experience that captivates audiences and drives business growth. The Art of Business Website Design Your website is...
0
10244
tracyyun
by: tracyyun | last post by:
Dear forum friends, With the development of smart home technology, a variety of wireless communication protocols have appeared on the market, such as Zigbee, Z-Wave, Wi-Fi, Bluetooth, etc. Each protocol has its own unique characteristics and advantages, but as a user who is planning to build a smart home system, I am a bit confused by the choice of these technologies. I'm particularly interested in Zigbee because I've heard it does some...
1
7775
isladogs
by: isladogs | last post by:
The next Access Europe User Group meeting will be on Wednesday 1 May 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM). In this session, we are pleased to welcome a new presenter, Adolph Dupré who will be discussing some powerful techniques for using class modules. He will explain when you may want to use classes instead of User Defined Types (UDT). For example, to manage the data in unbound forms. Adolph will...
0
6975
by: conductexam | last post by:
I have .net C# application in which I am extracting data from word file and save it in database particularly. To store word all data as it is I am converting the whole word file firstly in HTML and then checking html paragraph one by one. At the time of converting from word file to html my equations which are in the word document file was convert into image. Globals.ThisAddIn.Application.ActiveDocument.Select();...
0
5644
by: TSSRALBI | last post by:
Hello I'm a network technician in training and I need your help. I am currently learning how to create and manage the different types of VPNs and I have a question about LAN-to-LAN VPNs. The last exercise I practiced was to create a LAN-to-LAN VPN between two Pfsense firewalls, by using IPSEC protocols. I succeeded, with both firewalls in the same network. But I'm wondering if it's possible to do the same thing, with 2 Pfsense firewalls...
3
3103
bsmnconsultancy
by: bsmnconsultancy | last post by:
In today's digital era, a well-designed website is crucial for businesses looking to succeed. Whether you're a small business owner or a large corporation in Toronto, having a strong online presence can significantly impact your brand's success. BSMN Consultancy, a leader in Website Development in Toronto offers valuable insights into creating effective websites that not only look great but also perform exceptionally well. In this comprehensive...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.