Multiple XML-RPC calls over the same connection?

[The Fumigator]

Hi.

I want to be able to create a persistent connection between an XML-RPC
client and server... i.e. I want to be able to login once, carry
out a number of calls and then logout rather than send login info for
every call.

Is there a way to do this with the xml-rpc code in the standard library?

Thanks in advance

[Diez B. Roggisch]

The Fumigator wrote:

Hi.

I want to be able to create a persistent connection between an XML-RPC
client and server... i.e. I want to be able to login once, carry
out a number of calls and then logout rather than send login info
every call.
Is there a way to do this with the xml-rpc code in the standard library?

Not to my knowlegde - and the reason is not python, but the primitiveness of
xmlrpc. It simply doesn't allow for connection state to be associated.

What I do is to create handles, and pass these around - that at least limits
and sort of normalizes the api usage:

proxy = xmlrpclib.Server('http://localhost:7080/')
ak = proxy.authenticate(user, password)
print proxy.some_method(ak, argument)

If you want stateful connection, use better IPC-standards, as CORBA - its
way more advanced. SOAP gets attention lately, but its similarily limited
as xmlrpc, and while session state associoated with soap connections is
possible sometimes, its not standarized....
--
Regards,

Diez B. Roggisch

[Jeremy Jones]

The Fumigator wrote:

Hi.

I want to be able to create a persistent connection between an XML-RPC
client and server... i.e. I want to be able to login once, carry
out a number of calls and then logout rather than send login info for
every call.

Is there a way to do this with the xml-rpc code in the standard library?

Thanks in advance

What do you mean by login, logout and login info? Do you mean establish
connection, close connection and <I don't know what would be equivalent
for login info>? Are you wanting a keepalive? If so, I don't know of
how to do that with the standard library. If you've got some sort of
login mechanism on your XMLRPCServer, I would assume you would have to
pass back whatever "key" you received upon login per call.... But I
guess it depends on what you mean.
Jeremy Jones

[Skip Montanaro]

I want to be able to create a persistent connection between an
XML-RPC client and server... i.e. I want to be able to login once,
carry out a number of calls and then logout rather than send login
info for every call. Is there a way to do this with the xml-rpc code in the standard
library?

Look at the multicall capability of the xmlrpclib library. It may do what
you want.

Skip

[Diez B. Roggisch]

> Look at the multicall capability of the xmlrpclib library. It may do what

you want.

Verifying my response I found that, too - but thats more a bundling of calls
together instead of what the op wanted - a way to have some server-side
"cookie" that identifies an incoming call as authenticated. That's not
possible.
--
Regards,

Diez B. Roggisch

[Roger Binns]

Diez B. Roggisch wrote:

a way to have some
server-side "cookie" that identifies an incoming call as
authenticated. That's not possible.

The standard pattern for doing that is having an xmlrpc call
that you supply username/password/other credentials to. It
then responds with a pair of tokens. You then use them as
part of the URL where usernames and passwords are usually
used. For example:

sp=xmlrpclib.ServerProxy("http://foo.example.com")
c1,c2=sp.login("username", "password", "etc")

sp=xmlrpclib.ServerProxy("http://%s:[e-mail]" % (c1,c2))
sp.DoSomething(1,2)
sp.DoSomethingElse(3,4)

This will work fine with the current client side xmlrpclib. If
you server side is written in Python then you will need to make
a derived request handler that grabs and verifies the authentication
headers before calling the intended routines (except for the login
function :-)

Roger

[Diez B. Roggisch]

Hi,

The standard pattern for doing that is having an xmlrpc call
that you supply username/password/other credentials to. It
then responds with a pair of tokens. You then use them as
part of the URL where usernames and passwords are usually
used. For example:

sp=xmlrpclib.ServerProxy("http://foo.example.com")
c1,c2=sp.login("username", "password", "etc")

sp=xmlrpclib.ServerProxy("http://%s:[e-mail]" % (c1,c2))
sp.DoSomething(1,2)
sp.DoSomethingElse(3,4)

This will work fine with the current client side xmlrpclib. If
you server side is written in Python then you will need to make
a derived request handler that grabs and verifies the authentication
headers before calling the intended routines (except for the login
function :-)

While I have to admit that it is a nice trick, its something I expect beeing
part of a IPC-standard - otherwise, it renders the standard short of beeing
useless.

There are alternatives out there, like corba, which allow for much better
implementations - in terms of ease of use from the api side as well as
performance and interoperability. Having to tweak a standard to make it
work in a certainly common pattern strikes me as awkward. Unfortunately, I
have to use xmlrpc myself as php corba support isn't too good - but I keep
it to a minimum, and its dead ugly...

--
Regards,

Diez B. Roggisch

[Jeremy Bowers]

On Fri, 29 Oct 2004 20:49:50 +0200, Diez B. Roggisch wrote:

While I have to admit that it is a nice trick, its something I expect beeing
part of a IPC-standard - otherwise, it renders the standard short of beeing
useless.

XML-RPC was designed from the get go to be very, very simple, as in
"person skilled in a language ought to be able to get one going in a
matter of hours". It breaks out of that domain, but it is nice to have a
cross-platform RPC mechanism that meets that description. If I want to get
a weather report for zip-code XXXXX, I shouldn't need CORBA.

It isn't useless, but it definitely has established its niche at the lower
end of the scale.

I would point out the XML-RPC is built on HTTP and XML and theoretically
inherits the benefits and disadvantages of both. Specifically, if your
server and client supported it, you should be able to use Cookie: headers
just as you would in HTTP. That would sort of be a grey area in the
standard.

(It's probably about time for XML-RPC 1.1 that seperates the data encoding
from the transport; right now according to the standard they are
intertwined and "XML-RPC over SMTP" or "over Jabber" are meaningless
phrases; those things are just "XML-RPC-like". Then maybe we could allow
various transports to be used more fully, according to the defined
transport; Jabber is another example where the transport has useful
capabilities but they are just thrown away for RPC purposes.)

[Diez B. Roggisch]

XML-RPC was designed from the get go to be very, very simple, as in
"person skilled in a language ought to be able to get one going in a
matter of hours". It breaks out of that domain, but it is nice to have a
cross-platform RPC mechanism that meets that description. If I want to get
a weather report for zip-code XXXXX, I shouldn't need CORBA.
And not much beyond that - more or less all serious applications have _some_
sort of authentication scheme - and thus associating state with a
connection is neccessary in nearly all but the most trivial applications.

And I think we can agree that these days a object orientied interface is how
apis are designed, and thats also beyond the scope of xmlrpc/soap.

Its not so complicated to get a corba server running - at least imho.
Writing a idl is a thing every programmer should be able to do. if its a
well-designed one - well, thats a totally different beast, and beyond rpc
in general - but the inherent clumsiness of xmlrpc/soap based interface
doesn't help here, too.
It isn't useless, but it definitely has established its niche at the lower
end of the scale.

I would point out the XML-RPC is built on HTTP and XML and theoretically
inherits the benefits and disadvantages of both. Specifically, if your
server and client supported it, you should be able to use Cookie: headers
just as you would in HTTP. That would sort of be a grey area in the
standard.

Exactly - its grey. SOAP is grey in that field, too. So it renders e.g. .NET
and AXIS un-interoperable when more than the most trivial tasks are
planned.

I've been in contact with xmlrpc/soap after I became aquainted with corba,
and I have to say that to me its a huge step back in time. But it gets the
attention these days, due to clever marketing and the fact that its based
on xml - something that seems to be a must these days.
--
Regards,

Diez B. Roggisch

[Irmen de Jong]

Diez B. Roggisch wrote:

Exactly - its grey. SOAP is grey in that field, too. So it renders e.g. .NET
and AXIS un-interoperable when more than the most trivial tasks are
planned.
Which is a big pain in the butt! We had to deal with this a few times
in our company, and (if I recall correctly) had to decide on a custom
XML structure that was sent inside a very very basic SOAP wrapper.
(the client wanted SOAP, they got SOAP... :-( )
I've been in contact with xmlrpc/soap after I became aquainted with corba,
and I have to say that to me its a huge step back in time. But it gets the
attention these days, due to clever marketing and the fact that its based
on xml - something that seems to be a must these days.

Back in 2002 I wrote a very extensive article/whitepaper on this subject.
It's available on the OMG website:

http://www.omg.org/news/whitepapers/CORBA_vs_SOAP1.pdf

Interesting stuff inside, if you're into this kind of technology.
(it's PDF, other formats also available on my own website)
--Irmen de Jong

[Roger Binns]

Jeremy Bowers wrote:

I would point out the XML-RPC is built on HTTP and XML and
theoretically inherits the benefits and disadvantages of both.

The default implementation uses HTTP as a transport, but that is
not a requirement. You just have to be able to serialise the request
and response parameters into an XML stream. The xmlrplic loads and
dumps functions will do that for you.

In one of my projects I actually use XML-RPC over SSH and deal with
the authentication etc at the SSH level. (I use the excellent pure
Python paramiko library for the SSH protocol part).

Roger

[The Fumigator]

Some good stuff here... Thanks people.

[Jeremy Bowers]

On Fri, 29 Oct 2004 18:47:57 -0700, Roger Binns wrote:

Jeremy Bowers wrote:

I would point out the XML-RPC is built on HTTP and XML and
theoretically inherits the benefits and disadvantages of both.

The default implementation uses HTTP as a transport, but that is
not a requirement.

No, it is actually written into the spec.

http://xmlrpc.scripting.com/spec

"An XML-RPC message is an HTTP-POST request." - right there in the
overview, and the author means it; he's said it directly to me.

Like I said, I think it is time to revise it to make the encoding
separate from the transport, but it already has been so revised out in the
wild.

[Jeremy Bowers]

On Sat, 30 Oct 2004 00:39:13 +0200, Diez B. Roggisch wrote:

Its not so complicated to get a corba server running - at least imho.
http://xmlrpc.scripting.com/director...mplementations

I would be surprised you could get one in all those environments, and
doesn't CORBA have ORB interop concerns rather often?

Besides, how exactly do you get CORBA going in Javascript? Squeak, from a
quick web search, doesn't have an ORB either. That's just two I plucked
out of the list.

Yes, it is in some sense a step back. That's the whole point. If it
doesn't meet your needs, don't use it, but it isn't useless, either; if
CORBA met all needs XML-RPC would never have been created, as it
post-dates CORBA by quite a lot.

I've used it in a few situations where CORBA either doesn't exist, or
would be damned inconvenient; a couple of other enviroments without CORBA,
importing my weblog from one system to another, that sort of thing.

I know CORBA seems easy to you, I'm willing to bet CORBA would seem easy
to me. (It can't be worse than COM, from what I hear, and I've engaged in
mortal combat with that technology a couple of times.) But there is
still a lot of programmers/scripters who really can't follow it, and
XML-RPC works better for them. (I speak of users, not implementers.)
But it gets the
attention these days, due to clever marketing and the fact that its
based on xml - something that seems to be a must these days.

The latter most likely rather than the former; the "marketing" for XML-RPC
has been "Here it is, use it if you like." The is no significant company
behind it, pushing for it, just this one guy, really. SOAP is the one with
the big marketing push.

BTW, the usual solution to the authentication problem is to require a
username and password on each call. It has its disadvantages, yes, but it
actually has a couple of security advantages, too. (There is no persistant
connection you can "break into" and get the privs of the authenticated
account; I forget what this attack is called. **Obviously** there are
other concerns, I'm just pointing this one thing out.) If authentication
takes a long time, cache it on the server side. It may seem clumsy but it
isn't much different than the HTTP solution with cookies (one way or
another you are passing a handful of bytes of extra state to the server),
and it's not like it isn't trivial to fix in any real language with a
wrapper or something. XML-RPC has real problems on large scale use but
authentication is hardly one of them. You don't use XML-RPC when
processing bajillions of queries where the overhead of parsing XML would
be signficant is your goal. Outside of that, while persistent connections
might be nice, they would complexify the spec and smack of premature
optimization.

[Roger Binns]

Jeremy Bowers wrote:

On Fri, 29 Oct 2004 18:47:57 -0700, Roger Binns wrote:

The default implementation uses HTTP as a transport, but that is
not a requirement.

No, it is actually written into the spec.

http://xmlrpc.scripting.com/spec

"An XML-RPC message is an HTTP-POST request." - right there in the
overview, and the author means it; he's said it directly to me.

Err, a few paragraphs before that it also says:

"This specification documents the XML-RPC protocol implemented
in UserLand Frontier 5.1."

If I want to talk to UserLand Frontier or be a replacement server
then I would have to (at least) do HTTP-POST support. I don't see
any requirement for a program to have to do HTTP-POST in order to
claim XML-RPC compliance, although I would certainly agree that
HTTP is the default and most common transport.

Roger

[Diez B. Roggisch]

Jeremy Bowers wrote:

http://xmlrpc.scripting.com/director...mplementations

I would be surprised you could get one in all those environments, and
doesn't CORBA have ORB interop concerns rather often?
Not to my knowledge - I've been sucessfully interoperating TAO, Jacorb and a
VB orb without any problems - and that was 4 years ago. There used to be
problems at some time, but they seem to be solved. CORBA is much more
mature than SOAP in that respect - and others as well.
Besides, how exactly do you get CORBA going in Javascript? Squeak, from a
quick web search, doesn't have an ORB either. That's just two I plucked
out of the list.
The OP didn't say much about his environment - so I can only speculate if
there are ORBs for them available. He wanted a feature xmlrpc didn't offer
to him, so I proposed one that for a majority of projects out there is
usable.
Yes, it is in some sense a step back. That's the whole point. If it
doesn't meet your needs, don't use it, but it isn't useless, either; if
CORBA met all needs XML-RPC would never have been created, as it
post-dates CORBA by quite a lot.
It isn't useless - I use it myself. But its very limited, and if I had the
chance not to use it, I wouldn't. Unfortunately, making php running corba
is a royal pain in the neck, so I had to use xmlrpc. SOAP support is better
in php, but python lacks in that field.
I know CORBA seems easy to you, I'm willing to bet CORBA would seem easy
to me. (It can't be worse than COM, from what I hear, and I've engaged in
mortal combat with that technology a couple of times.) But there is
still a lot of programmers/scripters who really can't follow it, and
XML-RPC works better for them. (I speak of users, not implementers.)
Might be - but everybody who feels in the need of IPC should be able to use
it. Invoking an idl compiler and extending the generated server classes
isn't exactly black magic. While the learning curve for corba might be a
little bit steeper in the first place, I think that the headaches caused by
then rolling out your own session/authentication mechanism justify for
this. But this is of course an opininon - I never observed a inexpirienced
programmer choosing either technique.

But it gets the
attention these days, due to clever marketing and the fact that its
based on xml - something that seems to be a must these days.

The latter most likely rather than the former; the "marketing" for XML-RPC
has been "Here it is, use it if you like." The is no significant company
behind it, pushing for it, just this one guy, really. SOAP is the one with
the big marketing push.

Sure, but the marketing hype after SOAP clearly helps xmlrpc - after all,
from a non-proggramers POV (read: management and adminstrators) they are
basically the same: http-driven, thus firewall-compatible, and use xml. I
very well know that soap is _more_ than xmlrpc, can use smtp and so on -
but in 95% of usecases, it will be used in conjunction with http, making it
really close to xmlrpc.

BTW, the usual solution to the authentication problem is to require a
username and password on each call. It has its disadvantages, yes, but it
actually has a couple of security advantages, too. (There is no persistant
connection you can "break into" and get the privs of the authenticated
account; I forget what this attack is called. **Obviously** there are
other concerns, I'm just pointing this one thing out.) If authentication
I see these two as equal - in fact, a session at least can only be captured
once and only in a limited time window, where using name/password everytime
gives an attacker the key to the system at any time. Preventing either of
them only works through a secured comm-channel, like https.

And furthermore, they're basically the same - its simply a piece of data
associated with the request - no big difference here.

SOAP at least allows for http headers to be used for this for the connection
as whole, so you can e.g. with axis establish a session mechanism. But this
exactly is one of the grey areas pointed out before - it won't interoperate
between implementatitons. And the xmlrpc-hack suggested by roger binns even
needs tweaking at a deeper level.
takes a long time, cache it on the server side. It may seem clumsy but it
isn't much different than the HTTP solution with cookies (one way or
another you are passing a handful of bytes of extra state to the server),
Exactly.
and it's not like it isn't trivial to fix in any real language with a
wrapper or something. XML-RPC has real problems on large scale use but
So you end up writing wrappers - I've done that myself, of course that works
- but corba _generates_ these wrappers for you....
authentication is hardly one of them. You don't use XML-RPC when
processing bajillions of queries where the overhead of parsing XML would
be signficant is your goal. Outside of that, while persistent connections
might be nice, they would complexify the spec and smack of premature
optimization.

Calling often wanted features premature optimization strikes me a bid odd -
nearly any application that needs to communicate over several calls needs
to have some sort of session scheme, with or without authentication. And as
the underlying protocols have means to deal with such information, I wonder
why the people specifying xmlrpc as well as soap didn't think of that -
after all, we're in the 21st century, and all serious webapps make use of
sessions - cookie or url-based. So why do the specs lack in that field?

To me it looks as if they're unneccesarily reinvent the wheel, and the wheel
is close to beeing square right now. Maybe it shapes up - bdt until then,
lots of time and money has been invested to overcome the shortcomings of a
square wheel.

--
Regards,

Diez B. Roggisch