Twisted - extending portforward (simple example)

[Paul Moore]

I hit a problem yesterday with my mail connection. In a desparate
attempt to understand what was going on, I wanted to log the
connection traffic. After a bit of searching, I found a post on c.l.p
from Andrew Bennetts explaining how to run a port forwarder in 2 lines
using Twisted.

$ mktap portforward -p 8000 -h remote -d 20
$ twistd -f portforward.tap

This looked brilliant - all I needed to do was add logging. A quick
look (I was *very* short of time, so I couldn't spend long) showed me
no obvious way of adding a hook (at least, not one which wouldn't turn
the above 2-liner into something more complex, that I didn't have time
to work out...) So I gave in, and just added a print statement in the
twisted code.

This gave me the results I wanted, but left me wondering (and not
getting far in finding out!) how I *should* have done this.

Ideally, I was expecting to be able to write a small .py file,
importing and minimally overriding portforward.py, and then be able to
do the mktap/twistd thing. But I couldn't see how.

Can anyone give me a simple example of how I should have done this?
The logging portforwarder example seems like a nice simple starting
point, from which I can go on and learn how to do clever stuff :-)

Thanks,
Paul.

PS Many thanks to the twisted crew - even if I didn't do things the
"right" way, I got my problem diagnosed in no time at all, when I'd
previously struggled with it for days!

[Andrew Bennetts]

On Fri, Jun 27, 2003 at 03:42:20AM -0700, Paul Moore wrote:

I hit a problem yesterday with my mail connection. In a desparate
attempt to understand what was going on, I wanted to log the
connection traffic. After a bit of searching, I found a post on c.l.p
from Andrew Bennetts explaining how to run a port forwarder in 2 lines
using Twisted.

$ mktap portforward -p 8000 -h remote -d 20
$ twistd -f portforward.tap

This looked brilliant - all I needed to do was add logging. A quick
look (I was *very* short of time, so I couldn't spend long) showed me
no obvious way of adding a hook (at least, not one which wouldn't turn
the above 2-liner into something more complex, that I didn't have time
to work out...) So I gave in, and just added a print statement in the
twisted code.

This gave me the results I wanted, but left me wondering (and not
getting far in finding out!) how I *should* have done this.

Ideally, I was expecting to be able to write a small .py file,
importing and minimally overriding portforward.py, and then be able to
do the mktap/twistd thing. But I couldn't see how.
The easiest way is probably to do this:

--- loggingpf.py ---

from twisted.internet import app
from twisted.python import log
from twisted.protocols import portforward
from twisted.tap.portforward import Options

class LoggingProxyClient(portforward.ProxyClient):
def dataReceived(self, data):
log.msg('server sent: ' + repr(data))
portforward.ProxyClient.dataReceived(self, data)

class LoggingProxyClientFactory(portforward.ProxyClientF actory):
protocol = LoggingProxyClient

class LoggingProxyServer(portforward.ProxyServer):
clientProtocolFactory = LoggingProxyClientFactory

def dataReceived(self, data):
log.msg('client sent: ' + repr(data))
portforward.ProxyServer.dataReceived(self, data)

class LoggingProxyFactory(portforward.ProxyFactory):
protocol = LoggingProxyServer

application = app.Application('LoggingPortForward')
application.listenTCP(8000, LoggingProxyFactory('localhost', 80))

--- end --

Then you can use

$ twistd -n -o -y loggingpf.py

You could easily adapt this to become a mktap plugin; see

http://twistedmatrix.com/documents/howto/plugin

(For instance, the normal portforward plugin is defined in
twisted/plugins.tml and twisted/tap/portforward.py)

An even lazier approach would be to add an "application.run(save=0)" to the
end of loggingpf.py -- then you'd be able to run it directly as

$ python loggingpf.py

(but using twistd gives you the ability to daemonise, use log files, etc).
Can anyone give me a simple example of how I should have done this?
The logging portforwarder example seems like a nice simple starting
point, from which I can go on and learn how to do clever stuff :-)

Unfortunately, it's not very simple, as you can see -- although you only
need to override two methods (ProxyClient.dataReceived and
ProxyServer.dataReceived), you need to make four subclasses to tie it all
together.

-Andrew.

[Lee Harr]

In article <ma*********************************@python.org> , Andrew Bennetts:

On Fri, Jun 27, 2003 at 03:42:20AM -0700, Paul Moore wrote:

I hit a problem yesterday with my mail connection. In a desparate
attempt to understand what was going on, I wanted to log the
connection traffic. After a bit of searching, I found a post on c.l.p
from Andrew Bennetts explaining how to run a port forwarder in 2 lines
using Twisted.

$ mktap portforward -p 8000 -h remote -d 20
$ twistd -f portforward.tap

This looked brilliant - all I needed to do was add logging. A quick
look (I was *very* short of time, so I couldn't spend long) showed me
no obvious way of adding a hook (at least, not one which wouldn't turn
the above 2-liner into something more complex, that I didn't have time
to work out...) So I gave in, and just added a print statement in the
twisted code.

This gave me the results I wanted, but left me wondering (and not
getting far in finding out!) how I *should* have done this.

Ideally, I was expecting to be able to write a small .py file,
importing and minimally overriding portforward.py, and then be able to
do the mktap/twistd thing. But I couldn't see how.
The easiest way is probably to do this:

--- loggingpf.py ---

Unfortunately, it's not very simple, as you can see -- although you only
need to override two methods (ProxyClient.dataReceived and
ProxyServer.dataReceived), you need to make four subclasses to tie it all
together.

I was thinking about this too, and I was wondering if it might be good to
add a "verbosity" switch to either mktap or twistd, along the lines of
what netcat or ssh does. (ie, include 2 or 3 -v switches for additional
verbosity). It seems like it would definitely be helpful in certain
situations.

[Markus Wankus]

On 27 Jun 2003 03:42:20 -0700, Paul Moore <pa********@atosorigin.com>
wrote:

I hit a problem yesterday with my mail connection.

Well, if you are unlucky enough to be a Bell Sympatico customer (like me)
on the 26th the blocked all traffic on port 25 without telling anybody. It
was really rather nice of them...

Gee, let's first raise the rates for the same service, then put a bandwidth
cap on it, and now block port 25...and see how many people get pissed.

--
Markus

[Steve Holden]

"Markus Wankus" <ma***********@hotmail.com> wrote in message
news:op**************@news1.on.sympatico.ca...

On 27 Jun 2003 03:42:20 -0700, Paul Moore <pa********@atosorigin.com>
wrote:

I hit a problem yesterday with my mail connection.
Well, if you are unlucky enough to be a Bell Sympatico customer (like me)
on the 26th the blocked all traffic on port 25 without telling anybody.

It was really rather nice of them...

Gee, let's first raise the rates for the same service, then put a bandwidth cap on it, and now block port 25...and see how many people get pissed.

Yeah, Cox recently installed a block on outgoing port 25 connections. Their
justification is "it helps us to reduce the amount of spam coming out of our
domain". At least they will let me use any email address I like as long as I
connect to their SMTP servers from inside their network, but unfotunately
they don't appear to have any way to let me do the same thing when I'm
somewhere else (like a customer location), so they've effectively forced me
to use a dual setup for my mail.

All done without any advance notice, of course. Internet "access" takes on a
whole new meaning in these guys' hands :-)
regards
--
Steve Holden http://www.holdenweb.com/
Python Web Programming http://pydish.holdenweb.com/pwp/