convert ftp.retrbinary to file object? - Python language lacks expression?

[Robert]

I just tried to convert a (hugh size) ftp.retrbinary run into a
pseudo-file object with .read(bytes) method in order to not consume
500MB on a copy operation.

First I thought, its easy as usual with python using something like
'yield' or so.

Yet I didn't manage to do (without using threads or rewriting
'retrbinary')? Any ideas?

#### I tried a pattern like:
....
def open(self,ftppath,mode='rb'):
class FTPFile: #TODO
...
def iter_retr()
...
def callback(blk):
how-to-yield-from-here-to-iter_retr blk???
ftp.retrbinary("RETR %s" % relpath,callback)
def read(self, bytes=-1):
...
self.buf+=self.iter.next()
...
....

[Martin Franklin]

Robert wrote:

I just tried to convert a (hugh size) ftp.retrbinary run into a
pseudo-file object with .read(bytes) method in order to not consume
500MB on a copy operation.

First I thought, its easy as usual with python using something like
'yield' or so.

Yet I didn't manage to do (without using threads or rewriting
'retrbinary')? Any ideas?

#### I tried a pattern like:
....
def open(self,ftppath,mode='rb'):
class FTPFile: #TODO
...
def iter_retr()
...
def callback(blk):
how-to-yield-from-here-to-iter_retr blk???
ftp.retrbinary("RETR %s" % relpath,callback)
def read(self, bytes=-1):
...
self.buf+=self.iter.next()
...
....

Hmmmm this is nearly there I think...:

import ftplib

class TransferAbort(Exception): pass

class FTPFile:
def __init__(self, server, filename):
self.server = server
self.filename = filename
self.offset = 0

def callback(self, data):
self.offset = self.offset + len(data)
self.data = data
## now quit the RETR command?
raise TransferAbort("stop right now")

def read(self, amount):
self.ftp = ftplib.FTP(self.server)
self.ftp.login()
try:
self.ftp.retrbinary("RETR %s" %self.filename, self.callback,
blocksize=amount,
rest=self.offset)
except TransferAbort:
return self.data
f = FTPFile("HOSTNAME", "FILENAME")

print f.read(24)
print f.read(24)
I open the ftp connection inside the read method as it caused an error
(on the second call to read) when I opened it in __init__ ???

HTH
Martin