By using this site, you agree to our updated Privacy Policy and our Terms of Use. Manage your Cookies Settings.
455,441 Members | 1,741 Online
Bytes IT Community
+ Ask a Question
Need help? Post your question and get tips & solutions from a community of 455,441 IT Pros & Developers. It's quick & easy.

Multi-threaded FTP Question

P: n/a
I'm trying to use ftp in python in a multi-threaded way on a windows
box - python version 2.4.3. Problem is that it appears that it's only
possible to have five instances/threads at one point in time. Errors
look like:

File "C:\Python24\lib\ftplib.py", line 107, in __init__
self.connect(host)
File "C:\Python24\lib\ftplib.py", line 132, in connect
self.welcome = self.getresp()
File "C:\Python24\lib\ftplib.py", line 208, in getresp
resp = self.getmultiline()
File "C:\Python24\lib\ftplib.py", line 194, in getmultiline
line = self.getline()
File "C:\Python24\lib\ftplib.py", line 184, in getline
if not line: raise EOFError
EOFError

Is it possible to have more than five simultaneous ftp connections?

Thanks.

Derek

Jul 11 '06 #1
Share this Question
Share on Google+
5 Replies


P: n/a

db******@gmail.com wrote:
I'm trying to use ftp in python in a multi-threaded way on a windows
box - python version 2.4.3. Problem is that it appears that it's only
possible to have five instances/threads at one point in time. Errors
look like:

File "C:\Python24\lib\ftplib.py", line 107, in __init__
self.connect(host)
File "C:\Python24\lib\ftplib.py", line 132, in connect
self.welcome = self.getresp()
File "C:\Python24\lib\ftplib.py", line 208, in getresp
resp = self.getmultiline()
File "C:\Python24\lib\ftplib.py", line 194, in getmultiline
line = self.getline()
File "C:\Python24\lib\ftplib.py", line 184, in getline
if not line: raise EOFError
EOFError

Is it possible to have more than five simultaneous ftp connections?

Thanks.

Derek
I replied to this about 4 hours ago from my gmail email account (not my
google groups account associated with the same email addres), but
haven't seen it show up, so I apologize if this is a dupe.

Would you mind posting your code? Are you trying to pass the same FTP
connection object to all 5 threads?

- Jeremy M. Jones

Jul 11 '06 #2

P: n/a
There are n instances of ftplib.FTP.

Funny thing is that I tried to run the code twice, concurrently, in two
separate IDLE instances. Each instance had four threads/ftp calls.
Again, only five ftp connections were allowed on my computer. The code
errored out on the sixth attempt.

I wonder if there' s some underlying software being called that has a
limit on the number of ftp connections allowed at any point in time.
I'm tempted to try this on cygwin to see if the behavior is different.

Derek

import ftplib, zipfile, datetime, os
from threading import Thread, Lock
class ftpDownload(Thread):
def __init__(self, year, quarter):
Thread.__init__(self)
self.quarter = str(quarter)
self.year = str(year)
self.filename = ""
self.ftp = ftplib.FTP('xxxxxxx')
self.ftp.login()
self.lock = Lock()

def run(self):
filehandle = open(self.filename,'wb')
self.ftp.retrbinary('RETR /xxxxx/xxxxxxx/' + self.year + '/QTR'
+ self.quarter + '/xxxxxxx.zip', filehandle.write)
filehandle.close()
self.ftp.close()

ftplist = []
for year in range(1990,2000):
for quarter in range(1,5):
current = ftpDownload(year, quarter)
ftplist.append(current)
current.start()

for job in ftplist:
job.join()


Jeremy Jones wrote:
db******@gmail.com wrote:
I'm trying to use ftp in python in a multi-threaded way on a windows
box - python version 2.4.3. Problem is that it appears that it's only
possible to have five instances/threads at one point in time. Errors
look like:

File "C:\Python24\lib\ftplib.py", line 107, in __init__
self.connect(host)
File "C:\Python24\lib\ftplib.py", line 132, in connect
self.welcome = self.getresp()
File "C:\Python24\lib\ftplib.py", line 208, in getresp
resp = self.getmultiline()
File "C:\Python24\lib\ftplib.py", line 194, in getmultiline
line = self.getline()
File "C:\Python24\lib\ftplib.py", line 184, in getline
if not line: raise EOFError
EOFError

Is it possible to have more than five simultaneous ftp connections?

Thanks.

Derek

I replied to this about 4 hours ago from my gmail email account (not my
google groups account associated with the same email addres), but
haven't seen it show up, so I apologize if this is a dupe.

Would you mind posting your code? Are you trying to pass the same FTP
connection object to all 5 threads?

- Jeremy M. Jones
Jul 11 '06 #3

P: n/a

Dennis Lee Bieber wrote:
On 11 Jul 2006 06:45:42 -0700, db******@gmail.com declaimed the
following in comp.lang.python:

Could it be that the SERVER is limiting things to 5
concurrent/parallel connections from any single IP?

I know I've encountered sites that only allowed two FTP downloads at
a time...
This is what I was starting to think as well. The only thing that
looked funky with the OP's code was that it looked like he was writing
everything to a filename of "" (unless he's intentionally modified his
code to not show where he's setting that).

- Jeremy M. Jones

Jul 12 '06 #4

P: n/a

db******@gmail.com wrote:
I'm trying to use ftp in python in a multi-threaded way on a windows
box - python version 2.4.3. Problem is that it appears that it's only
possible to have five instances/threads at one point in time. Errors
look like:

File "C:\Python24\lib\ftplib.py", line 107, in __init__
self.connect(host)
File "C:\Python24\lib\ftplib.py", line 132, in connect
self.welcome = self.getresp()
File "C:\Python24\lib\ftplib.py", line 208, in getresp
resp = self.getmultiline()
File "C:\Python24\lib\ftplib.py", line 194, in getmultiline
line = self.getline()
File "C:\Python24\lib\ftplib.py", line 184, in getline
if not line: raise EOFError
EOFError

Is it possible to have more than five simultaneous ftp connections?

Thanks.

Derek
It might be XP SP2's worm protection as well:

http://www.speedguide.net/read_articles.php?id=1497

Jul 12 '06 #5

P: n/a
Thanks so much for your help on this. The server that I'm connecting
to is the culprit. They only allow five connections at a time.

I assumed that it was a code issue. I think that we're conditioned to
expect that the problem is on the software side of things.

-Derek
ol*****@verizon.net wrote:
db******@gmail.com wrote:
I'm trying to use ftp in python in a multi-threaded way on a windows
box - python version 2.4.3. Problem is that it appears that it's only
possible to have five instances/threads at one point in time. Errors
look like:

File "C:\Python24\lib\ftplib.py", line 107, in __init__
self.connect(host)
File "C:\Python24\lib\ftplib.py", line 132, in connect
self.welcome = self.getresp()
File "C:\Python24\lib\ftplib.py", line 208, in getresp
resp = self.getmultiline()
File "C:\Python24\lib\ftplib.py", line 194, in getmultiline
line = self.getline()
File "C:\Python24\lib\ftplib.py", line 184, in getline
if not line: raise EOFError
EOFError

Is it possible to have more than five simultaneous ftp connections?

Thanks.

Derek

It might be XP SP2's worm protection as well:

http://www.speedguide.net/read_articles.php?id=1497
Jul 12 '06 #6

This discussion thread is closed

Replies have been disabled for this discussion.