473,396 Members | 2,111 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,396 software developers and data experts.

Multi-threaded FTP Question

I'm trying to use ftp in python in a multi-threaded way on a windows
box - python version 2.4.3. Problem is that it appears that it's only
possible to have five instances/threads at one point in time. Errors
look like:

File "C:\Python24\lib\ftplib.py", line 107, in __init__
self.connect(host)
File "C:\Python24\lib\ftplib.py", line 132, in connect
self.welcome = self.getresp()
File "C:\Python24\lib\ftplib.py", line 208, in getresp
resp = self.getmultiline()
File "C:\Python24\lib\ftplib.py", line 194, in getmultiline
line = self.getline()
File "C:\Python24\lib\ftplib.py", line 184, in getline
if not line: raise EOFError
EOFError

Is it possible to have more than five simultaneous ftp connections?

Thanks.

Derek

Jul 11 '06 #1
5 2557

db******@gmail.com wrote:
I'm trying to use ftp in python in a multi-threaded way on a windows
box - python version 2.4.3. Problem is that it appears that it's only
possible to have five instances/threads at one point in time. Errors
look like:

File "C:\Python24\lib\ftplib.py", line 107, in __init__
self.connect(host)
File "C:\Python24\lib\ftplib.py", line 132, in connect
self.welcome = self.getresp()
File "C:\Python24\lib\ftplib.py", line 208, in getresp
resp = self.getmultiline()
File "C:\Python24\lib\ftplib.py", line 194, in getmultiline
line = self.getline()
File "C:\Python24\lib\ftplib.py", line 184, in getline
if not line: raise EOFError
EOFError

Is it possible to have more than five simultaneous ftp connections?

Thanks.

Derek
I replied to this about 4 hours ago from my gmail email account (not my
google groups account associated with the same email addres), but
haven't seen it show up, so I apologize if this is a dupe.

Would you mind posting your code? Are you trying to pass the same FTP
connection object to all 5 threads?

- Jeremy M. Jones

Jul 11 '06 #2
There are n instances of ftplib.FTP.

Funny thing is that I tried to run the code twice, concurrently, in two
separate IDLE instances. Each instance had four threads/ftp calls.
Again, only five ftp connections were allowed on my computer. The code
errored out on the sixth attempt.

I wonder if there' s some underlying software being called that has a
limit on the number of ftp connections allowed at any point in time.
I'm tempted to try this on cygwin to see if the behavior is different.

Derek

import ftplib, zipfile, datetime, os
from threading import Thread, Lock
class ftpDownload(Thread):
def __init__(self, year, quarter):
Thread.__init__(self)
self.quarter = str(quarter)
self.year = str(year)
self.filename = ""
self.ftp = ftplib.FTP('xxxxxxx')
self.ftp.login()
self.lock = Lock()

def run(self):
filehandle = open(self.filename,'wb')
self.ftp.retrbinary('RETR /xxxxx/xxxxxxx/' + self.year + '/QTR'
+ self.quarter + '/xxxxxxx.zip', filehandle.write)
filehandle.close()
self.ftp.close()

ftplist = []
for year in range(1990,2000):
for quarter in range(1,5):
current = ftpDownload(year, quarter)
ftplist.append(current)
current.start()

for job in ftplist:
job.join()


Jeremy Jones wrote:
db******@gmail.com wrote:
I'm trying to use ftp in python in a multi-threaded way on a windows
box - python version 2.4.3. Problem is that it appears that it's only
possible to have five instances/threads at one point in time. Errors
look like:

File "C:\Python24\lib\ftplib.py", line 107, in __init__
self.connect(host)
File "C:\Python24\lib\ftplib.py", line 132, in connect
self.welcome = self.getresp()
File "C:\Python24\lib\ftplib.py", line 208, in getresp
resp = self.getmultiline()
File "C:\Python24\lib\ftplib.py", line 194, in getmultiline
line = self.getline()
File "C:\Python24\lib\ftplib.py", line 184, in getline
if not line: raise EOFError
EOFError

Is it possible to have more than five simultaneous ftp connections?

Thanks.

Derek

I replied to this about 4 hours ago from my gmail email account (not my
google groups account associated with the same email addres), but
haven't seen it show up, so I apologize if this is a dupe.

Would you mind posting your code? Are you trying to pass the same FTP
connection object to all 5 threads?

- Jeremy M. Jones
Jul 11 '06 #3

Dennis Lee Bieber wrote:
On 11 Jul 2006 06:45:42 -0700, db******@gmail.com declaimed the
following in comp.lang.python:

Could it be that the SERVER is limiting things to 5
concurrent/parallel connections from any single IP?

I know I've encountered sites that only allowed two FTP downloads at
a time...
This is what I was starting to think as well. The only thing that
looked funky with the OP's code was that it looked like he was writing
everything to a filename of "" (unless he's intentionally modified his
code to not show where he's setting that).

- Jeremy M. Jones

Jul 12 '06 #4

db******@gmail.com wrote:
I'm trying to use ftp in python in a multi-threaded way on a windows
box - python version 2.4.3. Problem is that it appears that it's only
possible to have five instances/threads at one point in time. Errors
look like:

File "C:\Python24\lib\ftplib.py", line 107, in __init__
self.connect(host)
File "C:\Python24\lib\ftplib.py", line 132, in connect
self.welcome = self.getresp()
File "C:\Python24\lib\ftplib.py", line 208, in getresp
resp = self.getmultiline()
File "C:\Python24\lib\ftplib.py", line 194, in getmultiline
line = self.getline()
File "C:\Python24\lib\ftplib.py", line 184, in getline
if not line: raise EOFError
EOFError

Is it possible to have more than five simultaneous ftp connections?

Thanks.

Derek
It might be XP SP2's worm protection as well:

http://www.speedguide.net/read_articles.php?id=1497

Jul 12 '06 #5
Thanks so much for your help on this. The server that I'm connecting
to is the culprit. They only allow five connections at a time.

I assumed that it was a code issue. I think that we're conditioned to
expect that the problem is on the software side of things.

-Derek
ol*****@verizon.net wrote:
db******@gmail.com wrote:
I'm trying to use ftp in python in a multi-threaded way on a windows
box - python version 2.4.3. Problem is that it appears that it's only
possible to have five instances/threads at one point in time. Errors
look like:

File "C:\Python24\lib\ftplib.py", line 107, in __init__
self.connect(host)
File "C:\Python24\lib\ftplib.py", line 132, in connect
self.welcome = self.getresp()
File "C:\Python24\lib\ftplib.py", line 208, in getresp
resp = self.getmultiline()
File "C:\Python24\lib\ftplib.py", line 194, in getmultiline
line = self.getline()
File "C:\Python24\lib\ftplib.py", line 184, in getline
if not line: raise EOFError
EOFError

Is it possible to have more than five simultaneous ftp connections?

Thanks.

Derek

It might be XP SP2's worm protection as well:

http://www.speedguide.net/read_articles.php?id=1497
Jul 12 '06 #6

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

37
by: ajikoe | last post by:
Hello, Is anyone has experiance in running python code to run multi thread parallel in multi processor. Is it possible ? Can python manage which cpu shoud do every thread? Sincerely Yours,...
4
by: Frank Jona | last post by:
Intellisense with C# and a multi-file assembly is not working. With VB.NET it is working. Is there a fix availible? We're using VisualStudio 2003 Regards Frank
12
by: * ProteanThread * | last post by:
but depends upon the clique: ...
0
by: frankenberry | last post by:
I have multi-page tiff files. I need to extract individual frames from the multi-page tiffs and save them as single-page tiffs. 95% of the time I receive multi-page tiffs containing 1 or more black...
6
by: cody | last post by:
What are multi file assemblies good for? What are the advantages of using multiple assemblies (A.DLL+B.DLL) vs. a single multi file assembly (A.DLL+A.NETMODULE)?
6
by: Joe | last post by:
I have 2 multi-list boxes, 1 displays course categories based on a table called CATEGORIES. This table has 2 fields CATEGORY_ID, CATEGORY_NAME The other multi-list box displays courses based on...
4
by: mimmo | last post by:
Hi! I should convert the accented letters of a string in the correspondent letters not accented. But when I compile with -Wall it give me: warning: multi-character character constant Do the...
5
by: Shane Story | last post by:
I can seem to get the dimensions of a frame in a multiframe tiff. After selecting activeframe, the Width/Height is still really much larger than the page's actual dimensions. When I split a...
0
by: Sabri.Pllana | last post by:
We apologize if you receive multiple copies of this call for papers. *********************************************************************** 2008 International Workshop on Multi-Core Computing...
1
by: mknoll217 | last post by:
I am recieving this error from my code: The multi-part identifier "PAR.UniqueID" could not be bound. The multi-part identifier "Salary.UniqueID" could not be bound. The multi-part identifier...
0
by: ryjfgjl | last post by:
In our work, we often receive Excel tables with data in the same format. If we want to analyze these data, it can be difficult to analyze them because the data is spread across multiple Excel files...
0
by: emmanuelkatto | last post by:
Hi All, I am Emmanuel katto from Uganda. I want to ask what challenges you've faced while migrating a website to cloud. Please let me know. Thanks! Emmanuel
1
by: nemocccc | last post by:
hello, everyone, I want to develop a software for my android phone for daily needs, any suggestions?
1
by: Sonnysonu | last post by:
This is the data of csv file 1 2 3 1 2 3 1 2 3 1 2 3 2 3 2 3 3 the lengths should be different i have to store the data by column-wise with in the specific length. suppose the i have to...
0
by: Hystou | last post by:
There are some requirements for setting up RAID: 1. The motherboard and BIOS support RAID configuration. 2. The motherboard has 2 or more available SATA protocol SSD/HDD slots (including MSATA, M.2...
0
marktang
by: marktang | last post by:
ONU (Optical Network Unit) is one of the key components for providing high-speed Internet services. Its primary function is to act as an endpoint device located at the user's premises. However,...
0
by: Hystou | last post by:
Overview: Windows 11 and 10 have less user interface control over operating system update behaviour than previous versions of Windows. In Windows 11 and 10, there is no way to turn off the Windows...
0
tracyyun
by: tracyyun | last post by:
Dear forum friends, With the development of smart home technology, a variety of wireless communication protocols have appeared on the market, such as Zigbee, Z-Wave, Wi-Fi, Bluetooth, etc. Each...
0
agi2029
by: agi2029 | last post by:
Let's talk about the concept of autonomous AI software engineers and no-code agents. These AIs are designed to manage the entire lifecycle of a software development project—planning, coding, testing,...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.