By using this site, you agree to our updated Privacy Policy and our Terms of Use. Manage your Cookies Settings.
440,441 Members | 1,831 Online
Bytes IT Community
+ Ask a Question
Need help? Post your question and get tips & solutions from a community of 440,441 IT Pros & Developers. It's quick & easy.

Automating FTP file transfers

P: n/a
Hi all,

Firstly :-) , is there any where I can search through archived newsgroup
posts so I am not cluttering up the newsgroup with repeated queries ?

And secondly :-), I know this has probably been discussed before but.

I am wanting to do some scripting to automate a few administration tasks,
one of the first tasks is automating FTP file transfers.

I would like to know if the following could be made more robust using python
and its FTP libraries rather than executing native OS commands in a shell
script.

Basically I need to...
1. Check on the local system for new files in a specific directory
If new files exist then...
2. Connect to a remote FTP server.
3. Transfer local files to a specific directory on the remote FTP server.
4. Then remove/archive local files and end session.

This is critical operation and though while in the past I have written DOS
scripts etc. to do simple tasks I have not needed to check for errors as
they weren't absolutely critical tasks. I am going to need to run this
overnight using AT or CRON then I need to know about file transfer errors
etc. and other problems that f I were actually performing the FTP transfer
manually I would get feedback of the problem and could re-try the operation.

Any help would be very appreciated.

Jul 18 '05 #1
Share this Question
Share on Google+
7 Replies


P: n/a
Limey Drink:
is there any where I can search through archived newsgroup posts
http://groups.google.com
I would like to know if the following could be made more robust using python
and its FTP libraries rather than executing native OS commands in a shell
script.

Basically I need to... [upload a file]


Sure. Use ftplib. See
http://www.python.org/doc/2.3.2/lib/module-ftplib.html

Here's a code snippet from one of my scripts.

import os, ftplib
ftp = ftplib.FTP(Hostname,Username,Password)
ftp.cwd(WorkingDirectory)
ftp.storbinary("STOR " + RemoteZipFile, file(LocalZipFile, "rb"))
ftp.quit()
os.remove(LocalZipFile)

--
René Pijlman
Jul 18 '05 #2

P: n/a
Limey Drink wrote:
Hi all,

Firstly :-) , is there any where I can search through archived newsgroup
posts so I am not cluttering up the newsgroup with repeated queries ?

Yep, info on all the python lists, including archives, are available here:

http://python.org/community/lists.html
And secondly :-), I know this has probably been discussed before but.

I am wanting to do some scripting to automate a few administration tasks,
one of the first tasks is automating FTP file transfers.

I would like to know if the following could be made more robust using python
and its FTP libraries rather than executing native OS commands in a shell
script.

Basically I need to...
1. Check on the local system for new files in a specific directory
If new files exist then...
2. Connect to a remote FTP server.
3. Transfer local files to a specific directory on the remote FTP server.
4. Then remove/archive local files and end session.

Most of this should be fairly straightforward. (1) and (4) should be
covered using the os and os.path modules; for (2) and (3) you need the
ftplib module.

The only problem you are likely to encounter is getting a remote
directory listing. The FTP protocol neglected to formalise the response
to FTP's LIST command and different servers send different results back
:(. If this is a problem I would recommend using one of the ftpparse
Python APIs:

* http://effbot.org/downloads/#ftpparse
* http://c0re.23.nu/c0de/ftpparsemodule/ftpparse.html
This is critical operation and though while in the past I have written DOS
scripts etc. to do simple tasks I have not needed to check for errors as
they weren't absolutely critical tasks. I am going to need to run this
overnight using AT or CRON then I need to know about file transfer errors
etc. and other problems that f I were actually performing the FTP transfer
manually I would get feedback of the problem and could re-try the operation.

The ftplib module reports errors using the Python exception mechanism so
you can make it as robust as you need and take whatever action is
appropriate on errors.

Hope this helps.

Cheers, Matt

--
Matt Goodall, Pollenation Internet Ltd
w: http://www.pollenation.net
e: ma**@pollenation.net

Jul 18 '05 #3

P: n/a
Thanks very much for the help guys I now know how to connect using python
ftp library and I am happy with this but...

Just wondering about the bigger picture with regards to sys admin scripts,
in your opinion, is creating these type of system administration scripts
preferable/easier than using say bash scripts, or DOS .bat scripts ?

Because I don't have alot of experience of sys admin and shell scripting I'm
just wondering if I am applying Python for one of its intended uses doing
these type of scripts.

IMHO i would rather learn and stick with Python than to learn all the
different scripting languages as it is cleaner and more powerful and also if
the script is written correctly it seems could be cross platform.

Just wondering if you think I should be trying to execute the in built OS
programs such as FTP etc. calling them from python or should I be using the
libraries that come with Python such as FTPlib and say if I wanted to send
email should I be using the SMTP lib rather than say executing the mail
program in unix and passing arguments from a Python script.

It would hear interesting to hear your views on this as it would be nice if
I can gain from others experience and not make the same mistakes.

Thanks again

"Limey Drink" <bh******@bhicking.plus.com> wrote in message
news:HQ******************@wards.force9.net...
Hi all,

Firstly :-) , is there any where I can search through archived newsgroup
posts so I am not cluttering up the newsgroup with repeated queries ?

And secondly :-), I know this has probably been discussed before but.

I am wanting to do some scripting to automate a few administration tasks,
one of the first tasks is automating FTP file transfers.

I would like to know if the following could be made more robust using python and its FTP libraries rather than executing native OS commands in a shell
script.

Basically I need to...
1. Check on the local system for new files in a specific directory
If new files exist then...
2. Connect to a remote FTP server.
3. Transfer local files to a specific directory on the remote FTP server.
4. Then remove/archive local files and end session.

This is critical operation and though while in the past I have written DOS
scripts etc. to do simple tasks I have not needed to check for errors as
they weren't absolutely critical tasks. I am going to need to run this
overnight using AT or CRON then I need to know about file transfer errors
etc. and other problems that f I were actually performing the FTP transfer
manually I would get feedback of the problem and could re-try the operation.
Any help would be very appreciated.

Jul 18 '05 #4

P: n/a
In article <ap******************@wards.force9.net>,
Limey Drink <bh******@bhicking.plus.com> wrote:
Thanks very much for the help guys I now know how to connect using python
ftp library and I am happy with this but...

Just wondering about the bigger picture with regards to sys admin scripts,
in your opinion, is creating these type of system administration scripts
preferable/easier than using say bash scripts, or DOS .bat scripts ?

Because I don't have alot of experience of sys admin and shell scripting I'm
just wondering if I am applying Python for one of its intended uses doing
these type of scripts.

IMHO i would rather learn and stick with Python than to learn all the
different scripting languages as it is cleaner and more powerful and also if
the script is written correctly it seems could be cross platform.

Just wondering if you think I should be trying to execute the in built OS
programs such as FTP etc. calling them from python or should I be using the
libraries that come with Python such as FTPlib and say if I wanted to send
email should I be using the SMTP lib rather than say executing the mail
program in unix and passing arguments from a Python script.

It would hear interesting to hear your views on this as it would be nice if
I can gain from others experience and not make the same mistakes.

Jul 18 '05 #5

P: n/a
Limey Drink:
Just wondering about the bigger picture with regards to sys admin scripts,
in your opinion, is creating these type of system administration scripts
preferable/easier than using say bash scripts, or DOS .bat scripts ?
Yes! You get real data structures, classes, exception handling, powerful
libraries and such.

For example, you can write system administration scripts in Python that
use a relational database, parse XML config files, send e-mail,
up/download with FTP ...
Just wondering if you think I should be trying to execute the in built OS
programs such as FTP etc. calling them from python
No. Why would you want to do that? Only as a last resort.
or should I be using the libraries that come with Python such as FTPlib and
say if I wanted to send email should I be using the SMTP lib


Yes. Otherwise you'll loose the exception handling, classes, data
structures etc. :-)

--
René Pijlman
Jul 18 '05 #6

P: n/a
OK then, case closed, Python it is :-)

Its just nice to have your gut feeling backed up by other peoples
experience.

Thanks for all the help guys.
"Rene Pijlman" <re********************@my.address.is.invalid> wrote in
message news:m4********************************@4ax.com...
Limey Drink:
Just wondering about the bigger picture with regards to sys admin scripts,in your opinion, is creating these type of system administration scripts
preferable/easier than using say bash scripts, or DOS .bat scripts ?


Yes! You get real data structures, classes, exception handling, powerful
libraries and such.

For example, you can write system administration scripts in Python that
use a relational database, parse XML config files, send e-mail,
up/download with FTP ...
Just wondering if you think I should be trying to execute the in built OS
programs such as FTP etc. calling them from python


No. Why would you want to do that? Only as a last resort.
or should I be using the libraries that come with Python such as FTPlib andsay if I wanted to send email should I be using the SMTP lib


Yes. Otherwise you'll loose the exception handling, classes, data
structures etc. :-)

--
René Pijlman

Jul 18 '05 #7

P: n/a
Limey Drink wrote:
Just wondering about the bigger picture with regards to sys admin scripts,
in your opinion, is creating these type of system administration scripts
preferable/easier than using say bash scripts, or DOS .bat scripts ?

Just wondering if you think I should be trying to execute the in built OS
programs such as FTP etc. calling them from python or should I be using the
libraries that come with Python such as FTPlib and say if I wanted to send
email should I be using the SMTP lib rather than say executing the mail
program in unix and passing arguments from a Python script.


Just to give you my $.02 on this. I have literally dozens of Python ftp
scripts that run on a Linux server at my company daily via cron. All of
my scripts use ftplib and smtplib. My basic script structure is:

1). Check local directory to see if files are available for sending.
2). Encrypt local files via PGP.
3). FTP files to external vendor using ftplib.
4). Send "success" email to myself and other interested parties via
smtplib.
5). If the script error's out instead, send error email to me so I know
there is a problem.

IMO, Python is an excellent choice for this kind of stuff.

Hope this helps!

Jamey

Jul 18 '05 #8

This discussion thread is closed

Replies have been disabled for this discussion.