Problem using FTP in a unix shell script

[niteck07]

I am using following shell script to ftp files to another server
but this is failing as the shell script changes the user name for the ftp login
the correct user name is 'ag\invprint' which the script is fetching from a file using grep command but the ftp log says the script is trying to logon as user 'aginvprint'

Looks like the script is removing the character '\' when trying to ftp

Can someone please help and let me know how to fix this issue ?

Script is as follows .............
-------------------------------------------------------------------------------------------------------------------
cd /home/DWETL
server=`grep "AG_PS" $XXTNG_TOP/etc/ftp.dat | awk '{print $2}'`
FTP_USERID=`gre p "AG_PS" $XXTNG_TOP/etc/ftp.dat | awk '{print $3}'`
PASSWD=`grep "AG_PS" $XXTNG_TOP/etc/ftp.dat | awk '{print $4}'`
PS_DESTDIR=`gre p "AG_PS" $XXTNG_TOP/etc/ftp.dat | awk '{print $5}'`
dest_path="cd $PS_DESTDIR"
echo "User name .. " $FTP_USERID > ~/AG_Priory_Park. log
echo "Password .. " $PASSWD > ~/AG_Priory_Park. log
`ftp -i -n -v $server <<... >~/AG_Priory_Park. ftp
user $FTP_USERID $PASSWD
ascii
$dest_path
put copy.sch
quit
...`

-------------------------------------------------------------------------------------------------------------------

Output of AG_Priory_Park. ftp is as follows
-------------------------------------------------------------------

Connected to 10.160.200.22.
220 Microsoft FTP Service
331 Password required for 'aginvprint'.
530 User 'aginvprint' cannot log in.
Login failed.
530 Please login with USER and PASS.
530 Please login with USER and PASS.
200 PORT command successful.
530 Please login with USER and PASS.
221


Cheers
Nitin

[coaxfiber]

I am using following shell script to ftp files to another server
but this is failing as the shell script changes the user name for the ftp login
the correct user name is 'ag\invprint' which the script is fetching from a file using grep command but the ftp log says the script is trying to logon as user 'aginvprint'

Looks like the script is removing the character '\' when trying to ftp

Can someone please help and let me know how to fix this issue ?

Script is as follows .............
-------------------------------------------------------------------------------------------------------------------
cd /home/DWETL
server=`grep "AG_PS" $XXTNG_TOP/etc/ftp.dat | awk '{print $2}'`
FTP_USERID=`gre p "AG_PS" $XXTNG_TOP/etc/ftp.dat | awk '{print $3}'`
PASSWD=`grep "AG_PS" $XXTNG_TOP/etc/ftp.dat | awk '{print $4}'`
PS_DESTDIR=`gre p "AG_PS" $XXTNG_TOP/etc/ftp.dat | awk '{print $5}'`
dest_path="cd $PS_DESTDIR"
echo "User name .. " $FTP_USERID > ~/AG_Priory_Park. log
echo "Password .. " $PASSWD > ~/AG_Priory_Park. log
`ftp -i -n -v $server <<... >~/AG_Priory_Park. ftp
user $FTP_USERID $PASSWD
ascii
$dest_path
put copy.sch
quit
...`

-------------------------------------------------------------------------------------------------------------------

Output of AG_Priory_Park. ftp is as follows
-------------------------------------------------------------------

Connected to 10.160.200.22.
220 Microsoft FTP Service
331 Password required for 'aginvprint'.
530 User 'aginvprint' cannot log in.
Login failed.
530 Please login with USER and PASS.
530 Please login with USER and PASS.
200 PORT command successful.
530 Please login with USER and PASS.
221


Cheers
Nitin

hi,


Can I ask you something? I'm not yet familiar to your script. But would you tell me how to do some basics of FTP.

how could i command a "ls -al" to another server and get that log to my current server.

my user name is: coax
my password is: jazz
command: ls -al


- I want to do this continuously w/o being ask with my password. Of course I could enter my password "jazz" with my script instead of manually entering FTP
and then ask for my password.
- I've already asked this question before but they advised me to use 'SSH with no password' by didn't succeeded to that method because our administrator doesn't allow that one.

Thank You Very Much
-coaxfiber

[prn]

It looks like you're trying to ftp to a windows box using active directory authentication. It also looks like you're extracting the destination information from a file in /etc (odd in itself).

The traditional (more than traditional, actually specified in the standards) is that autologin data goes not in /etc/ftp.dat, but in $HOME/.netrc. Check the ftp and netrc man pages for information.

Another (good) possibility is to use ncftp instead of the standard ftp. The open source ncftpput and ncftpget were developed specifically to make ftp scripting work well. Check out http://www.ncftp.com/ for documentation and downloads.

I have used both .netrc and ncftp with success. Much may depend on what all you want to do and what kinds of restrictions you may face.

As far as backslashes go, you will need a pair of them in order to hand one off to the process. In general, a backslash means "take the next character literally" to the unix shell. In your case, the "next character" is "i". You want it to be a backslash, so you need to use a backslash to tell the shell that you actually want a backslash there.

HTH,
Paul

[prn]

- I've already asked this question before but they advised me to use 'SSH with no password' by didn't succeeded to that method because our administrator doesn't allow that one.

Another alternative is to use RSA authentication instead of a password. Look up the SSH docs and set up your account to accept your RSA key (which you keep on the box you're trying to connect from. See, e.g., http://sial.org/howto/openssh/publickey-auth/ for how to do it.

RSA authentication is enabled by default in a typical SSH installation. You are evidently not the sysadmin on the box you are connecting to, but it seems unlikely that your system admin would object to RSA public key authentication in principle. (Of course, s/he may have a problem with whatever it is you are intending to do, but that is a different matter and I can't comment on that.)

Paul

[coaxfiber]

Another alternative is to use RSA authentication instead of a password. Look up the SSH docs and set up your account to accept your RSA key (which you keep on the box you're trying to connect from. See, e.g., http://sial.org/howto/openssh/publickey-auth/ for how to do it.

RSA authentication is enabled by default in a typical SSH installation. You are evidently not the sysadmin on the box you are connecting to, but it seems unlikely that your system admin would object to RSA public key authentication in principle. (Of course, s/he may have a problem with whatever it is you are intending to do, but that is a different matter and I can't comment on that.)

Paul

Hi,

I'm currently login in unix not in Windows.
I tried to use RSA/DSA , I followed the procedures on how to do it, but still not working.

I have read some articles on the net and found this useful.:



#!/bin/sh
HOST='ftp.users .qwest.net'
USER='yourid'
PASSWD='yourpw'


ftp -n $HOST <<END_SCRIPT
quote USER $USER
quote PASS $PASSWD
ls -al > FILE
get FILE
quit
END_SCRIPT
exit 0

this works good. but my problem now is that limited command runs on ftp. not all unix command works on ftp. FYI.

So I'm still thinking that I can to this using TELNET.

quote USER $USER
quote PASS $PASSWD

--->something like this.

Thanks Very Much.

[coaxfiber]

Another alternative is to use RSA authentication instead of a password. Look up the SSH docs and set up your account to accept your RSA key (which you keep on the box you're trying to connect from. See, e.g., http://sial.org/howto/openssh/publickey-auth/ for how to do it.

RSA authentication is enabled by default in a typical SSH installation. You are evidently not the sysadmin on the box you are connecting to, but it seems unlikely that your system admin would object to RSA public key authentication in principle. (Of course, s/he may have a problem with whatever it is you are intending to do, but that is a different matter and I can't comment on that.)

Paul

Hi, this is what i got..
------------------------------------------------------------
Host key not found from database.
Key fingerprint:
ximen-cohel-ganup-zacet-soseh-sycab-hekig-huzud-raron-hovet-raxox
You can get a public key's fingerprint by running
% ssh-keygen -F publickey.pub
on the keyfile.
Host key saved to /home/coax/.ssh2/hostkeys/key_22_10.120.1 38.11.pub
host key for 10.120.138.11, accepted by coax Wed Aug 08 2007 02:10:39 +0800

------------------------------------------------------------

i dunno why is it .ssh2 ? what's the difference between .ssh and .ssh2?

Tnx..

[prn]

I'm currently login in unix not in Windows.

The link I posted http://sial.org/howto/openssh/publickey-auth/ is really much more oriented to unix/linux than windows, so that should be fine

I tried to use RSA/DSA , I followed the procedures on how to do it, but still not working.

I have tested those procedures myself and they worked fine for me. We'll need more details.

this works good. but my problem now is that limited command runs on ftp. not all unix command works on ftp. FYI.

That's right. ftp is NOT INTENDED to be a login shell. It's intended for file transfer, not for general use.

So I'm still thinking that I can to this using TELNET.

Personally, I always discourage using TELNET for almost anything. SSH is much better. Security is important and telnet's security is lousy. SSH is far preferable. One of the first things I always do when I set up a unix or linux box is to disable telnet and ftp. (Use sftp or scp instead.)

From your next post:

Hi, this is what i got..

What you got *how*?

You can get a public key's fingerprint by running
% ssh-keygen -F publickey.pub
on the keyfile.

That's not right at all. Check the man page for ssh-keygen. The -F switch is to

Search for the specified hostname in a known_hosts file, listing any occurrences found. This option is useful to find hashed host names or addresses and may also be used in conjunction with the -H option to print found keys in a hashed format.

The -l (lowercase L) switch is for showing a fingerprint. For example:

$ ssh-keygen -l
Enter file in which the key is (/home/prn/.ssh/id_rsa):
2048 76:2a:13:20:98:36:5f:5f:ea:9a:46:36:13:22:af:b4 /home/prn/.ssh/id_rsa.pub

i dunno why is it .ssh2 ? what's the difference between .ssh and .ssh2?

Interesting question, but even more relevant is the question of what you did to generate this output.

I'd suggest wiping out everything in .ssh2 and probably most of what's in the .ssh directory of both your login machine and the one you want to connect to. Then start over following the directions for how to generate the keys. Note that the reference page says "Do not use your account password, nor an empty passphrase", but if I understand correctly that you want to run the script when you are not logged in, you would not be able to use the ssh-agent solution and you would need to generate the key with no passphrase, i.e., an empty passphrase, so just hit enter. Otherwise, follow the instructions right through the part about "key distribution". You should then be able to ssh from your account on the "current" computer to your account on the "other" server without being asked for a password.

Once you can do that, you can run any command or script you want on the "other" server. In particular, you can run commands that are not part of the limited repertoire of FTP.

Of course, what you actually said you were going to do was run "ls -al" "continuous ly". I hope you mean that you are going to run it "at intervals" rather than all the time. I presume this means you are polling for some condition on the remote machine and then you plan to do something with the result, but what you plan to do with the result is completely unclear and I do have to wonder if I am just giving you the ammunition to shoot yourself in the foot. Please do try not to run anything "continuous ly". Your sysadmin will not thank me for enabling you if you do. Please be considerate of your sysadmin and of other users and not tie up resources on the remote host unnecessarily. If you have to poll the other machine, do it at reasonable intervals and then do whatever you must.

Also, instead of running a script on your local machine that polls the other machine, perhaps it would be a better idea to run a script on the remote machine that polls for the relevant condition and then either does what must be done or notifies your local machine. Have you thought of that? You can set up a cron job on the remote host and then have it do what must be done there.

If both your "local" and "remote" hosts need to be involved, then perhaps the remote host should poll for the file and then scp it to your local box. There are always many ways to organize such tasks. The more I think about what little you have told us so far, the less confident I am that I understand what you are doing.

Paul

[coaxfiber]

The link I posted http://sial.org/howto/openssh/publickey-auth/ is really much more oriented to unix/linux than windows, so that should be fine


I have tested those procedures myself and they worked fine for me. We'll need more details.


That's right. ftp is NOT INTENDED to be a login shell. It's intended for file transfer, not for general use.


Personally, I always discourage using TELNET for almost anything. SSH is much better. Security is important and telnet's security is lousy. SSH is far preferable. One of the first things I always do when I set up a unix or linux box is to disable telnet and ftp. (Use sftp or scp instead.)

From your next post:

What you got *how*?


That's not right at all. Check the man page for ssh-keygen. The -F switch is to


The -l (lowercase L) switch is for showing a fingerprint. For example:

$ ssh-keygen -l
Enter file in which the key is (/home/prn/.ssh/id_rsa):
2048 76:2a:13:20:98:36:5f:5f:ea:9a:46:36:13:22:af:b4 /home/prn/.ssh/id_rsa.pub

Interesting question, but even more relevant is the question of what you did to generate this output.

I'd suggest wiping out everything in .ssh2 and probably most of what's in the .ssh directory of both your login machine and the one you want to connect to. Then start over following the directions for how to generate the keys. Note that the reference page says "Do not use your account password, nor an empty passphrase", but if I understand correctly that you want to run the script when you are not logged in, you would not be able to use the ssh-agent solution and you would need to generate the key with no passphrase, i.e., an empty passphrase, so just hit enter. Otherwise, follow the instructions right through the part about "key distribution". You should then be able to ssh from your account on the "current" computer to your account on the "other" server without being asked for a password.

Once you can do that, you can run any command or script you want on the "other" server. In particular, you can run commands that are not part of the limited repertoire of FTP.

Of course, what you actually said you were going to do was run "ls -al" "continuous ly". I hope you mean that you are going to run it "at intervals" rather than all the time. I presume this means you are polling for some condition on the remote machine and then you plan to do something with the result, but what you plan to do with the result is completely unclear and I do have to wonder if I am just giving you the ammunition to shoot yourself in the foot. Please do try not to run anything "continuous ly". Your sysadmin will not thank me for enabling you if you do. Please be considerate of your sysadmin and of other users and not tie up resources on the remote host unnecessarily. If you have to poll the other machine, do it at reasonable intervals and then do whatever you must.

Also, instead of running a script on your local machine that polls the other machine, perhaps it would be a better idea to run a script on the remote machine that polls for the relevant condition and then either does what must be done or notifies your local machine. Have you thought of that? You can set up a cron job on the remote host and then have it do what must be done there.

If both your "local" and "remote" hosts need to be involved, then perhaps the remote host should poll for the file and then scp it to your local box. There are always many ways to organize such tasks. The more I think about what little you have told us so far, the less confident I am that I understand what you are doing.

Paul

Hi,

I'm not in the office right now. But remembering what i was doing in my terminal last night and following the instructions given from the page you had posted. I think I removed the option "-f" because it gives an error message.

I just need to access the other servers just to interrogate some elements to have information and fetch in to my current server. Not exactly the 'ls -al' command.

[prn]

Hi coax,

I'm not in the office right now. But remembering what i was doing in my terminal last night and following the instructions given from the page you had posted. I think I removed the option "-f" because it gives an error message.

Be careful to note that "-f" is NOT the same as "-F" As switches to ssh-keygen, -F searches for a hostname and -f specifies a filename (when used WITH -l or various other switches).

I just need to access the other servers just to interrogate some elements to have information and fetch in to my current server. Not exactly the 'ls -al' command.

OK. That's good. It sounds even more to me like you ought to run a script on the remote and have it send something to your local host if and when if finds the trigger conditions.

And BTW, here's an even better reference for how to set up SSH to do that sort of thing unattended: http://www.ibm.com/developerworks/library/l-keyc.html

Using the "keychain" utility that part 2 of this article series describes, you should be able to run an unattended cron job that is much more secure than with an empty passphrase. The keychain page in the links says that "Current versions of keychain are known to run on Linux, BSD, Cygwin, Tru64 UNIX, HP-UX, Mac OS X, and Solaris". I don't know what variety of Unix/Linux you are using, but this seems fairly promising.

HTH,
Paul

[eileen365]

Another alternative is to use RSA authentication instead of a password. Look up the SSH docs and set up your account to accept your RSA key (which you keep on the box you're trying to connect from. See, e.g., http://sial.org/howto/openssh/publickey-auth/ for how to do it.

RSA authentication is enabled by default in a typical SSH installation. You are evidently not the sysadmin on the box you are connecting to, but it seems unlikely that your system admin would object to RSA public key authentication in principle. (Of course, s/he may have a problem with whatever it is you are intending to do, but that is a different matter and I can't comment on that.)

Paul

This is exactly the problem I'm having! I'm ftp'ing from an hp Tru64 machine to a windows machine and normally I don't have a problem. With this particular windows machine I have to use domain name\login id and the '\' is throwing me off. I'm building a .netrc file and I've tried double \, single quotes aroung the \...seems like I've tried everything. Here is a portion of my script. First I define the variables, then I build the .netrc:

USERID="domain\ loginid"
PASSWORD="xxxxx x"
and I build the .netrc below:

# Create a .netrc
echo "machine $MACHINE" >> ~/.netrc
echo "login $USERID" >> ~/.netrc
echo "password $PASSWORD" >> ~/.netrc
echo "macdef init" >> ~/.netrc
echo "prompt off" >> ~/.netrc
echo "bin" >> ~/.netrc
echo "lcd $WRKDIR" >> ~/.netrc
echo "cd $TGTDIR" >> ~/.netrc
echo "mput *.*" >> ~/.netrc
echo "quit" >> ~/.netrc
echo "" >> ~/.netrc

# Change the permissions of the .netrc
/sbin/chmod 600 ~/.netrc

When I run as is, the script tries to login as
domainloginid.

If I put quotes around the \ I get:
domain" " loginid

If I try single quotes I get a mismatched quote error.

If i use \\, both get ignored and the script tries to login as
domainloginid

I've used \ in front of special characters when specifying the password but I didn't expect this kind of problem. Is there a combination I'm missing.