Essentials for Using Linux FTP
Brought to you by Andrea Cordingly and the team at reallylinux.com

This guide shares the basics of FTP and a number of unique ways to automate FTP updating and maintaining of files on a server. The tips include applying basic shell scripting and using text editors.

In a recent article on WebServers, I provided insights how to password protect your Website directories using the Apache overrides and .htaccess files. You may want to review the article exclusively on reallylinux.com.

Now, this article focuses exclusively on transferring files to your web server or between a Linux server and Windows PCs. My hope is that anyone who needs to use FTP on a regular basis will gain some insights to make their life easier, and enjoy the power of a Linux server.

In the following steps, I share those essential tips needed for using FTP and creating special command files to automate FTP tasks, to ensure you get everything done successfully.

FTP In a Web Browser
It's very important to note that in most cases and to do almost all basic FTP file handling you can simply login to your server through the interface of a web browser. For example, I can use either Windows or Linux and FTP to my example site. Once login is complete I can simply use my browser to navigate and then use drag and drop to copy files.


Simple drag-and-drop copying of files from Windows to Linux FTP Server.
Note the use of this command in a web browser: ftp://website.com (replace website with your site name)

So, for the most basic copying of files from a local computer to a host/server using FTP, I still think using the browser or one of the many GUI programs is great!

But, what about in an emergency, or when certain commands such as chmod (change the permissions of a directory or file) need to be performed? I hope to give you ideas not only how powerful the command line is in Linux, but also to encourage you that through the command line you can automate FTP to a really fantastic level! Also remember we have a number of other commands articles on our reallylinux.com Help Page.


Getting to Know FTP
Most people have used FTP from the command line at some point in their lives. For those that have not I include this simple summary.

To use FTP (file transfer protocol) you must understand that it is an independent tool to login to a remote server and copy files. Therefore every FTP sessions requires at least login information as well as the destination server name, also called hostname. In most cases your website or server on the Internet will have a name such as mywebsite.com or an IP address such as 212.212.212.1. You can access the server using FTP using either of these identifiers.

Once you start an FTP session you need to know that you are no longer in the command line or shell but rather within the FTP tool itself. It's easy to tell since your prompt will change to FTP>_ and you will be able to type FTP related commands.
See all of the FTP related commands you can use by typing ? when you're in the FTP tool or at the FTP> prompt.


Vital FTP Commands
The most common and important commands within FTP are:

cd    change directory on the remote SERVER

cd ..    change directory up one level on the remote SERVER

pwd    print the current directory you are within on the remote SERVER

ls    list files within the current directory on the remote SERVER

binary    prepare the FTP tool to transfer binary files such as applications or images

ascii    prepare the FTP tool to transfer ascii or text files such as .html files

put    copy a specific file from the local machine to the remote SERVER

get    copy a specific file from the remote SERVER to the local PC

chmod    change the file permissions on a remote SERVER if you have access

del    delete a specific file on the remote SERVER

bye    end your FTP connection

Okay, while you're in FTP you can also do things on your local PC, which is often necessary to make life easier.

lcd change directory on your local PC
lcd .. change directory up one level on your local PC
lpwd print the current directory you are within on your local PC


For an example of a common set of commands used to change an existing photograph on my website then I could, once logged in, use this:
cd www
cd Photos
binary
put newimage.jpg

I change directory to www then to Photos then set the file transfer mode to binary (used for images or applications). The final command copies the photo newimage.jpg from my local PC to the remote Server.

Copying in Batches
But wait, you say, I know all that jibberish above! What I really need is a way to copy batches of files or make changes to a whole directory of files.

Not to worry. The FTP tool allows for this as well.

mput copy multiple files using a wild card like * from the local PC to the server
mget copy multiple files using a wild card like * from the server to the local PC

For example:
lpwd
local directory /home/andrea/photos
pwd
/www/Photos
mput *.jpg

It's always a good idea to check which local directory I'm in. Next, I confirm that I'm in the correct remote server directory to send the files to. Finally, I run the command to copy only my jpeg images to the remote server.

For beginners this is plenty of good useful information. However, let me share some tidbits and tricks used by those Linux experts to manage systems easier.


Automate FTP
If you've used the command line FTP before, you know that there are a number of manual inputs required. You obviously need to designate your FTP site, your login, password and then change to the appropriate directory.

In the next few steps I will help you create two "script" files. One is called a shell script and will automate running certain Linux commands. The second is a simple FTP script that automates your FTP work.

Some of the lingo may get a bit techno but please bear with me. At the very least, you can simply copy my examples and try them out. There's no better way to learn!

Create Shell Script Startup File: ftpstartup
Begin by creating a file named ftpstartup. Create and save this file into the directory you will be transferring data to/from using your favorite text editor.

In the ftpstartup file include these lines:

#!/bin/sh
# This script allows you to divert output of
# date command to the mkdir command
# resulting in creating a directory of todays date
Today="`date +%m%d`"
mkdir $Today
cp * $Today
FTP < autoftp

Now save this file.

This file does wonderful and helpful things!

Once you run this file, which we haven't done yet the script will automate the following tasks:

1. It creates a means of naming a backup directory today's date (lines 5 and 6)
   
2. It then copies all existing files you plan to FTP to the newly created backup directory
   
3. Finally it starts the FTP script file we will create next to automate all FTP functions
   

How do I do all this in four lines?! Well, for those who are interested here are the details. OTHERWISE, PLEASE SKIP AHEAD TO FINISH THE TASK.

The starting line #!/bin/sh indicates that this is a shell script that should use the bash shell in Linux. If that confuses you don't worry. It allows you to automate tasks.

We automate a MAJOR painful task of creating backup directories based on today's date by passing the variable "Today" into the commands we want like mkdir and cp.

Once this happens you run the date command, get the output of the current Month and Day and pass this to the mkdir command.

The mkdir command then makes a backup directory with today's date. You can use this technique for thousands of different automated tasks in Linux.

The cp command copies all of the files to the backup directory, and thus we ensure that even if FTP flakes out we have a daily backup of our files.

You can change the backup from daily to weekly, monthly, whatever. For instance just remove the %d from the date command in line 5 and you will only retain monthly backups of files. This is very useful since ftping using automated processes sometimes may hiccup.

Now let's keep going, this is going to get even more useful.


Create FTP Script File: autoftp
It's time to create your autoftp file that automates your FTP work for you. In the text editor write these lines:

open mysamplesite.com
andreac
samplepasswd
cd www
cd LatestPhotos
binary
mput *.jpg
bye

In this file:
I first open an FTP connection to my website; replace with your own site and relevant information of course;
I then login with my personal account and my password;
Then I perform the needed change directory commands to go to the appropriate location;
In most cases you need to drill down a few directories. You can also use one simple line like: cd /www/LatestPhotos;
I change my FTP file type to Binary since I will be transferring photographs;
Then I use the mput command to send all of the jpeg images in my local directory to the destination website through FTP.
Of course, I need to close the FTP connection when you're done!

When finished entering your own specific commands save the file.

It's that easy!

Run Your Scripts
From now on when ever I need to update my website Photos I run the following command from the Linux command line:
bash ftpstartup

The result should be a totally automated process you can just watch and enjoy!

That's it! And hopefully along the way I also helped you learn about shell scripting, how powerful FTP script files are, and finally the joy of automation!


Secure FTP Use
Obviously, some of the techniques above are not very useful in a security sensitive environment where passwords and data should not be passed in the clear. For those situations where things must be highly secure, you can use the sftp command.

The benefit of using sftp is that it allows all data to be transferred using an SSH encrypted transport method. Some of the sftp commands differ and unfortunately you must connect to an FTP server that supports ssh connectivity to use this.

However, you will also find that sftp offers more control over your remote and local server including providing a local mkdir and other commands to control both computers through the sftp session.

Next time I'll share how you can automatically start FTP or other sessions at certain times of day using the at command or during certain days of the month using the cron tables.


Hopefully this beginner guide was helpful to you!

Many more beginner articles are available here.