Installing the s3cmd to Access Amazon S3 cloud file storage

A convenient way to access amazon S3 storage is with the s3cmd. It is similar to an ftp command and rsync combined so as to allow you to script and automate copying of files into S3 storage. You might do this to mirror your static files in S3 from where you are using them for CloudFront as a CDN for faster web access or it might be a method of backing up or distributing software for S3.

To install the command you need to be on a UNIX machine. It can be a dedicated host, virtual host, in the cloud or simply your MacBook.

Check to see the latest version at http://sourceforge.net/projects/s3tools/files/

cd
mkdir Downloads
cd Downloads
wget http://sourceforge.net/projects/s3tools/files/s3cmd/1.0.0/s3cmd-1.0.0.zip
unzip s3*zip

python setup.py install


This will give an error if it is not able to place the resulting command into some root owned system directories however if this is the case the command still generates a local copy which you can further copy into a personal bin directory under your account allowing you to use the command. Instructions to copy the command to a personal bin directory:

mkdir ~/bin
cp s3cmd ~/bin
cp -R S3 ~/bin

You should now have access the command and need to configure it to have access to your Amazon S3 account. Get the keys from https://aws-portal.amazon.com/gp/aws/developer/account/index.html

cd
s3cmd –configure (That is two hyphens)

If the s3cmd isn’t found add the following to your ~/.profile so that your PATH is searching ~/bin for commands.

# set PATH so it includes user's private bin if it exists
if [ -d "$HOME/bin" ] ; then
PATH="$HOME/bin:$PATH"
fi
then type:
. ~/profile
s3cmd –configure (That is two hyphens)

If you run into problems with prerequisites (Software that needs to be on your machine before you run the process above) then please leave a comment. For example you may need to install Xcode (The developer package in the AppStore) on the Mac if you find that you do not have python or wget installed.

The command can be used for example to copy a file or synchronize a directory. In these examples the list of buckets defined (Either from a command line or in the Amazon AWS console) is listed and a set of javaScript files are placed in a S3 bucket.

s3cmd ls

s3cmd put --acl-public --guess-mime-type mymasterdirectory/*js s3://somebucket