• uname -a
• cat /proc/version
• cat /etc/issue
• uname -r
• cat /etc/*version
• cat /etc/*release
• cat /etc/*config

To understand if it is a 64bit or 32bit system look for a 32 or 64 in the result.

The following strings indicate 64bit:

• x86_64
• ia64

The following strings indicate 32bit:

• i386
• i486
• i686

The command “uname -m” should also indicate the machine type.

If in doubt another indicator is to look at an executable such as “file /sbin/init” where the result may indicate something like “ELF 32-bit LSB executable”.

The performance difference between the two is typically 4% to 10% but can be as high as a modest 20% so stability and availability of drivers is probably a more important consideration.

To install the command you need to be on a UNIX machine. It can be a dedicated host, virtual host, in the cloud or simply your MacBook.

Check to see the latest version at http://sourceforge.net/projects/s3tools/files/

cd
mkdir Downloads
cd Downloads
wget http://sourceforge.net/projects/s3tools/files/s3cmd/1.0.0/s3cmd-1.0.0.zip
unzip s3*zip

python setup.py install


This will give an error if it is not able to place the resulting command into some root owned system directories however if this is the case the command still generates a local copy which you can further copy into a personal bin directory under your account allowing you to use the command. Instructions to copy the command to a personal bin directory:

mkdir ~/bin
cp s3cmd ~/bin
cp -R S3 ~/bin

You should now have access the command and need to configure it to have access to your Amazon S3 account. Get the keys from https://aws-portal.amazon.com/gp/aws/developer/account/index.html

cd
s3cmd –configure (That is two hyphens)

If the s3cmd isn’t found add the following to your ~/.profile so that your PATH is searching ~/bin for commands.

# set PATH so it includes user's private bin if it exists
if [ -d "$HOME/bin" ] ; then
PATH="$HOME/bin:$PATH"
fi
then type:
. ~/profile
s3cmd –configure (That is two hyphens)

If you run into problems with prerequisites (Software that needs to be on your machine before you run the process above) then please leave a comment. For example you may need to install Xcode (The developer package in the AppStore) on the Mac if you find that you do not have python or wget installed.

The command can be used for example to copy a file or synchronize a directory. In these examples the list of buckets defined (Either from a command line or in the Amazon AWS console) is listed and a set of javaScript files are placed in a S3 bucket.

s3cmd ls

s3cmd put --acl-public --guess-mime-type mymasterdirectory/*js s3://somebucket


Some serious complexity arrises for sites which have authentication in the form of a login and which need to operate across multiple regions but this in many cases can be handled in a less than perfect manner to reduce the complexity and cost of the implementation. The final stages of segmenting data from different users so as to reduce the volume of data being synchronized across databases is necessary only in situations where the frequently required user related data is greater than 5 GB causing data not to be cacheable in memory. Once over 100GB of database data is needed by the system as a whole it may also be necessary to segment data over multiple databases in order to improve the manageability of the databases.

The series of posts will include:

• Use of online storage (S3)
• Relational Databases (RDS)
• Authentication
• Database Synchronization
• Geo Location by DNS
• Geo Location with Edge Servers
• Data Segmentation with Edge Servers
• Workflow management with SQS

There will also be some pre-requisites such as the installation and configuration of tool sets which also warrant separate posts.

Instead of using this I had to format a drive. Plug it into different machines to do this. Deal with fact that OS X file systems were not supported and that OS X 10.5 can read but not write to NTFS. At the end of the process I was left with stubborn permisioning errors from the Mac which need resonlution through changes to the SMB server which had a hard wired smb.conf.

Another elusive goal was media server services being able to operate from the device. This is of course of no use for any equipment in a primarily Mac based home. The services which failed to materialize after about ten hours of research and s significant hard cost on equipment were no more functional than with dropbox and youtube. In fact the throughput of the local server at 3MB / second was less than the FIOS line to online services.

Once again one is reminded why the changes to the cloud are happening. It saves time and money and actually works better than integrating solutions ones self.

The applications appear as windows which are tabs within the browsers. For the technically sophisticated we know that different technologies are being used as compared to native applications but for the end user it is the user experience that is so improved. They now have tabs. Tabs which don’t obscure each other and which can be managed and grouped. If you want to see a natural extension of how this can work use the new Beta of Firefox 4.0 which allows you to group tabs into a desktop like layout.

The user experience is so much improved that even users who are unaware of where there files actually are stored and where they are cached will prefer the experience and people will choose to shift towards the environment. The new CR-48 which demos the Chrome Operating System (OS), is going to be just the beginning of the first of many tools using Chrome OS which assumes that the Chrome browser is the sole user interface and which supports the use of cloud based tools and a client which is again a thin client with cached data from the cloud.

Maybe this time around the thin client will succeed as the setup of the server end is a one time process within the cloud. I think on the tablet type environments this outcome will appear natural and there will be users flocking towards low cost tablet solutions providing a Chrome/Chrome OS environment.

The line:

echo -n "$((${NUMBER}*100/${MAXPROG})) %     "

calculates the percentage of NUMBER/MAXPROG which in this case was a number that indicated the progress across an ordered and evenly spaced set of numbers. Your application will of course vary but NUMBER should be something that indicates how far along the process is as a percent where as MAXPROG is some value that indicates the maximum amount of progress. How you calculate NUMBER will depend on your application and the accuracy and ease of calculating the progress is of course your real task.

The echo -n means that the percentage is printed without moving the cursor to the next line. The series of spaces after the percentage symbol and the double quotes which cause these spaces to be printed. Are there to cope with a situation where you are resetting the percentage or where NUMBER doesn’t always follow a strict numerical sequence. i.e. having shown 15% progress you suddenly estimate that it was actually only 5%. In this case it may be that you increased MAXPROG.

In the script below I calculated MAXPROG as the number of items in the list. Your mechanism will vary. Using a number as the value from the list will only result in a true percentage of progress in a situation where the list is a list of incrementing integers from 1. i.e. counting. You will have to calculate NUMBER in a way that is appropriate for your application as well and operate the loop in whatever way makes sense for your situation.

The echo -n R | tr ‘R’ ‘\r’ is really not elegant. It is just a way of me printing a carriage return which is portable. A carriage return code was used on early electrical printers as a instruction to move the print head back to the left and start again at the first column. It moves the cursor to the left and again due to the -n flag does not move the cursor to a new line. As a result the next time you go around your loop the percentage will be printed over the top of the first percentage.

The final echo statement moves the cursor to a new line so that your prompt is not printed over the top of the final 100%.

#!/bin/bash

LIST="1 2 3 4 5"

MAXPROG=$(echo ${LIST} | wc -w)

for NUMBER in ${LIST};
do
 echo -n "$((${NUMBER}*100/${MAXPROG})) %     "
 echo -n R | tr 'R' '\r'
 sleep 2
done

echo