What/Why

This is a simple command line tool to transfer files to/from S3. I wrote this tool to be used to transfer files from S3 while setting up Amazon EC2 instances in an automated fashion. It makes most sense if you start your EC2 instance with an IAM profile role which assigns temporary AWS credentials to your EC2 instance - obviating the need to specify or embed your real credentials in the instance.

Requirements

You need to have java runtime installed on your system. So technically, anywhere you have java installed, this tool will run.

Installation

There are two choices - you can either build using source or just download the self executing JAR file binary.

Option #1 - Download binary

Go to the download section and just download the binary executable JAR file. I use wget to download the binary on my EC2 instances.

wget https://bitbucket.org/techwarriors/simple-s3-tool/downloads/simple-s3-tool-latest-executable.jar

Option #2 - Build from source

You will need maven for this to work. First, clone the git repository:

git clone https://salmanmalik@bitbucket.org/techwarriors/simple-s3-tool.git

and then run maven to create the JARs:

mvn package

This will produce two JAR files. All you need is the *-executable.jar to run the tool.

Running

To run, simply invoke the executable JAR file with appropriate arguments. If you forget the syntax, just run without any arguments like so:

java -jar simple-s3-tool-latest-executable.jar

... this outputs

Usages:

java -jar xxx-executable.jar put {source-file} {s3-destination}

 * Transfers a single local file to S3. If the destination ends in
 * '/', it is used as a prefix, otherwise it is used as the key.

java -jar xxx-executable.jar putdir {source-dir} {s3-destination}

 * Transfers a directory and all its contents to S3. If the
 * destination ends in '/', it is used as a prefix, otherwise
 * it is used as the key.

java -jar xxx-executable.jar get {s3-source-key} {destination-dir-or-file}

 * Transfers a single S3 object to local file. If the destination
 * is a directory, the object is transfered into it, otherwise it
 * is used as the filename.

java -jar xxx-executable.jar getdir {s3-source-prefix} {destination-dir}

 * Transfers all S3 objects with the given prefix to local directory.
 * If the destination directory does not exist, it is created.

java -jar xxx-executable.jar ls {s3-source-prefix}

 * Lists all S3 objects with the given prefix to stdout.

Runtime arguments

The destination or source S3 URIs for all commands are of the following format:

{s3-buket-name}/{object-or-path-key}

AWS Credentials

Since, the tool uses AWS Java SDK, it can determine the AWS credentials to access s3 via the different ways outlined in the javadoc. If you are running on an EC2 instance started with an IAM profile role name, you don't have to specify any AWS credentials.

Examples

These examples assume that you are either running on an EC2 instance started with IAM profile role or have specified your AWS credentials via environment variables.

To copy foo.txt to my-bucket under the key of bar/foo.txt

java -jar xxx-executable.jar put foo.txt my-bucket/bar/

To copy foo.txt to my-bucket under the key of ding/bat.txt

java -jar xxx-executable.jar put foo.txt my-bucket/ding/bat.txt

To transfer s3 object with the key ding/bat.txt from my-bucket to the current directory. File will be saved as bat.txt in the current directory.

java -jar xxx-executable.jar get my-bucket/ding/bat.txt .