Using s3cmd to connect to AWS S3

TL/DR: Use S3cmd and create specific policies for it in IAM

S3cmd is a very useful command line tool that allows files to be added and removed from an AWS bucket, directly from the command line of a server such as Ubuntu or CentOS.

I use it for automated backups by writing a Bash script to compress a folder or database dump and then upload it directly to a folder in an S3 bucket.

To get S3cmd to work, you must give it an Access Key ID and a Secret Access Key from your AWS account.

To do this in a secure way, use AWS’s IAM to create a new user and then create a custom policy to give this user only permission to ListBuckets, ListAllBuckets and PutObject. Those are the minimum permissions that s3cmd needs to work.

Once you have created the policy, it works straight away.

Back on your server, while logged in as the user you want to give access to s3, run s3cmd –configure to set up s3cmd to connect to your s3 bucket.

Answer the questions on screen and give it the IAM Access and Secret Key from above. These will be saved in to a file calles /home/YOURUSERNAME/.s3cfg

If you want to change the settings later you can re run the configure or edit the file directly.

Once in place, you’ll be able to run a command like this to put a file in your bucket:

s3cmd put path/to/local/file s3://bucketname/folder/filename

Leave a Reply

Your email address will not be published. Required fields are marked *