s3cmd example commands

Advertisements

s3cmd is a tool for uploading, downloading and managing file and directories with amazon simple storage s3 which is a storage service in aws. Here we will see how to create and remove amazon simple storage s3 buckets, how to upload, download, delete files from and to your Linux system using s3cmd, sync directories etc.

List the storage buckets in amazon s3 storage. This command lists all the s3 buckets created under your account.
[root@xxxxxxx ~]# s3cmd ls

How to create a bucket using s3cmd command:
s3cmd mb s3://s3cmdtestbucket_randeep
Bucket 's3://s3cmdtestbucket_randeep/' created
 
How to delete a bucket:
s3cmd rb s3://s3cmdtestbucket_randeep
Bucket 's3://s3cmdtestbucket_randeep/' removed


To upload a file abc.txt to a bucket named bucket_name. This command uploads a single file to the specified s3 bucket.
[root@xxxxxxx ~]# s3cmd put abc.txt s3://bucket_name/
abc.txt -> s3://bucket_name/abc.txt  [1 of 1]
 4 of 4   100% in    0s     9.21 B/s  done


Checking the uploaded file by listing.
[root@xxxxxxx ~]# s3cmd ls s3://bucket_name/
2012-04-09 09:17         4   s3://bucket_name/abc.txt


To download a single file from the s3 bucket to the local Linux host.
[root@xxxxxxx ~]# s3cmd get s3://bucket_name/test.txt
s3://bucket_name/test.txt -> ./test.txt  [1 of 1]
 3 of 3   100% in    0s    14.15 B/s  done


To delete a file from the s3 bucket
[root@xxxxxxx ~]# s3cmd del s3://bucket_name/test.txt
File s3://bucket_name/test.txt deleted
[root@xxxxxxx ~]#


To sync between a directory in local host. Here we will create a directory folder in local host and sync with directory dir inside the bucket bucket_name. Now in folder there are 4 files and no files in dir directory of bucket_name.


[root@xxxxxxx ~]# mkdir folder
[root@xxxxxxx ~]# cd folder
[root@xxxxxxx folder]# touch tt gg ff ss
[root@xxxxxxx folder]# cd ..


Syntax:
#s3cmd sync source destination
The source or destination can be directory or bucket_name


Now syncing:
[root@xxxxxxx ~]# s3cmd sync folder/ s3://bucket_name/dir/
folder/ff -> s3://bucket_name/dir/ff  [1 of 4]
 0 of 0     0% in    0s     0.00 B/s  done
folder/gg -> s3://bucket_name/dir/gg  [2 of 4]
 0 of 0     0% in    0s     0.00 B/s  done
folder/ss -> s3://bucket_name/dir/ss  [3 of 4]
 0 of 0     0% in    0s     0.00 B/s  done
folder/tt -> s3://bucket_name/dir/tt  [4 of 4]
 0 of 0     0% in    0s     0.00 B/s  done


Now checking by listing.
[root@xxxxxxx ~]# s3cmd ls s3://bucket_name/dir/
2012-04-09 09:38         0   s3://bucket_name/dir/ff
2012-04-09 09:38         0   s3://bucket_name/dir/gg
2012-04-09 09:38         0   s3://bucket_name/dir/ss
2012-04-09 09:38         0   s3://bucket_name/dir/tt
[root@xxxxxxx ~]#
Now all the files in folder is synced with dir in bucket_name.


To download a bucket or directory to local directory recursively.
[root@xxxxxxx ~] s3cmd get --recursive  s3://bucket_name/directory/
 dest_directoryRestore file from Glacier storage to s3 standard storage:
#s3cmd restore s3://BUCKET/OBJECT

Checking the size of s3  bucket using s3cmd command:
#s3cmd du -H s3://cloudtaillogs
1671k    s3://cloudtaillogs/

Copy a s3 object using s3cmd command
#s3cmd cp s3://BUCKET1/OBJECT1 s3://BUCKET2[/OBJECT2]

Move a s3 object using s3cmd command
#s3cmd mv s3://BUCKET1/OBJECT1 s3://BUCKET2[/OBJECT2]

Related tutorials:
1. installing s3cmd in amazon ec2-instance
2. installing s3cmd in ubuntu

---

Recommended Reading

1. Host Your Web Site In The Cloud: Amazon Web Services Made Easy: Amazon EC2 Made Easy
2. Programming Amazon Web Services: S3, EC2, SQS, FPS, and SimpleDB
3. Middleware and Cloud Computing: Oracle on Amazon Web Services (AWS), Rackspace Cloud and RightScale (Volume 1)

---