Wednesday, April 11, 2012

Creating Amazon Linux AMI

We cannot say Amazon ec2 instance wont go down or data wont be lost. Its always better to hve backups. But a data backup is not a easy to restore option. So it is always better to make a image of your amazon ec2 instance and keep it somewhere, for example in amazon simple storage or s3. In this post we will discuss how to create image or full backup of an amazon ec2 instance, how to upload the amazon ec2 instace image or AMI to amazon s3 bucket, how to register the AMI with ec2 account and how to create a amazon ec2 instance based on the created AMI. I'm sure all those things we'll discuss here will be possible to do with GUI but some of them are possible with the Mozilla addon ElasticFox. But we will do everything from the command line.

installing s3cmd in amazon ec2-instance

s3cmd is a command line tool for uploading, downloading and managing file and directories with amazon simple storage s3. It is very useful when running scripts and scheduling scripts with cron. First you have to install s3cmd package which is available from s3tools.org. Here we are installing s3cmd in a Centos 5 instance using yum.

Tuesday, April 10, 2012

Multiple passowrdless ssh logins


We have discussed the passwordless authentication or passwordless logins in our previous post. But what if you have to allow more than one hosts to login to a server without password? Then you have to add the dsa/rsa keys of the initiator servers to the destination server authorized_keys file.
Suppose we have three systems A,B abd C. And we want to login to system C without password from A and B.

All we have to do is

1. Generage dsa/rsa key in system A and copy that to authorized_keys file of C.
2. Generate dsa/rsa key in system B and APPEND that key to the authorized_key file of C.

Generating the key in system A:
[root@nagios ~]# ssh-keygen -t rsa
Generating public/private rsa key pair.
Enter file in which to save the key (/root/.ssh/id_rsa):
Created directory '/root/.ssh'.
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /root/.ssh/id_rsa.
Your public key has been saved in /root/.ssh/id_rsa.pub.
The key fingerprint is:
14:13:25:f1:c6:ed:51:c6:08:a4:3f:af:eb:2c:80:97 root@nagios.lap.work

Copying the key to the authorized_keys of system C:
[root@nagios ~]# scp /root/.ssh/id_rsa.pub 192.168.137.85:/root/.ssh/authorized_keys
The authenticity of host '192.168.137.85 (192.168.137.85)' can't be established.
RSA key fingerprint is 63:6d:4a:08:b4:b4:19:3c:d0:58:f3:60:8a:ec:7a:a0.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added '192.168.137.85' (RSA) to the list of known hosts.
root@192.168.137.85's password:
id_rsa.pub                                                                              100%  402     0.4KB/s   00:00
[root@nagios ~]#

Checking the key from the system C:
[root@test ~]# cat .ssh/authorized_keys
ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAydSsh5wlG/lvWFeZcI+Rlxr2hTWJ4diU7b1/OsDWE72goA72eIx+tfzg6/aT4vPbWA8GC8arK6XxLOWJbv2Y5tFRGmXwn+Trw3RzWOHFT76NTv6NP+SCvBciwTr55Tt6jIgGrVu6f/pBvU8tIgctu/5efH611w/pToIJbezlooJ/1GGWaydEc3eTJernwzia5UMEsRGIztT6GN8zqkVtKIRhql3y2lQjgg3jA4ceAXwJ8h49xFuo8ZIEo4mWmEwW8Kn2VaTnJVh/YsO7tMRs8KsWXonbTm0vtD2OQv59Lswjs5fMmBv0EGZJvZ3uDypQw/IH33MWKbAotwQ1fewbiw== root@nagios.lap.work
[root@test ~]#

Now creating the key in system B:
[root@server ~]# ssh-keygen -t rsa
Generating public/private rsa key pair.
Enter file in which to save the key (/root/.ssh/id_rsa):
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /root/.ssh/id_rsa.
Your public key has been saved in /root/.ssh/id_rsa.pub.
The key fingerprint is:
5e:7f:e6:bc:3e:bc:9f:65:2f:b3:95:89:d6:0e:9d:5f root@server.lap.work
[root@server ~]#

Now APPENDING (Do not copy it will overwrite the key of the system A) the key of system B to the authorized_keys of system C:
First we will copy the key to a file abc.txt in system C.
Then we will append the file abc.txt to athorized_keys of system C.

[root@server ~]# scp /root/.ssh/id_rsa.pub 192.168.137.85:/root/.ssh/abc.txt
The authenticity of host '192.168.137.85 (192.168.137.85)' can't be established.
RSA key fingerprint is 63:6d:4a:08:b4:b4:19:3c:d0:58:f3:60:8a:ec:7a:a0.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added '192.168.137.85' (RSA) to the list of known hosts.
root@192.168.137.85's password:
id_rsa.pub                                                                              100%  402     0.4KB/s   00:01
[root@server ~]#

Now in system C:
[root@test ~]# cat .ssh/abc.txt
ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAys2LlRFyQZay+9QWaCT6mS7gmM6qN0GzCGM7AXAMlEDWHUSmXSC9EPih4uOAGH6IWGqRk7EVerVEMq39vVchDAE5B3nMofQkc2fAlC9Ct/5+TirQaQxmHCN0If6O+RlO4F3hVhqX7d0ZNjJhvWLezRXsXkZY+g0215nd+qeZSz39N8NtkKBuuYW7LFdEU8dmiUaFrUjkBpZYuP5THaGqD/wZr8Pxf7t/MIpRbkuleP7b6S8kEreR9AdDX5DWJOy3qqxZzJVfXgYH6wq/MDuY14X+p1zJjzqQRV8cD7rA2Q8WQy4R7oBAJvZk9Q5gkyt50rDfiMXLPYF1myrfo/kDpQ== root@server.lap.work
[root@test ~]#

Appending the key in the file abc.txt to authorized_keys
[root@test ~]# cat .ssh/abc.txt >> .ssh/authorized_keys

Now checking the authorized_keys:
[root@test ~]# cat .ssh/authorized_keys
ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAydSsh5wlG/lvWFeZcI+Rlxr2hTWJ4diU7b1/OsDWE72goA72eIx+tfzg6/aT4vPbWA8GC8arK6XxLOWJbv2Y5tFRGmXwn+Trw3RzWOHFT76NTv6NP+SCvBciwTr55Tt6jIgGrVu6f/pBvU8tIgctu/5efH611w/pToIJbezlooJ/1GGWaydEc3eTJernwzia5UMEsRGIztT6GN8zqkVtKIRhql3y2lQjgg3jA4ceAXwJ8h49xFuo8ZIEo4mWmEwW8Kn2VaTnJVh/YsO7tMRs8KsWXonbTm0vtD2OQv59Lswjs5fMmBv0EGZJvZ3uDypQw/IH33MWKbAotwQ1fewbiw== root@nagios.lap.work
ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAys2LlRFyQZay+9QWaCT6mS7gmM6qN0GzCGM7AXAMlEDWHUSmXSC9EPih4uOAGH6IWGqRk7EVerVEMq39vVchDAE5B3nMofQkc2fAlC9Ct/5+TirQaQxmHCN0If6O+RlO4F3hVhqX7d0ZNjJhvWLezRXsXkZY+g0215nd+qeZSz39N8NtkKBuuYW7LFdEU8dmiUaFrUjkBpZYuP5THaGqD/wZr8Pxf7t/MIpRbkuleP7b6S8kEreR9AdDX5DWJOy3qqxZzJVfXgYH6wq/MDuY14X+p1zJjzqQRV8cD7rA2Q8WQy4R7oBAJvZk9Q5gkyt50rDfiMXLPYF1myrfo/kDpQ== root@server.lap.work
[root@test ~]#

Now checking the passwordless login from A to C
[root@nagios ~]# ssh 192.168.137.85 ls
anaconda-ks.cfg
Desktop
install.log
install.log.syslog
[root@nagios ~]#

Now checking the passwordless login from B to C
[root@server ~]# ssh 192.168.137.85 ls
anaconda-ks.cfg
Desktop
install.log
install.log.syslog
[root@server ~]#

You should not expose the keys to others. My system is for testing use and the domain is private. That is why I don't mind to share them.

ssh passwordless login


Configuring password authentication or login via ssh. This post explains how to enable password less authentication between two nodes. The configuration is very simple. You have to generate dsa public and private keys of the server which you want to login from  and copy that to the authorized_keys of the host you want to login to without password. We will generate the keys using the command ssh-keygen.

We have two nodes:
Node1 - hb_test1.lap.work
Node2 - hb_test2.lap.work

On node1:
Generate the key:

[root@hb_test1 ~]# ssh-keygen -t dsa
Generating public/private dsa key pair.
Enter file in which to save the key (/root/.ssh/id_dsa):
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /root/.ssh/id_dsa.
Your public key has been saved in /root/.ssh/id_dsa.pub.
The key fingerprint is:
9f:5d:47:6b:2a:2e:c8:3e:ee:8a:c2:28:5c:ad:57:79 root@hb_test1.lap.work

Pass the key to node2:
[root@hb_test1 ~]# scp .ssh/id_dsa.pub hb_test2.lap.work:/root/.ssh/authorized_keys

On node2:
Generate the key:

[root@hb_test2 ~]# ssh-keygen -t dsa
Generating public/private dsa key pair.
Enter file in which to save the key (/root/.ssh/id_dsa):
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /root/.ssh/id_dsa.
Your public key has been saved in /root/.ssh/id_dsa.pub.
The key fingerprint is:
40:66:t8:bd:ac:bf:68:38:22:60:d8:9f:18:7d:94:21 root@hb_test2.lap.work

Pass the key to node1:
[root@hb_test2 ~]# scp .ssh/id_dsa.pub hb_test1.lap.work:/root/.ssh/authorized_keys

Now you will be able to login to node1 to node2 and vice versa without passwords.

Monday, April 9, 2012

s3cmd example commands


s3cmd is a tool for uploading, downloading and managing file and directories with amazon simple storage s3 which is a storage service in aws. Here we will see how to create and remove amazon simple storage s3 buckets, how to upload, download, delete files from and to your Linux system using s3cmd, sync directories etc.

/usr/bin/s3cmd: unrecognized option `--configure'


s3cmd is a tool for uploading, downloading and managing file and directories with amazon simple storage s3. But while configuring s3cmd in your amazon ce2 cloud instance you may get the following error. I got this error in my centos 5.4 instance on amazon ec2.
/usr/bin/s3cmd: unrecognized option `--configure'

[root@xxxxxxxx ~]# s3cmd --configure
/usr/bin/s3cmd: unrecognized option `--configure'
s3cmd [options] <command> [arg(s)]              version 1.2.6
  --help    -h        --verbose     -v     --dryrun    -n
  --ssl     -s        --debug       -d     --progress
  --expires-in=( <# of seconds> | [#d|#h|#m|#s] )

Commands:
s3cmd  listbuckets  [headers]
s3cmd  createbucket  <bucket>  [constraint (i.e. EU)]
s3cmd  deletebucket  <bucket>  [headers]
s3cmd  list  <bucket>[:prefix]  [max/page]  [delimiter]  [headers]
s3cmd  location  <bucket> [headers]
s3cmd  delete  <bucket>:key  [headers]
s3cmd  deleteall  <bucket>[:prefix]  [headers]
s3cmd  get|put  <bucket>:key  <file>  [headers]
[root@xxxxxxx ~]#

Solution:
You have to reinstall the s3cmd package as follows.
you can get the repo from here
http://s3tools.org/repo/RHEL_5/

save the repo in your /etc/yum.repos.d/ as follows


[root@xxxxxxx ~]# cat /etc/yum.repos.d/s3cmd.repo
#
# Save this file to /etc/yum.repos.d on your system
# and run "yum install s3cmd"
#
[s3tools]
name=Tools for managing Amazon S3 - Simple Storage Service (RHEL_5)
type=rpm-md
baseurl=http://s3tools.org/repo/RHEL_5/
gpgcheck=1
gpgkey=http://s3tools.org/repo/RHEL_5/repodata/repomd.xml.key
enabled=1
[root@xxxxxxx ~]#


After that

Install it using yum:

yum install s3cmd

Now configure it. It will ask your access key, secret key and encryption key(just hit enter if you don't want).
s3cmd --configure

Now you will be able to list your buckets in your amazon s3 storage using the following command.
s3cmd ls