Saturday, August 20, 2011

drop or clear cache in linux

Many times you may find the system is running out of memory. When checked you can see lots of memory is assigned to buffers and caches.Allocating lots of memory to buffers and caches is not necessary. If you are running mysql and oracle like softwares, they have their own buffers and caches. So mostly you can free or drop this buffers and caches. This post explains how to drop caches in Linux. Also the entry for sysctl.conf so that it will remember the action.

According to the linux documentations, the variable drop_caches defined as, Writing to this will cause the kernel to drop clean caches, dentries and inodes from memory, causing that memory to become free.

To free pagecache:
echo 1 > /proc/sys/vm/drop_caches
To free dentries and inodes:
echo 2 > /proc/sys/vm/drop_caches
To free pagecache, dentries and inodes:
echo 3 > /proc/sys/vm/drop_caches

As this is a non-destructive operation and dirty objects are not freeable, the user should run `sync' first.
So the command to drop all caches are,
#sync; echo 3 > /proc/sys/vm/drop_caches

Or you can specify this in /etc/sysctl.conf
#echo "vm.drop_caches = 3" >> /etc/sysctl.conf
Now reload sysctl.conf
#sysctl -p

Thursday, August 18, 2011

Installing nagios with nrpe to monitor remote hosts

This post explains installing nagios with nrpe to monitor remote hosts. Nagios is one of the most used monitoring tools today.

On Remote client server to be monitored:

Create the user nagios and set password:
# useradd nagios
# passwd nagios

Download the nagios plugin from http://www.nagios.org/download/plugins

# mkdir -p /opt/Nagios/Nagios_Plugins
# cd /opt/Nagios/Nagios_Plugins
# cd ..
# tar xzf nagios-plugins-1.4.15.tar.gz
# cd nagios-plugins-1.4.15

Compiling and Installing:
Pere-requisite openssl-devel package.
#rpm -q openssl-devel
if not installed, then
# yum -y install openssl-devel

Configuring: 

# cd /opt/Nagios/nagios-plugins-1.4.15
# ./configure --with-nagios-user=nagios --with-nagios-group=nagios
If the configure struck with ICMP ping check run as below
./configure --with-nagios-user=nagios --with-nagios-group=nagios --with-ping-command=ping
# make
# make install

Changing permissions:
# chown nagios.nagios /usr/local/nagios
# chown -R nagios.nagios /usr/local/nagios/libexec

Installing xinetd super demon if not installed
# yum install xinetd

Now downloading and installing nrpe demon from

# mkdir -p /opt/Nagios/Nagios_NRPE
# cd /opt/Nagios/Nagios_NRPE
#cd ..
# tar -xzf nrpe-2.12.tar.gz
# cd nrpe-2.12

Compiling and Configuring nrpe
# cd /opt/Nagios/nrpe-2.12
# ./configure 
# make all
# make install-plugin
# make install-daemon
# make install-daemon-config
# make install-xinetd

Add Nagios Monitoring server to the “only_from” directive
# vi /etc/xinetd.d/nrpe
only_from =  

Add entry for nrpe daemon to services
# vi /etc/services
nrpe      5666/tcp    # NRPE

Restart Xinetd and set chkconfig on
# chkconfig xinetd on
# service xinetd restart
Checking whether NRPE daemon is running and listening on port 5666:
# netstat -at |grep nrpe
tcp    0    0 *:nrpe    *.*    LISTEN

Open Port 5666 on Firewall
if using csf add 5666 to TCP_IN and TCP_OUT in /etc/csf/csf.conf and restart as
#csf -r

And add the following lines to /usr/local/nagios/etc/nrpe.cfg

command[check_users]=/usr/local/nagios/libexec/check_users -w 5 -c 10
command[check_load]=/usr/local/nagios/libexec/check_load -w 15,10,5 -c 30,25,20
command[check_hda1]=/usr/local/nagios/libexec/check_disk -w 20% -c 10% -p /dev/hda1
command[check_zombie_procs]=/usr/local/nagios/libexec/check_procs -w 5 -c 10 -s Z
command[check_total_procs]=/usr/local/nagios/libexec/check_procs -w 150 -c 200
command[check_disk]=/usr/local/nagios/libexec/check_disk -w 20% -c 10% -p /dev/sda
command[check_mem]=/usr/local/nagios/libexec/check_mem 85 95


Nagios server  Setup (mail nagios server)
Downloading and installing nrpe demon from

# mkdir -p /opt/Nagios/Nagios_NRPE
# cd /opt/Nagios/Nagios_NRPE
#cd ..
# tar -xzf nrpe-2.12.tar.gz
# cd nrpe-2.12

Compiling and Configuring nrpe
# cd /opt/Nagios/nrpe-2.12
# ./configure 
# make all
# make install-plugin

Check NRPE daemon is functioning from nagios server. 
# /usr/local/nagios/libexec/check_nrpe -H
Output:
NRPE v2.12

Check whether it is defined or not.
# vi /usr/local/nagios/etc/objects/commands.cfg
define command{
        command_name check_nrpe
        command_line $USER1$/check_nrpe -H $HOSTADDRESS$ -c $ARG1$
        }

If you want to add a few ips then define them in hosts.cfg
and make a hostgroup in hostgroups.cfg with all the needed users as members
and now in services.cfg and the services you want and specidy hostgroup name as follows

### CPU LOAD/Load Average ###
define service{
        use                             basic-service
        hostgroup_name                  customer1
        contact_groups                  admins
        service_description             CPU LOAD
        check_command                   check_nrpe!check_load
}

### Disk Usage ###
define service{
        use                             basic-service
        hostgroup_name                  customer1
        contact_groups                  admins
        service_description             CHECK DISK
        check_command                   check_nrpe!check_disk
}

### RAM Usage ###
define service{
        use                             basic-service
        hostgroup_name                  customer1
        contact_groups                  admins
        service_description             CHECK MEM
        check_command                   check_nrpe!check_mem

Check the configuration  as :
#/usr/local/nagios/bin/nagios -v /usr/local/nagios/etc/nagios.cfg
Now restart
#/etc/init.d/nagios restart

Wednesday, August 17, 2011

Websites each system admin should know

System administrators will have to do a lot of stuffs by themselves. But there are a lot of online tools which are helpful and they save a lot of time. But most of the administrators don't know them. Here is a list I know of. If you know any please comment with their use, I'll include them.

To know your public ip. It shows location and other details too.
http://whatismyipaddress.com/

For checking nameservers, mx record, A record, PTR record and other DNS details.
http://www.intodns.com/

Example:
http://www.intodns.com/randeeppr.me

For checking the accessibility of your ip address or url. It gives ping results from many parts of the world.
http://www.just-ping.com/

For checking the domain details such as owner of the domain and contact details.
http://www.whois.sc/

Example:
http://whois.domaintools.com/randeeppr.me

To check whether your ip is blacklisted or not
http://www.mxtoolbox.com/blacklists.aspx

To check the loading time of the url from different parts of world
http://alertra.com/spotcheck_do

To check the domain propagation, NS, MX, A Record etc from different parts of the world at the moment.
http://www.whatsmydns.net/

To check the cpanel license is expired or not
http://www.cpanel.net/apps/verify/

To paste the error log and make a link to exchange
http://pastebin.com/

To encrypt a string with md5 algorithm
http://md5-encryption.com/

To decrypt a string with md5 algorithm
http://md5-decrypter.com/

To check the validity and other details of SSL certificates
http://www.digicert.com/help/

Tuesday, August 16, 2011

How to see history with time in linux

First set the variable HISTTIMEFORMAT as given below.
root@cpaneltest [~]# HISTTIMEFORMAT="%d/%m/%y %T "

Now checking the variable value :
root@cpaneltest [~]# echo $HISTTIMEFORMAT
%d/%m/%y %T

root@cpaneltest [~]#
Now run the command history.
root@cpaneltest [~]# history

   21  16/08/11 00:39:47 for i in $(cat ip) ; do echo $i; done
   22  16/08/11 00:39:47 for i in $(cat ip) ; do ping -t1 $i; done



How to change the values of numproc and privvmpages

You may get memory over consume errors in openvz vps'. Error can be fork error. For Eg:
fork : unable to fork new process

You can confirm the error is because of memory overconsumption by checking the following command.

root@cpanel [/]# cat /proc/user_beancounters
Version: 2.5
       uid  resource        held            maxheld          barrier           limit                                                   failcnt
      101:  kmemsize   21335253   359571636   9223372036854775807  9223372036854775807   0
            lockedpages     0                   10              4925440              4925440                                     0
            privvmpages   235064           7784716             10000000             10000000                45752
            shmpages       24                 1320  9223372036854775807  9223372036854775807            0
            dummy            0                    0                    0                    0                                                   0
            numproc         98                 1500                 4096                 5120                            1992344
            physpages     123667              4965574                    0  9223372036854775807                    0
            vmguarpages          0                    0              9820160  9223372036854775807                    0
            oomguarpages   23668            4969178              9820160  9223372036854775807             0
            numtcpsock     260              3726  9223372036854775807  9223372036854775807            0
            numflock      7                  182  9223372036854775807  9223372036854775807                 0
            numpty               1                    2                  255                  255                                          0
            numsiginfo              0                 1024                 1500                 2000                               905
            tcpsndbuf      13916424   119625936  9223372036854775807  9223372036854775807        0
            tcprcvbuf      4297032             61066072  9223372036854775807 223372036854775807      0
            othersockbuf    120688     5234368  9223372036854775807  9223372036854775807           0
            dgramrcvbuf     0     13080  9223372036854775807  9223372036854775807                    0
            numothersock   104     1519  9223372036854775807  9223372036854775807                    0
            dcachesize    2743654        16510107  9223372036854775807  9223372036854775807         0
            numfile   7077                72257  9223372036854775807  9223372036854775807                    0
            dummy                           0                    0                    0                    0                    0
            dummy                           0                    0                    0                    0                    0
            dummy                           0                    0                    0                    0                    0
            numiptent                   43                   43  9223372036854775807  9223372036854775807      0
root@cpanel [/]#

uid 101 means the veid of the vps. 
The main parameters are explained below:

held - how many kernel level threads you are using at the time you cat the file
maxheld - how many is the maximum number of kernel level threads you have had
barrier - guaranteed amount of resources your vps is allocated
limit - physical limit your vps can consume
failcnt - also known as fail count or the number of times your vps has hit the specified limit

You can increase the value of these parameters by editing the configuration file of the vps on the node server. 
# vi /etc/vz/conf/101.conf

#NUMPROC="1500:1800"
NUMPROC="4096:5120"

#PRIVVMPAGES="7772160:7784660"
PRIVVMPAGES="10000000:10000000"

#NUMSIGINFO="1024:1024"
NUMSIGINFO="1500:2000"

Restart the vps after that. 
# vzctl restart 101