How to create / build an rpm package in CentOS / Redhat Linux

Advertisements

RPM - is RPM package manager. Formerly known as Redhat package manager. It is the standard package management system for Redhat Linux distributions. The files manageable by rpm will have an extension of '.rpm'. It is also used in other Linux operating systems like CentOS, Fedora, Oracle Linux and some versions of IBM AIX.

So if we need a new software to be installed on the mentioned linux systems, we download the rpm package for the software and its dependencies and install it like

# rpm -ivh package_name.rpm

Well. If we are a software developer and we want to package our software as rpm? How do we create a rpm file? In this post we will learn how to build rpm package and understanding rpmbuild spec file.

In this example we will create a simple rpm which will install a single script "HelloWorld" which will give output  "Hello World! from HelloWorld RPM." when it is executed. yeah. That simple.

For building the rpm, we need a few softwares to be installed. We can install these using yum.

# yum install rpm-build
# yum install redhat-rpm-config
# yum install rpmdevtools

You need to create the rpm as a normal user. NOT as ROOT user. If you create rpm as root user, it will not create rpm, it will just install the software in your system.

[root@development ~]# adduser build
[root@development ~]# sudo su - build
[build@development ~]$

Once we install all the dependencies, we need to run the following command. It will create the necessary folder structure to put our source code, configuration files etc. 

[build@development ~]$ rpmdev-setuptree

It will create the folder structure in the home directory under the directory named rpmbuild. When you list the files in the directory you can see the following directories.

[build@development ~]$ ls
rpmbuild
[build@development ~]$ cd rpmbuild/
[build@development rpmbuild]$ ls
BUILD  RPMS  SOURCES  SPECS  SRPMS

Now we need to create the spec file for the rpm. Spec file is the main configuration file for building the rpm. We need to create the spec file under the SPECS directory inside rpmbuild.

[build@development rpmbuild]$ cd SPECS/
[build@development SPECS]$ ls
[build@development SPECS]$

Currently this directory is empty.

We will now create the spec file using the command rpmdev-newspec.

[build@development SPECS]$ rpmdev-newspec HelloWorld
HelloWorld.spec created; type minimal, rpm version >= 4.11.

We can see that a brand new spec file is created.

[build@development SPECS]$ ls
HelloWorld.spec
[build@development SPECS]$

We need to edit this file according to our needs. There are a lot of fields. But we will use only the fields we requires in the spec.

[build@development SPECS]$ cat HelloWorld.spec
Name:           HelloWorld
Version:        1.0
Release:        1%{?dist}
Summary:     Installs the HelloWorld script to print "Hello World! from HelloWorld RPM."
License:        GPL - 3
URL:             https://www.linuxhelp.in
Source0:        HelloWorld
%description
Installs the HelloWorld script
%install
mkdir -p %{buildroot}/usr/bin/
install -m 755 HelloWorld %{buildroot}/usr/bin/HelloWorld
%files
%attr(0755, root, root) /usr/bin/HelloWorld
%doc
%changelog
[build@development SPECS]$

We have to  keep the script to be installed with the rpm under the directory SOURCES in rpmuild.

[build@development SPECS]$cd ..
[build@development rpmbuild]$ cd SOURCES/
[build@development SOURCES]$ ls
HelloWorld
[root@development ~]# cat rpmbuild/SOURCES/HelloWorld
#/bin/bash
echo "Hello World! from HelloWorld RPM."
[root@development ~]#

Now we will run the command to create the rpm.

[build@development SPECS]$ rpmbuild -ba HelloWorld.spec
Executing(%install): /bin/sh -e /var/tmp/rpm-tmp.VKHbKH
+ umask 022
+ cd /home/build/rpmbuild/BUILD
+ '[' /home/build/rpmbuild/BUILDROOT/HelloWorld-1.0-1.el7.x86_64 '!=' / ']'
+ rm -rf /home/build/rpmbuild/BUILDROOT/HelloWorld-1.0-1.el7.x86_64
++ dirname /home/build/rpmbuild/BUILDROOT/HelloWorld-1.0-1.el7.x86_64
+ mkdir -p /home/build/rpmbuild/BUILDROOT
+ mkdir /home/build/rpmbuild/BUILDROOT/HelloWorld-1.0-1.el7.x86_64
+ mkdir -p /home/build/rpmbuild/BUILDROOT/HelloWorld-1.0-1.el7.x86_64/usr/bin/
+ install -m 755 HelloWorld /home/build/rpmbuild/BUILDROOT/HelloWorld-1.0-1.el7.x86_64/usr/bin/HelloWorld
+ '[' '%{buildarch}' = noarch ']'
+ QA_CHECK_RPATHS=1
+ case "${QA_CHECK_RPATHS:-}" in
+ /usr/lib/rpm/check-rpaths
+ /usr/lib/rpm/check-buildroot
+ /usr/lib/rpm/redhat/brp-compress
+ /usr/lib/rpm/redhat/brp-strip /usr/bin/strip
+ /usr/lib/rpm/redhat/brp-strip-comment-note /usr/bin/strip /usr/bin/objdump
+ /usr/lib/rpm/redhat/brp-strip-static-archive /usr/bin/strip
+ /usr/lib/rpm/brp-python-bytecompile /usr/bin/python 1
+ /usr/lib/rpm/redhat/brp-python-hardlink
+ /usr/lib/rpm/redhat/brp-java-repack-jars
Processing files: HelloWorld-1.0-1.el7.x86_64
Provides: HelloWorld = 1.0-1.el7 HelloWorld(x86-64) = 1.0-1.el7
Requires(rpmlib): rpmlib(CompressedFileNames) <= 3.0.4-1 rpmlib(FileDigests) <= 4.6.0-1 rpmlib(PayloadFilesHavePrefix) <= 4.0-1
Checking for unpackaged file(s): /usr/lib/rpm/check-files /home/build/rpmbuild/BUILDROOT/HelloWorld-1.0-1.el7.x86_64
Wrote: /home/build/rpmbuild/SRPMS/HelloWorld-1.0-1.el7.src.rpm
Wrote: /home/build/rpmbuild/RPMS/x86_64/HelloWorld-1.0-1.el7.x86_64.rpm
Executing(%clean): /bin/sh -e /var/tmp/rpm-tmp.lSteSm
+ umask 022
+ cd /home/build/rpmbuild/BUILD
+ /usr/bin/rm -rf /home/build/rpmbuild/BUILDROOT/HelloWorld-1.0-1.el7.x86_64
+ exit 0
[build@development SPECS]$

The command has exited with exit status 0. It means all right.

We will get the created rpm in the RPMS directory under rpmbuild

[build@development rpmbuild]$ cd RPMS/
[build@development RPMS]$ ls
x86_64
[build@development RPMS]$ cd x86_64/
[build@development x86_64]$ ls
HelloWorld-1.0-1.el7.x86_64.rpm

Here the RPM is created under x86_64 because we didn't specify any architecture, it took the current system architecture.

We can check what are the files included in the rpm.

[build@development x86_64]$ rpm -qpl HelloWorld-1.0-1.el7.x86_64.rpm
/usr/bin/HelloWorld

Lets install it.

[root@development ~]# rpm -ivh /tmp/HelloWorld-1.0-1.el7.x86_64.rpm
Preparing...                          ################################# [100%]
Updating / installing...
   1:HelloWorld-1.0-1.el7             ################################# [100%]

Checking if the script is installed:

[root@development ~]# ll /usr/bin/HelloWorld
-rwxr-xr-x. 1 root root 52 Feb 26 13:12 /usr/bin/HelloWorld

Checking the source of the script file. 

[root@development ~]# rpm -qf /usr/bin/HelloWorld
HelloWorld-1.0-1.el7.x86_64

Now running the command to check the result.

[root@development ~]# /usr/bin/HelloWorld
Hello World! from HelloWorld RPM.

All good. That is it. Now create your own rpms and let me know if  you have any queries.

introduction to ansible

Advertisements

Ansible is an automation tool or a configuration management tool. Mostly used by linux system administrators. Usually admins use ansible to deploy same things to many servers. It saves a lot of time and it doesn't need any agents to be installed on the remote servers.

Ansible has mainly 3 parts

1. yaml (the language playbooks will be written) yaml stands for yet another programming language
2. engine
3. tower (UI for Ansible management)

These are the main fundamental units in Ansible:
Inventory: Inventory file will have the servers on which the mentioned tasks to be executed. We can have groups in the inventory file. For example, if you have a lot of webservers, you can create a group webserver and put their hostnames or ip addresses in that.

Tasks : smallest executable unit in ansible

Playbook : A group of tasks specified with the inventory they need to run on.

So here we will create a inventory file named hosts. In this example, I have a group named git_server and there is one host with the ip address 172.31.20.156. I am using the user ansible with the specified key.

[ansible@ip-172-31-20-156 ~]$ cat hosts
[git_server]
172.31.20.156

[all:vars]
ansible_user=ansible
ansible_ssh_private_key_file=~/.ssh/id_rsa

Now we will check all the servers listed in the invetory files are up or not.
[ansible@ip-172-31-20-156 ~]$ ansible all -i hosts -m ping
172.31.20.156 | SUCCESS => {
    "changed": false,
    "ping": "pong"
}
we got response for pong. Which means the servers are up.

If you don't want to check the status of all the hosts mentioned in the inventory file but the hosts in the git_server group, you can do that also.
[ansible@ip-172-31-20-156 ~]$ ansible git_server -i hosts -m ping
172.31.20.156 | SUCCESS => {
    "changed": false,
    "ping": "pong"
}

You can check the uptime of the hosts in the groups also.
[ansible@ip-172-31-20-156 ~]$ ansible git_server -i hosts -m command -a "uptime"
172.31.20.156 | SUCCESS | rc=0 >>
 13:50:35 up 35 min,  2 users,  load average: 0.00, 0.01, 0.05

We can check the current ansible parameters of the localhost by using the following command.

[ansible@ip-172-31-20-156 ~]$ ansible localhost  -m setup

Now we will install git on the servers in the group git_server.
For that we need to use the module yum and pass name and state are variables to the ym module. We use -b options to rum the command as root. (sudo user)

[ansible@ip-172-31-20-156 ~]$ ansible git_server -i hosts -m yum -a "name=git state=present" -b
172.31.20.156 | SUCCESS => {
    "changed": true,
    "msg": "",
    "rc": 0,
    "results": [
        "Loaded plugins: fastestmirror\nLoading mirror speeds from cached hostfile\n * base: mirrors.unifiedlayer.com\n * epel: mirrors.kernel.org\n * extras: centos.unixheads.org\n * updates: mirrors.kernel.org\nResolving Dependencies\n--> Running transaction check\n---> Package git.x86_64 0:1.8.3.1-6.el7_2.1 will be installed\n--> Processing Dependency: perl-Git = 1.8.3.1-6.el7_2.1 for package: git-1.8.3.1-6.el7_2.1.x86_64\n--> Processing Dependency: perl(Term::ReadKey) for package: git-1.8.3.1-6.el7_2.1.x86_64\n--> Processing Dependency: perl(Git) for package: git-1.8.3.1-6.el7_2.1.x86_64\n--> Processing Dependency: perl(Error) for package: git-1.8.3.1-6.el7_2.1.x86_64\n--> Processing Dependency: libgnome-keyring.so.0()(64bit) for package: git-1.8.3.1-6.el7_2.1.x86_64\n--> Running transaction check\n---> Package libgnome-keyring.x86_64 0:3.8.0-3.el7 will be installed\n---> Package perl-Error.noarch 1:0.17020-2.el7 will be installed\n---> Package perl-Git.noarch 0:1.8.3.1-6.el7_2.1 will be installed\n---> Package perl-TermReadKey.x86_64 0:2.30-20.el7 will be installed\n--> Finished Dependency Resolution\n\nDependencies Resolved\n\n================================================================================\n Package                Arch         Version                   Repository  Size\n================================================================================\nInstalling:\n git                    x86_64       1.8.3.1-6.el7_2.1         base       4.4 M\nInstalling for dependencies:\n libgnome-keyring       x86_64       3.8.0-3.el7               base       109 k\n perl-Error             noarch       1:0.17020-2.el7           base        32 k\n perl-Git               noarch       1.8.3.1-6.el7_2.1         base        53 k\n perl-TermReadKey       x86_64       2.30-20.el7               base        31 k\n\nTransaction Summary\n================================================================================\nInstall  1 Package (+4 Dependent packages)\n\nTotal download size: 4.6 M\nInstalled size: 23 M\nDownloading packages:\n--------------------------------------------------------------------------------\nTotal                                              4.2 MB/s | 4.6 MB  00:01     \nRunning transaction check\nRunning transaction test\nTransaction test succeeded\nRunning transaction\n  Installing : 1:perl-Error-0.17020-2.el7.noarch                            1/5 \n  Installing : libgnome-keyring-3.8.0-3.el7.x86_64                          2/5 \n  Installing : perl-TermReadKey-2.30-20.el7.x86_64                          3/5 \n  Installing : git-1.8.3.1-6.el7_2.1.x86_64                                 4/5 \n  Installing : perl-Git-1.8.3.1-6.el7_2.1.noarch                            5/5 \n  Verifying  : perl-Git-1.8.3.1-6.el7_2.1.noarch                            1/5 \n  Verifying  : perl-TermReadKey-2.30-20.el7.x86_64                          2/5 \n  Verifying  : libgnome-keyring-3.8.0-3.el7.x86_64                          3/5 \n  Verifying  : 1:perl-Error-0.17020-2.el7.noarch                            4/5 \n  Verifying  : git-1.8.3.1-6.el7_2.1.x86_64                                 5/5 \n\nInstalled:\n  git.x86_64 0:1.8.3.1-6.el7_2.1                                                \n\nDependency Installed:\n  libgnome-keyring.x86_64 0:3.8.0-3.el7  perl-Error.noarch 1:0.17020-2.el7     \n  perl-Git.noarch 0:1.8.3.1-6.el7_2.1    perl-TermReadKey.x86_64 0:2.30-20.el7 \n\nComplete!\n"
    ]
}

You can see that ansbile installed git on the specified sever.
[ansible@ip-172-31-20-156 ~]$ rpm -qa | grep git
net-tools-2.0-0.17.20131004git.el7.x86_64
git-1.8.3.1-6.el7_2.1.x86_64
python-pillow-2.0.0-19.gitd1c6db8.el7.x86_64
crontabs-1.11-6.20121102git.el7.noarch
linux-firmware-20160830-49.git7534e19.el7.noarch

Now let us uninstall it.

[ansible@ip-172-31-20-156 ~]$ ansible git_server -i hosts -m yum -a "name=git state=absent" -b

172.31.20.156 | SUCCESS => {

    "changed": true,

    "msg": "",

    "rc": 0,

    "results": [

        "Loaded plugins: fastestmirror\nResolving Dependencies\n--> Running transaction check\n---> Package git.x86_64 0:1.8.3.1-6.el7_2.1 will be erased\n--> Processing Dependency: git = 1.8.3.1-6.el7_2.1 for package: perl-Git-1.8.3.1-6.el7_2.1.noarch\n--> Running transaction check\n---> Package perl-Git.noarch 0:1.8.3.1-6.el7_2.1 will be erased\n--> Finished Dependency Resolution\n\nDependencies Resolved\n\n================================================================================\n Package          Arch           Version                    Repository     Size\n================================================================================\nRemoving:\n git              x86_64         1.8.3.1-6.el7_2.1          @base          22 M\nRemoving for dependencies:\n perl-Git         noarch         1.8.3.1-6.el7_2.1          @base          57 k\n\nTransaction Summary\n================================================================================\nRemove  1 Package (+1 Dependent package)\n\nInstalled size: 22 M\nDownloading packages:\nRunning transaction check\nRunning transaction test\nTransaction test succeeded\nRunning transaction\n  Erasing    : git-1.8.3.1-6.el7_2.1.x86_64                                 1/2 \n  Erasing    : perl-Git-1.8.3.1-6.el7_2.1.noarch                            2/2 \n  Verifying  : perl-Git-1.8.3.1-6.el7_2.1.noarch                            1/2 \n  Verifying  : git-1.8.3.1-6.el7_2.1.x86_64                                 2/2 \n\nRemoved:\n  git.x86_64 0:1.8.3.1-6.el7_2.1                                                \n\nDependency Removed:\n  perl-Git.noarch 0:1.8.3.1-6.el7_2.1                                           \n\nComplete!\n"

    ]

}

Now its gone.
[ansible@ip-172-31-20-156 ~]$ rpm -qa | grep git
net-tools-2.0-0.17.20131004git.el7.x86_64
python-pillow-2.0.0-19.gitd1c6db8.el7.x86_64
crontabs-1.11-6.20121102git.el7.noarch
linux-firmware-20160830-49.git7534e19.el7.noarch
[ansible@ip-172-31-20-156 ~]$

Here we will create a playbook with a task of installing git.
[ansible@ip-172-31-20-156 ~]$ cat myinventory
[gitserver]
172.31.20.156

[all:vars]
ansible_user=ansible
ansible_ssh_private_key_file=~/.ssh/id_rsa
[ansible@ip-172-31-20-156 ~]$ cat  git.yaml
- name: Install git
  hosts: gitserver
  remote_user: ansible
  become: yes

  tasks:
  - name: Install git
    yum:  name=git state=present
[ansible@ip-172-31-20-156 ~]$ ansible-playbook -i myinventory git.yaml

PLAY [Install git] *************************************************************

TASK [setup] *******************************************************************
ok: [172.31.20.156]

TASK [Install git] *************************************************************
changed: [172.31.20.156]

PLAY RECAP *********************************************************************
172.31.20.156              : ok=2    changed=1    unreachable=0    failed=0

[ansible@ip-172-31-20-156 ~]$ rpm -qa | grep git
net-tools-2.0-0.17.20131004git.el7.x86_64
python-pillow-2.0.0-19.gitd1c6db8.el7.x86_64
crontabs-1.11-6.20121102git.el7.noarch
linux-firmware-20160830-49.git7534e19.el7.noarch
git-1.8.3.1-6.el7_2.1.x86_64
[ansible@ip-172-31-20-156 ~]$

Now we will create a playbook to install some essential softwares on a server.
[ansible@ip-172-31-20-156 ~]$ cat essentials.yaml
---
- name: Install softwares which are esessential for system administrators
  hosts: localhost
  remote_user: ansible
  become: yes

  tasks:
  - name: Install packages
    yum : name={{item}} state=latest
    with_items:
        - vim
        - wget
        - telnet
        - sysstat
        - htop

Running the playbook:
[ansible@ip-172-31-20-156 ~]$ ansible-playbook -i myinventory essentials.yaml
PLAY [Install softwares which are esessential for system administrators] *******
TASK [setup] *******************************************************************
ok: [172.31.20.156]
TASK [Install packages] ********************************************************
changed: [172.31.20.156] => (item=[u'vim', u'wget', u'telnet', u'sysstat', u'htop'])
PLAY RECAP *********************************************************************
172.31.20.156              : ok=2    changed=1    unreachable=0    failed=0

configuring snmp with ipv6 ip address on linux servers

Advertisements

When we configure snmpd with ipv6 ip address on Linux servers like CentOS or Redhat Enterprises Linux systems we may get an error like : SNMP Error: cannot resolve source hostname. It is because snmp is trying to resolve your ipv6 address as ipv4. To make it work, we need to modify it as below.
com2sec -> com2sec6

To make the snmpd listen on ipv6 address, we need to add the following line to the snmpd.conf file
agentaddress udp6:161
That will make snmpd bind to your ipv6 address.

To set the community string with ipv6 we have to use the parameter com2sec6 instead of com2sec which is used for ipv4.

Let us see the example below.
[root@Hostname ~]# cat /etc/snmp/snmpd.conf
com2sec6 notConfigUser  140b:c010:101:53e1::11 community_string

group   notConfigGroup v1           notConfigUser
group   notConfigGroup v2c         notConfigUser

view    systemview    included   .1.3.6
access  notConfigGroup ""      any       noauth    exact  systemview none none

syslocation Organization_name Project_name
syscontact IT Group <it@organization.com>

master agentx
pass .1.3.6.1.4.1.4413.4.1 /usr/bin/ucd5820stat

smuxpeer .1.3.6.1.4.1.674.10892.1
dontLogTCPWrappersConnects 1

interface bond0 6 2000000000
interface bond1 6 2000000000

agentaddress udp6:161

Then restart snmpd. If its centos7 or rhel7
[root@Hostname ~]# systemctl restart snmpd

Creating selinux policies for icinga nrpe on redhat/centos Linux

Advertisements

We all know how to configure nrpe in a system and configure remote checks with nrpe. We do this with lightning speed with SELINUX=disabled. But when some systems will have selinux enabled. Then how we will configure nrpe on it? Simple. We will disable selinux and configure nrpe.

Well, that is not recommended. We need to keep the selinux enforcing and configure nrpe with selinux. it was very difficult in old days. But not now. We will see how to install and configure nrpe with selinux.

How to upload and download files with https sftp and sftp protocols using python

Advertisements

We all use file servers. It may be ftp or http. to secure http communication we use https protocol. In case of ftp, there are two options. ftps and sftp.

FTP:
First we will see how the normal ftp works. FTP is one of the mostly used tool for file transfer. FTP user two channels for data transfer. Command channel and data channel. Ftp was created in beginning of the internet era when internet security was not a big issue. So they didn't encrypt the data transfer in these two channels. In effect all the user credentials and data transferred via ftp is clear. A usual man in the middle attack can see everything transferred via these channels.

As security became a big concern, techies created ways to make the data transfer secure using encryption.

FTPS:
FTPS is just like https. FTP + SSL(Secure socket layer). FTPS uses a user id and password, a certificate, or both. When connecting to a FTPS server, ftps client will first check if the server's certificate is trusted. The certificate is considered trusted if either the certificate was signed by a known certificate authority (CA) or if the certificate was self-signed and you have a copy of public certificate in your trusted key store. User id authentication can be used with any combination of certificate and/or password authentication.

In effect, FTPS just added another layer for security using ssl. It still uses two ports for commands and data transfer just like normal ftp.

SFTP:
SFTP works in another way. SFTP is based on SSH(secure shell) protocol. Unlike ftp and ftps which uses two ports, SFTP uses single ports and encrypts the transfer of commands and data through it. Just like normal ftp, in SFTP we can use just a user id and password. In SFTP, we can also use ssh keys for authentication.

TypeError: 'str' does not support the buffer interface

Advertisements

I got this error while writing a script in python 3.4. On centos linux 7

The code was like this

  # Create temp file
  tempfile.tempdir = '/tmp'
  temp, temp_name = tempfile.mkstemp()
  if verbose:
    print(temp_name)
  os.write(temp,'Temporary file created by check_file_operations.py script for testing upload & download.\n')
  os.close(temp)

While running it raised the following error.

TypeError: 'str' does not support the buffer interface

it was because of the buffer text. python 3, the socket accepts bytes, you need to convert string to bytes with a encode() function like this:

  # Create temp file
  tempfile.tempdir = '/tmp'
  temp, temp_name = tempfile.mkstemp()
  if verbose:
    print(temp_name)
  os.write(temp,'Temporary file created by check_file_operations.py script for testing upload & download.\n'.encode())
  os.close(temp)