Learn Puppet with Me – Day 3

Estimated Reading Time: 11 minutes

Puppet is an open source framework and toolset for managing the configuration of computer systems. Puppet can be used to manage configuration on UNIX (including OSX) and Linux platforms, and recently Microsoft Windows platforms as well. Puppet is often used to manage a host throughout its lifecycle: from initial build and installation, to upgrades, maintenance, and finally to end-of-life, whenyou move services elsewhere. Puppet is designed to continuously interact with your hosts, unlike provisioning tools which build your hosts and leave them unmanaged.

Puppet has a simple operating model that is easy to understand and implement. The model is made up of three components:-

• Deployment

• Configuration Language and Resource Abstraction Layer

• Transactional Layer

Puppet is usually deployed in a simple client-server model (Figure 1-2). The server is called a “Puppet master”, the Puppet client software is called an agent and the host itself is defined as a node.

The Puppet master runs as a daemon on a host and contains the configuration required for your environment. The Puppet agents connect to the Puppet master via an encrypted and authenticated connection using standard SSL, and retrieve or “pull” any configuration to be applied.

Importantly, if the Puppet agent has no configuration available or already has the required configuration then Puppet will do nothing. This means that Puppet will only make changes to your environment if they are required. The whole process is called a configuration run.

Each agent can run Puppet as a daemon via a mechanism such as cron, or the connection can be manually triggered. The usual practice is to run Puppet as a daemon and have it periodically check with the master to confirm that its configuration is up-to-date or to retrieve any new configuration. However,many people find being able to trigger Puppet via a mechanism such as cron, or manually, better suits their needs. By default, the Puppet agent will check the master for new or changed configuration once every 30 minutes. You can configure this period to suit your environment.

Configuration Language and Resource Abstraction Layer

Puppet uses a declarative language to define your configuration items, which Puppet calls “resources.” This declarative nature creates an important distinction between Puppet and many other configuration tools. A declarative language makes statements about the state of your configuration – for example, it declares that a package should be installed or a service should be started.

Most configuration tools, such as a shell or Perl script, are imperative or procedural. They describe HOW things should be done rather than the desired end state – for example, most custom scripts used to manage configuration would be considered imperative. This means Puppet users just declare what the state of their hosts should be: what packages should be installed, what services should be running, etc. With Puppet, the system administrator doesn’t care HOW this state is achieved – that’s Puppet’s problem. Instead, we abstract our host’s configuration into resources.

Configuration Language

What does this declarative language mean in real terms? Let’s look at a simple example. We have an environment with Red Hat Enterprise Linux, Ubuntu, and Solaris hosts and we want to install the vim application on all our hosts. To do this manually, we’d need to write a script that does the following:

• Connects to the required hosts (including handling passwords or keys)

• Checks to see if vim is installed

• If not, uses the appropriate command for each platform to install vim, for example on Red Hat the yum command and on Ubuntu the apt-get command

• Potentially reports the results of this action to ensure completion and success

Puppet approaches this process quite differently. In Puppet, we define a configuration resource for the vim package. Each resource is made up of a type (what sort of resource is being managed: packages, services, or cron jobs), a title (the name of the resource), and a series of attributes (values that specify the state of the resource – for example, whether a service is started or stopped).

Example:

A Puppet Resource

package { “vim”:

ensure => present,

}

Resource Abstraction Layer

With our resource created, Puppet takes care of the details of how to manage that resource when our agents connect. Puppet handles the “how” by knowing how different platforms and operating systems manage certain types of resources. Each type has a number of “providers.” A provider contains the “how” of managing packages using a particular package management tool. For the package type, forexample, for there are more than 20 providers covering a variety of tools including yum, aptitude, pkgadd, ports, and emerge.

When an agent connects, Puppet uses a tool called “Facter” to return information about that agent, including what operating system it is running. Puppet then chooses the appropriate package provider for that operating system and uses that provider to check if the vimpackage is installed. For example, on Red Hat it would execute yum, on Ubuntu it would execute aptitude, and on Solaris it would use the pkg command. If the package is not installed, then Puppet will install it. If the package is already installed, Puppet does nothing. Puppet will then report its success or failure in applying the resource back to the Puppet master.

INTRODUCING FACTER AND FACTS

Facter is a system inventory tool which returns “facts” about each agent, such as its hostname, IP address, operating system and version, and other configuration items. These facts are gathered when the agent runs. The facts are then sent to the Puppet master, and automatically created as variables available to Puppet.You can see the facts available on your clients by running the facter binary from the command line. Each fact is returned as a key => value pair. For example:

operatingsystem => Ubuntu

ipaddress => 10.0.0.10

Transactional Layer

Puppet’s transactional layer is its engine. A Puppet transaction encompasses the process of configuring each host including:

• Interpret and compile your configuration

• Communicate the compiled configuration to the agent

• Apply the configuration on the agent

• Report the results of that application to the master

The first step Puppet takes is to analyze your configuration and calculate how to apply it to your agent. To do this, Puppet creates a graph showing all resources, their relationships to each other and to each agent. This allows Puppet to work out in what order, based on relationships you create, to apply each resource to your host. This model is one of Puppet’s most powerful features. Puppet then takes the resources and compiles them into a “catalog” for each agent. The catalog is sent to the host and applied by the Puppet agent. The results of this application are then sent back to the master in the form of a report.

The transaction layer allows configurations to be created and applied repeatedly on the host.Puppet calls this idempotent, meaning multiple applications of the same operation will yield the same results. Puppet configuration can be safely run multiple times with the same outcome on your host and hence ensuring your configuration stays consistent.

3. Understanding Puppet Components

If you look at /etc/puppet directory , you will find various components of puppet underlying.

[root@puppet-server puppet]# ls -la

total 28

drwxr-xr-x. 4 puppet puppet 4096 Aug 23 11:41 .

drwxr-xr-x. 79 root root 4096 Aug 25 04:01 ..

-rwxr-xr-x. 1 puppet puppet 2552 Aug 21 17:49 auth.conf

-rwxr-xr-x. 1 puppet puppet 381 Jun 20 18:24 fileserver.conf

drwxr-xr-x. 4 puppet puppet 4096 Aug 25 06:50 manifests

drwxr-xr-x. 11 puppet puppet 4096 Aug 25 06:49 modules

-rwxr-xr-x. 1 puppet puppet 1059 Aug 23 11:41 puppet.conf

Let’s talk about what is manifest.

“Manifest” is Puppet’s term for files containing configuration information. Manifest files have a suffix of .pp. This directory and file is often already created when the Puppet packages are installed. If it hasn’t already been created, then create this directory and file now:

# mkdir /etc/puppet/manifests

Under manifests, there are important files such as nodes.pp, site.pp, template.pp and few classes and definitions. We are going to cover those too here.

Puppet manifests are made up of a number of major components:

• Resources – Individual configuration items

• Files – Physical files you can serve out to your agents

• Templates – Template files that you can use to populate files

• Nodes – Specifies the configuration of each agent

• Classes – Collections of resources

• Definitions – Composite collections of resources

These components are wrapped in a configuration language that includes variables, conditionals, arrays and other features. Later in this chapter we’ll introduce you to the basics of the Puppet language and its elements. In the next chapter, we’ll extend your knowledge of the language by taking you through an implementation of a multi-agent site managed with Puppet.

The site.pp file

The site.pp file tells Puppet where and what configuration to load for our clients. We’re going to store

this file in a directory called manifests under the /etc/puppet directory.

Please Note: Puppet will not start without the site.pp file being present.

Our first step in creating our first agent configuration is defining and extending the site.ppfile. See an example of this file in Listing 1-3.

The site.pp File

import ‘nodes.pp’

$puppetserver = ‘puppet.example.com’

The import directive tells Puppet to load a file called nodes.pp. This directive is used to include any Puppet configuration we want to load.

When Puppet starts, it will now load the nodes.pp file and process the contents. In this case, this file will contain the node definitions we create for each agent we connect. You can also import multiple files like so:

import ‘nodes/*’

import ‘classes/*’

The import statement will load all files with a suffix of .pp in the directories nodes andclasses.

The $puppetserver statement sets a variable. In Puppet, configuration statements starting with a dollar sign are variables used to specify values that you can use in Puppet configuration.

In Listing 1-3, we’ve created a variable that contains the fully qualified domain name of our Puppet master, enclosed in double quotes.

Agent Configuration

Let’s add our first agent definition to the nodes.pp file we’ve just asked Puppet to import. In Puppet manifests, agents are defined using node statements.

# touch /etc/puppet/manifests/nodes.pp.

You can see the node definition we’re going to add in Listing 1-4.

Listing 1-4. Our Node Configuration

node ‘puppet-client.test.com’ {

include sudo

}

Next, we specify an include directive in our node definition. The include directive specifies a collection of configuration that we want to apply to our host. There are two types of collections we can

include in a node:

• Classes – a basic collection of resources

• Modules – an advanced, portable collection of resources that can include classes, definitions, and other supporting configuration.You can include multiple collections by using multipleinclude directives or separating each

collection with commas.

include sudo

include sshd

include vim, syslog-ng

Creating our first module

The next step is to create the sudo module. A module is a collection of manifests, resources, files, templates, classes, and definitions. A single module would contain everything required to configure a particular application. For example, it could contain all the resources (specified in manifest files), files and associated configuration to configure Apache or the sudo command on a host.

Each module needs a specific directory structure and a file called init.pp. This structure allows Puppet to automatically load modules. To perform this automatic loading, Puppet checks a series of directories called the module path. The module path is configured with themodulepath configuration

option in the [main] section of the puppet.conf file. By default, Puppet looks for modules in the /etc/puppet/modules and /var/lib/puppet/modules directories, but you can add additional locations if

required:

[main]

moduledir = /etc/puppet/modules:/var/lib/puppet/modules:/opt/modules

The automatic loading of modules means, unlike our nodes.pp file, modules don’t need to be loaded into Puppet using the import directive.

Module Structure

Let’s start by creating a module directory and file structure in Listing 1-5. We’re going to create this structure under the directory /etc/puppet/modules. We will name the modulesudo. Modules (and classes) must be normal words containing only letters, numbers, underscores and dashes.

Listing 1-5. Module Structure

# mkdir –p /etc/puppet/modules/sudo/{files,templates,manifests}

# touch /etc/puppet/modules/sudo/manifests/init.pp

The manifests directory will hold our init.pp file and any other configuration. The init.pp file is the core of your module and every module must have one. The files directory will hold any files we wish to serve as part of our module. The templates directory will contain any templates that our module might use.

The init.pp file

Now let’s look inside our sudo module, starting with the init.pp file, which we can see in Listing 1-6.

Listing 1-6. The sudo module’s init.pp file

class sudo {

package { sudo:

ensure => present,

}

if $operatingsystem == “Ubuntu” {

package { “sudo-ldap”:

ensure => present,

require => Package[“sudo”],

}

}

file { “/etc/sudoers”:

owner => “root”,

group => “root”,

mode => 0440,

source => “puppet://$puppetserver/modules/sudo/etc/sudoers”,

require => Package[“sudo”],

}

}

Our sudo module’s init.pp file contains a single class, also called sudo. There are three resources in the class, two packages and a file resource.The first package resource ensures that the sudo package is installed, ensure => present. The second package resource uses Puppet’s if/else syntax to set a condition on the installation of the sudoldap package.

The next portion of our source value tells Puppet where to look for the file. This is the equivalent of the path to a network file share. The first portion of this share is modules,which tells us that the file is stored in a module. Next we specify the name of the module the file is contained in, in this case sudo.

Finally, we specify the path inside that module to find the file.

All files in modules are stored under the files directory; this is considered the “root” of the module’s file “share.” In our case, we would create the directory etc under the files directory and create the sudoers file in this directory.

puppet$ mkdir –p /etc/puppet/modules/sudo/files/etc

puppet$ cp /etc/sudoers /etc/puppet/manifests/files/etc/sudoers

Applying Our First Configuration

We’ve created our first Puppet module! Let’s step through what will happen when we connect an agent that includes this module.

1. It will install the sudo package.

2. If it’s an Ubuntu host, then it will also install the sudo-ldap package

3. Lastly, it will download the sudoers file and install it into /etc/sudoers.

Now let’s see this in action and include our new module on the agent we’ve created, Nod1.example.com.

Remember we created a node statement for our host in Listing 1.4:

node ‘node1.example.com’ {

include sudo

}

When the agent connects it will now include the sudo module. To do this we run the Puppet agent again, as shown in Listing 1-7.

Listing 1-7. Applying Our First Configuration

puppet# puppet agent –server=puppet.example.com –no-daemonize –verbose –onetime

notice: Starting Puppet client version 2.6.1

info: Caching catalog for node1.example.com

info: Applying configuration version ‘1272631279’

notice: //sudo/Package[sudo]/ensure: created

notice: //sudo/File[/etc/sudoers]/checksum: checksum changed

‘{md5}9f95a522f5265b7e7945ff65369acdd2’ to ‘{md5}d657d8d55ecdf88a2d11da73ac5662a4’

info: Filebucket[/var/lib/puppet/clientbucket]: Adding

/etc/sudoers(d657d8d55ecdf88a2d11da73ac5662a4)

info: //sudo/File[/etc/sudoers]: Filebucketed /etc/sudoers to puppet with sum

d657d8d55ecdf88a2d11da73ac5662a4

notice: //sudo/File[/etc/sudoers]/content: content changed

‘{md5}d657d8d55ecdf88a2d11da73ac5662a4’

In our next section, we are going to talk more on Puppet Modules.

How to install Nagios on Linux?

Estimated Reading Time: 7 minutes

Last week I thought of setting up Nagios on my Linux Box.I installed a fresh piece of RHEL on my Virtualbox and everything went fine. I thought of putting this complete setup on my blog and here it is : “A Complete Monitoring Tool for your Linux Box”

nagios

Here is my Machine Configuration:

[root@irc ~]# cat /etc/redhat-release

Red Hat Enterprise Linux Server release 5.3 (Tikanga)

[root@irc ~]#

[root@irc ~]# uname -arn

Linux irc.chatserver.com 2.6.18-128.el5 #1 SMP Wed Dec 17 11:41:38 EST 2008 x86_64 x86_64 x86_64 GNU/Linux

[root@irc ~]#

1) Create Account Information

Become the root user.

su -l

Create a new nagios user account and give it a password.

/usr/sbin/useradd -m nagios

passwd nagios

Create a new nagcmd group for allowing external commands to be submitted through the web interface. Add both the nagios user and the apache user to the group.

/usr/sbin/groupadd nagcmd

/usr/sbin/usermod -a -G nagcmd nagios

/usr/sbin/usermod -a -G nagcmd apache

2) Download Nagios and the Plugins

Create a directory for storing the downloads.

mkdir ~/downloads

cd ~/downloads

Download the source code tarballs of both Nagios and the Nagios plugins (visit http://www.nagios.org/download/ for links to the latest versions). These directions were tested with Nagios 3.1.1 and Nagios Plugins 1.4.11.

wget http://prdownloads.sourceforge.net/sourceforge/nagios/nagios-3.2.0.tar.gz

wget http://prdownloads.sourceforge.net/sourceforge/nagiosplug/nagios-plugins-1.4.11.tar.gz

3) Compile and Install Nagios

Extract the Nagios source code tarball.

cd ~/downloads

tar xzf nagios-3.2.0.tar.gz

cd nagios-3.2.0

Run the Nagios configure script, passing the name of the group you created earlier like so:

./configure –with-command-group=nagcmd

Compile the Nagios source code.

make all

Install binaries, init script, sample config files and set permissions on the external command directory.

make install

make install-init

make install-config

make install-commandmode

Don’t start Nagios yet – there’s still more that needs to be done…

4) Customize Configuration

Sample configuration files have now been installed in the /usr/local/nagios/etc directory. These sample files should work fine for getting started with Nagios. You’ll need to make just one change before you proceed…

Edit the /usr/local/nagios/etc/objects/contacts.cfg config file with your favorite editor and change the email address associated with the nagiosadmin contact definition to the address you’d like to use for receiving alerts.

vi /usr/local/nagios/etc/objects/contacts.cfg

5) Configure the Web Interface

Install the Nagios web config file in the Apache conf.d directory.

make install-webconf

Create a nagiosadmin account for logging into the Nagios web interface. Remember the password you assign to this account – you’ll need it later.

htpasswd -c /usr/local/nagios/etc/htpasswd.users nagiosadmin

Restart Apache to make the new settings take effect.

service httpd restart

Note: Consider implementing the ehanced CGI security measures described here to ensure that your web authentication credentials are not compromised.

6) Compile and Install the Nagios Plugins

Extract the Nagios plugins source code tarball.

cd ~/downloads

tar xzf nagios-plugins-1.4.11.tar.gz

cd nagios-plugins-1.4.11

Compile and install the plugins.

./configure –with-nagios-user=nagios –with-nagios-group=nagios

make

make install

7) Start Nagios

Add Nagios to the list of system services and have it automatically start when the system boots.

chkconfig –add nagios

chkconfig nagios on

Verify the sample Nagios configuration files.

/usr/local/nagios/bin/nagios -v /usr/local/nagios/etc/nagios.cfg

If there are no errors, start Nagios.

service nagios start

8) Modify SELinux Settings

Fedora ships with SELinux (Security Enhanced Linux) installed and in Enforcing mode by default. This can result in “Internal Server Error” messages when you attempt to access the Nagios CGIs.

See if SELinux is in Enforcing mode.

getenforce

Put SELinux into Permissive mode.

setenforce 0

To make this change permanent, you’ll have to modify the settings in /etc/selinux/config and reboot.

Instead of disabling SELinux or setting it to permissive mode, you can use the following command to run the CGIs under SELinux enforcing/targeted mode:

chcon -R -t httpd_sys_content_t /usr/local/nagios/sbin/

chcon -R -t httpd_sys_content_t /usr/local/nagios/share/

For information on running the Nagios CGIs under Enforcing mode with a targeted policy, visit the Nagios Support Portal or Nagios Community Wiki.

9) Login to the Web Interface

You should now be able to access the Nagios web interface at the URL below. You’ll be prompted for the username (nagiosadmin) and password you specified earlier.

http://localhost/nagios/

Click on the “Service Detail” navbar link to see details of what’s being monitored on your local machine. It will take a few minutes for Nagios to check all the services associated with your machine, as the checks are spread out over time.

10) Other Modifications

Make sure your machine’s firewall rules are configured to allow access to the web server if you want to access the Nagios interface remotely.

Configuring email notifications is out of the scope of this documentation. While Nagios is currently configured to send you email notifications, your system may not yet have a mail program properly installed or configured. Refer to your system documentation, search the web, or look to the Nagios Support Portal or Nagios Community Wiki for specific instructions on configuring your system to send email messages to external addresses. More information on notifications can be found here.

11) You’re Done

Congratulations! You sucessfully installed Nagios. Your journey into monitoring is just beginning.

Example:

Say, If You Nagios Server is 10.14.236.140. You need to monitor the Linux Machine with IP: 10.14.236.70. You need to follow up like this:

[root@irc objects]# pwd

/usr/local/nagios/etc/objects

[root@irc objects]#

[root@irc objects]# ls

commands.cfg localhost.cfg printer.cfg switch.cfg timeperiods.cfg

contacts.cfg localhost.cfg.orig remotehost.cfg templates.cfg windows.cfg

[root@irc objects]#

The File should looks like:

# HOST DEFINITION

#

###############################################################################

###############################################################################

# Define a host for the local machine

define host{

use linux-server ; Name of host template to use

; This host definition will inherit all variab les that are defined

; in (or inherited by) the linux-server host t emplate definition.

host_name localhost

alias localhost

address 127.0.0.1

}

define host{

use linux-server ; Name of host template to use

; This host definition will inherit all variab les that are defined

; in (or inherited by) the linux-server host t emplate definition.

host_name ideath.logic.com

alias ideath

address 10.14.236.140

}

###############################################################################

###############################################################################

#

# HOST GROUP DEFINITION

#

###############################################################################

###############################################################################

# Define an optional hostgroup for Linux machines

define hostgroup{

hostgroup_name linux-server ; The name of the hostgroup

alias Linux Servers ; Long name of the group

members localhost ; Comma separated list of hosts that belong to this group

}

###############################################################################

###############################################################################

#

# SERVICE DEFINITIONS

#

###############################################################################

###############################################################################

# Define a service to “ping” the local machine

define service{

use local-service ; Name of service template to use

host_name localhost

service_description PING

check_command check_ping!100.0,20%!500.0,60%

}

define service{

use local-service ; Name of service template to use

host_name ideath.logica.com

service_description PING

check_command check_ping!100.0,20%!500.0,60%

}

# Define a service to check the disk space of the root partition

# on the local machine. Warning if < 20% free, critical if # < 10% free space on partition. define service{ use local-service ; Name of service template to use host_name localhost service_description Root Partition check_command check_local_disk!20%!10%!/ } define service{ use local-service ; Name of service template to use host_name ideath.logic.com service_description Root Partition check_command check_local_disk!20%!10%!/ } # Define a service to check the number of currently logged in # users on the local machine. Warning if > 20 users, critical

# if > 50 users.

define service{

use local-service ; Name of service template to use

host_name localhost

service_description Current Users

check_command check_local_users!20!50

}

define service{

use local-service ; Name of service template to use

host_name ideath.logic.com

service_description Current Users

check_command check_local_users!20!50

}

# Define a service to check the number of currently running procs

# on the local machine. Warning if > 250 processes, critical if

# > 400 users.

define service{

use local-service ; Name of service template to use

host_name localhost

service_description Total Processes

check_command check_local_procs!250!400!RSZDT

}

define service{

use local-service ; Name of service template to use

host_name ideath.logic.com

service_description Total Processes

check_command check_local_procs!250!400!RSZDT

}

# Define a service to check the load on the local machine.

define service{

use local-service ; Name of service template to use

host_name localhost

service_description Current Load

check_command check_local_load!5.0,4.0,3.0!10.0,6.0,4.0

}

define service{

use local-service ; Name of service template to use

host_name ideath.logic.com

service_description Current Load

check_command check_local_load!5.0,4.0,3.0!10.0,6.0,4.0

}

# Define a service to check the swap usage the local machine.

# Critical if less than 10% of swap is free, warning if less than 20% is free

define service{

use local-service ; Name of service template to use

host_name localhost

service_description Swap Usage

check_command check_local_swap!20!10

}

define service{

use local-service ; Name of service template to use

host_name ideath.logic.com

service_description Swap Usage

check_command check_local_swap!20!10

}

# Define a service to check SSH on the local machine.

# Disable notifications for this service by default, as not all users may have SSH enabled.

define service{

use local-service ; Name of service template to use

host_name localhost

service_description SSH

check_command check_ssh

notifications_enabled 0

}

define service{

use local-service ; Name of service template to use

host_name ideath.logic.com

service_description SSH

check_command check_ssh

check_period 24×7

notifications_enabled 0

is_volatile 0

max_check_attempts 4

normal_check_interval 5

retry_check_interval 1

contact_groups admins

notification_options w,c,u,r

notification_interval 960

notification_period 24×7

check_command check_ssh

}

# Define a service to check HTTP on the local machine.

# Disable notifications for this service by default, as not all users may have HTTP enabled.

define service{

use local-service ; Name of service template to use

host_name localhost

service_description HTTP

check_command check_http

notifications_enabled 0

}

define service{

use local-service ; Name of service template to use

host_name ideath.logic.com

service_description HTTP

check_command check_http

notifications_enabled 0

is_volatile 0

max_check_attempts 4

normal_check_interval 5

retry_check_interval 1

contact_groups admins

notification_options w,c,u,r

notification_interval 960

notification_period 24×7

check_command check_http

}

Ideath.logic.com is the hostname of 10.14.236.70.

Do make entry in /etc/hosts if it is unable to resolve the IP(or else check the DNS).

How to configure Directory Indexing in Apache?

Estimated Reading Time: 1 minute

While attending RHCE examination, I faced a lot of question related to Apache. While installation and configuration of Apache was the first topic, I found this topic very useful and would like to share with everyone who is going to attend RHCE certification.

apache318x260

A Quick one line step to configure it is:

Edit the /etc/httpd/conf/httpd.conf file :

Just Look at the line starting:

[Please note: Do add lesser than sign in front of directory]
directory “/var/www/html/pdfs”

Options Indexes FollowSymLinks
AllowOverride None
Order allow,deny
Allow from all

/Directory

Restart the Apache.
Try browsing http://localhost/pdfs