How to configure Directory Indexing in Apache?

Estimated Reading Time: 1 minute

While attending RHCE examination, I faced a lot of question related to Apache. While installation and configuration of Apache was the first topic, I found this topic very useful and would like to share with everyone who is going to attend RHCE certification.

apache318x260

A Quick one line step to configure it is:

Edit the /etc/httpd/conf/httpd.conf file :

Just Look at the line starting:

[Please note: Do add lesser than sign in front of directory]
directory “/var/www/html/pdfs”

Options Indexes FollowSymLinks
AllowOverride None
Order allow,deny
Allow from all

/Directory

Restart the Apache.
Try browsing http://localhost/pdfs

Installing Open-Xchange on Ubuntu 12.04

Estimated Reading Time: 6 minutes

I thought of starting my day today with Open-Xchange. I had Vmware Workstation installed on one of Windows 7 Enterprise machine. I installed a minimal Ubuntu 12.04 as VM and ready to install. Here it goes:

ox

1. Pre-Requisite:

Installed Ubuntu 12.04

With apt-get utility working.(Internet Connectivity)

Ensure you have a FQDN name under /etc/hosts

iRedMail(Mail Server) software downloaded from http://iredmail.org/download.html

2. Edit the /etc/apt/sources.list and add the following entry:

deb http://download.opensuse.org/repositories/server:/OX:/ox6/xUbuntu_12.04/ /

3. Import the key:

$sudo wget http://software.open-xchange.com/oxbuildkey.pub-O – | apt-key add –

Ensure it showing as “Ok”.

4. Update the machine:

sudo apt-get update

4. Lets install iRedMail server

$ apt-Get install bzip2

$ cd / root

$ mkdir install

$ cd /root/install

$ wget https://bitbucket.org/zhb/iredmail/downloads/iRedMail-0.8.4.tar.bz2

$ tar xjf iRedMail-0.8.2.tar.bz2

$ cd / root/install/iRedMail-0.8.2 /

$ bash iRedMail.sh

It will finish as shown below:

********************************************************************

* Start iRedMail Configurations

********************************************************************

< INFO > Create self-signed SSL certification files.

< INFO > Create required system accounts: vmail, iredapd, iredadmin.

< INFO > Configure Apache web server and PHP.

< INFO > Configure MySQL database server.

mysqladmin: connect to server at ‘localhost’ failed

error: ‘Access denied for user ‘root’@’localhost’ (using password: NO)’

< INFO > Configure Postfix (Message Transfer Agent).

< INFO > Configure Policyd (postfix policy server, code name cluebringer).

< INFO > Configure Dovecot (pop3/imap/managesieve server, version 2).

< INFO > Configure ClamAV (anti-virus toolkit).

< INFO > Configure Amavisd-new (interface between MTA and content checkers).

drop_priv: No such username:

< INFO > Configure SpamAssassin (content-based spam filter).

< INFO > Configure iRedAPD (postfix policy daemon).

< INFO > Configure iRedAdmin (official web-based admin panel).

< INFO > Configure Fail2ban (authentication failure monitor).

< INFO > Configure Awstats (logfile analyzer for mail and web server).

< INFO > Configure Roundcube webmail.

< INFO > Configure phpMyAdmin (web-based MySQL management tool).

*************************************************************************

* iRedMail-0.8.4 installation and configuration complete.

*************************************************************************

< Question > Would you like to *REMOVE* sendmail now? [Y|n]Y

< INFO > Removing package(s): sendmail

Reading package lists… Done

Building dependency tree

Reading state information… Done

Package sendmail is not installed, so not removed

0 upgraded, 0 newly installed, 0 to remove and 107 not upgraded.

< Question > Would you like to use firewall rules provided by iRedMail now?

< Question > File: /etc/default/iptables, with SSHD port: 22. [Y|n]n

< INFO > Skip firewall rules.

< INFO > Deliver administration emails to postmaster@ubuntu.mail.com.

< INFO > Updating ClamAV database (freshclam), please wait …

ClamAV update process started at Fri Jun 21 08:18:21 2013

WARNING: DNS record is older than 3 hours.

WARNING: Invalid DNS reply. Falling back to HTTP mode.

Downloading main.cvd [100%]

main.cvd updated (version: 54, sigs: 1044387, f-level: 60, builder: sven)

Reading CVD header (daily.cvd): OK (IMS)

daily.cvd is up to date (version: 17389, sigs: 1361238, f-level: 63, builder: guitar)

Reading CVD header (bytecode.cvd): OK (IMS)

bytecode.cvd is up to date (version: 214, sigs: 41, f-level: 63, builder: neo)

Database updated (2405666 signatures) from db.local.clamav.net (IP: 203.178.137.175)

********************************************************************

* URLs of installed web applications:

*

* – Webmail: httpS://ubuntu.localdomain/mail/

* – Admin Panel (iRedAdmin): httpS://ubuntu.localdomain/iredadmin/

* + Username: postmaster@ubuntu.mail.com, Password: ajeetraina@ubuntu.mail.com

*

********************************************************************

* Congratulations, mail server setup completed successfully. Please

* read below file for more information:

*

* – /root/install/iRedMail-0.8.4/iRedMail.tips

*

* And it’s sent to your mail account postmaster@ubuntu.mail.com.

*

* Please reboot your system to enable mail services.

6. Now install Open-Xchange server through the below command:

aptitude install \

open-xchange open-xchange-authentication-database \

open-xchange-admin-client open-xchange-admin-lib \

open-xchange-admin-plugin-hosting open-xchange-admin-plugin-hosting-client \

open-xchange-admin-plugin-hosting-lib open-xchange-configjump-generic \

open-xchange-admin-doc open-xchange-contactcollector \

open-xchange-conversion open-xchange-conversion-engine \

open-xchange-conversion-servlet open-xchange-crypto \

open-xchange-data-conversion-ical4j open-xchange-dataretention \

open-xchange-genconf open-xchange-genconf-mysql \

open-xchange-imap open-xchange-mailfilter \

open-xchange-management open-xchange-monitoring \

open-xchange-passwordchange-database open-xchange-passwordchange-servlet \

open-xchange-pop3 open-xchange-publish open-xchange-publish-basic \

open-xchange-publish-infostore-online open-xchange-publish-json \

open-xchange-publish-microformats open-xchange-push-udp \

open-xchange-resource-managerequest open-xchange-server \

open-xchange-settings-extensions open-xchange-smtp \

open-xchange-spamhandler-default open-xchange-sql open-xchange-subscribe \

open-xchange-xerces-sun open-xchange-subscribe-json \

open-xchange-subscribe-microformats open-xchange-subscribe-crawler \

open-xchange-templating open-xchange-threadpool open-xchange-unifiedinbox \

open-xchange-admin-plugin-hosting-doc open-xchange-charset \

open-xchange-group-managerequest open-xchange-i18n open-xchange-jcharset \

open-xchange-sessiond open-xchange-calendar-printing \

open-xchange-user-json open-xchange-gui-wizard-plugin \

open-xchange-report-client \

open-xchange-configjump-generic-gui \

open-xchange-gui open-xchange-gui-wizard-plugin-gui \

open-xchange-online-help-de \

open-xchange-online-help-en open-xchange-online-help-fr open-xchange-gui-lang-community-ru-ru \

9. Run this command:

$ /etc/init.d/mysql restart

10.

echo PATH=$PATH:/opt/open-xchange/sbin/ >> ~/.bashrc &&. ~/.bashrc

11.

echo “GRANT ALL PRIVILEGES ON *.* TO ‘openexchange’@’localhost’ IDENTIFIED BY ‘open_password’;” > /tmp/openXchange_pri.sql

12. $ mysql -u root < /tmp/openXchange_pri.sql mysql -p

13.

$ /opt/open-xchange/sbin/initconfigdb –configdb-pass=open_password

14.

$/opt/open-xchange/sbin/initconfigdb –configdb-pass=open_password

15.

$ /opt/open-xchange/sbin/oxinstaller –no-license –servername=oxserver \

–configdb-pass=open_password –master-pass=open_master_password –ajp-bind-port=localhost –servermemory 1024

$ /opt/open-xchange/sbin/registerserver -n oxserver -A oxadminmaster -P mysql123

16.

mkdir /var/opt/filestore

chown open-xchange:open-xchange /var/opt/filestore

17.

/opt/open-xchange/sbin/registerfilestore -A oxadminmaster -P mysql123 \

-t file:/var/opt/filestore -s 1000000

18.

/opt/open-xchange/sbin/registerdatabase -A oxadminmaster -P mysql123 \

-n oxdatabase -p mysql123 -m true

19.

$ a2enmod proxy proxy_ajp proxy_balancer expires deflate headers rewrite mime setenvif

20.

$ /etc/init.d/apache2 force-reload

21.

mcedit /etc/apache2/conf.d/proxy_ajp.conf

22.

$ vim /etc/apache2/sites-available/default <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/ <Directory /var/www/> AllowOverride None Order allow,deny allow from all RedirectMatch ^/$ /ox6/ Options +FollowSymLinks +SymLinksIfOwnerMatch </Directory> # deflate AddOutputFilterByType DEFLATE text/html text/plain text/javascript application/javascript text/css text/xml application/xml text/x-js application/x-javascript # pre-compressed files AddType text/javascript .jsz AddType text/css .cssz AddType text/xml .xmlz AddType text/plain .po AddEncoding gzip .jsz .cssz .xmlz SetEnvIf Request_URI “\.(jsz|cssz|xmlz)$” no-gzip ExpiresActive On <Location /ox6> # Expires (via ExpiresByType to override global settings) ExpiresByType image/gif “access plus 6 months” ExpiresByType image/png “access plus 6 months” ExpiresByType image/jpg “access plus 6 months” ExpiresByType image/jpeg “access plus 6 months” ExpiresByType text/css “access plus 6 months” ExpiresByType text/html “access plus 6 months” ExpiresByType text/xml “access plus 6 months” ExpiresByType text/javascript “access plus 6 months” ExpiresByType text/x-js “access plus 6 months” ExpiresByType application/x-javascript “access plus 6 months” ExpiresDefault “access plus 6 months” Header append Cache-Control “private” Header unset Last-Modified Header unset Vary # Strip version RewriteEngine On RewriteRule v=\w+/(.+) $1 [L] # Turn off ETag Header unset ETag FileETag None </Location> <Location /ox6/ox.html> ExpiresByType text/html “now” ExpiresDefault “now” Header unset Last-Modified Header set Cache-Control “no-store, no-cache, must-revalidate, post-check=0, pre-check=0” # Turn off ETag Header unset ETag FileETag None </Location> <Location /ox6/index.html> ExpiresByType text/html “now” ExpiresDefault “now” Header unset Last-Modified Header set Cache-Control “no-store, no-cache, must-revalidate, post-check=0, pre-check=0” # Turn off ETag Header unset ETag FileETag None </Location> </VirtualHost>22.$ sudo /etc/init.d/apache2 restart
23.
sudo /etc/init.d/open-xchange-groupware start
24.$ /opt/open-xchange/sbin/createcontext -A oxadminmaster -P open_master_password -c 1 \-u oxadmin -d “Context Admin” -g Admin -s User -p admin_password -L defaultcontext \-e oxadmin@company.com -q 1024 –access-combination-name=all

svn –password “” –username anonymous co https://svn.open-xchange.com/ox-quickinstall/

http://paste.ubuntu.com/5789872/

http://paste.ubuntu.com/5789888/

Next,

$ /opt/open-xchange/sbin/createcontext -A oxadminmaster -P mysql123 -c 1 \

-u oxadmin -d “Context admin” -g aAdmin -s User -p mysql123 -L defaultcontext \

-e oxadmin@clevercircuits.com -q 1024 –access-combination-name=all

$ /opt/open-xchange/sbin/createcontext -A oxadminmaster -P mysql123 -c 1 \

-u oxadmin -d “Context admin” -g admin -s User -p mysql123d -L defaultcontext \

-e oxadmin@clevercircuits.com -q 1024 –access-combination-name=groupware_standard

$ /opt/open-xchange/sbin/createuser -c 1 -A oxadmin -P mysql123 -u testuser \

-d “Test User” -g test -s User -p secret -e testuser@clevercircuits.com \

–imaplogin testuser –imapserver 127.0.0.1 –smtpserver 127.0.0.1

Learn Puppet with Me – Day 2

Estimated Reading Time: 2 minutes

Today we are going to learn about Puppet Modules.

What is Puppet Modules? Puppetlabs defines it as “Modules are self-contained bundles of code and data. You can write your own modules or you can download pre-built modules from the Puppet Forge.”Nearly all Puppet manifests belong in modules. The sole exception is the main site.pp manifest, which contains site-wide and node-specific code.

puppetlabs-memcache

Modules are how Puppet finds the classes and types it can use — it automatically loads any classor defined type stored in its modules.

Module Layout

On disk, a module is simply a directory tree with a specific, predictable structure:

  • <MODULE NAME>
    • manifests
    • files
    • templates
    • lib
    • facts.d
    • tests
    • spec

We will start with basic module and slowly move towards the complex module structure.

Let’s begin:

#mkdir modules/memcached
#mkdir modules/memcached/manifests
#mkdir modules/memcached/files
#vi nodes.pp

node ‘puppetagent1.cse.com’ {
include memcached
}
#define memcached class in the file init.pp
#vi modules/memcached/manifests/init.pp

class memcached {
package { ‘memcached’:
ensure => installed,
}

file { ‘/etc/memcached.conf’:
source => puppet:///modules/memcached/memcached.conf’,
owner => ‘root’,
group => ‘root’,
mode => ‘0644’,
require => Package[‘memcached’],
}

service { ‘memcached’:
ensure => running,
enable => true,
require => [Package[‘memcached’], File[ ‘/etc/memcached.conf’]]
}
}

That’s all. You can go ahead and run puppet agent -t on puppet client machine to get memcache ready.

Learn Puppet With Me – Day 1

Estimated Reading Time: 2 minutes

Today is the day 1 of Learn Puppet with Me. I am starting this thread for those who want to learn Puppet smoothly.

Puppet is an Automation IT tool and I have already talked about its capabilities in my last post related to Puppet.

Puppet-in-bits

Let’s demystify the puppet fundamentals through this easy step.

Day 1: How to create a file with content “Hello, World” on puppet agent?

Say, I have a puppet master and agent ready. All I want is to create a file in puppet agent either running the command in puppet agent or directly fetching it from puppet master. Here we go –

Run the below commands on Puppetmaster Machine:

1. Create a directory called puppet:

#mkdir puppet

2. Change to puppet directory:

#cd puppet

3. Under it , create a subfolder called manifests:

#mkdir manifests

4. Create a file called site.pp under manifests:

#vi manifests/site.pp

import ‘nodes.pp’

5. Create a file called nodes.pp under manifest and add the following entries:

#vi manifests/nodes.pp

node’puppetagent1.cse.com’ {
file { ‘/tmp/hello’:
content => “hello, world\n”,
}
}

6. That’s all. Now test your manifests with the puppet apply command.

#puppet apply manifests/site.pp

OR

7. Run the following command on puppet agent:

# puppet agent -t

Verify if puppet created the file with the contents on the puppet agent machine.

It’s very simple way to create a file on puppet client through puppet master.

In next episode, we will talk about the Puppet Style and parameters.

How to setup RAID 1 on Ubuntu Linux?

Estimated Reading Time: 4 minutes

RAID 1 creates a mirror on the second drive. .You will need to create RAID aware partitions on your drives before you can create RAID and you will need to install mdadm on Ubuntu.

raid1

You may have to create the RAID device first by indicating the RAID device with the block major and minor numbers. Be sure to increment the “2” number by one each time you create an additional RAID device.

# mknod /dev/md1 b 9 2

This will create the device if you have already used /dev/md0.

Create RAID 1

# mdadm –create /dev/md1 –level=1 –raid-devices=2 /dev/sdb7 /dev/sdb8

–create
This will create a RAID array. The device that you will use for the first RAID array is /dev/md1.

–level=1
The level option determines what RAID level you will use for the RAID.

–raid-devices=2 /dev/sdb7 /dev/sdb8
Note: for illustration or practice this shows two partitions on the same drive. This is NOT what you want to do, partitions must be on separate drives. However, this will provide you with a practice scenario. You must list the number of devices in the RAID array and you must list the devices that you have partitioned with fdisk. The example shows two RAID partitions.
mdadm: array /dev/md0 started.

Verify the Create of the RAID

# cat /proc/mdstat

Personalities : [raid0] [raid1]

md2 : active raid1 sdb8[1] sdb7[0]

497856 blocks [2/2] [UU]

[======>…………..] resync = 34.4% (172672/497856) finish=0.2min speed=21584K/sec

md0 : active raid0 sdb6[1] sdb5[0]

995712 blocks 64k chunks

unused devices:

# tail /var/log/messages

You can also verify that RAID is being built in /var/log/messages.

May 19 09:21:45 ub1 kernel: [ 5320.433192] md: raid1 personality registered for level 1

May 19 09:21:45 ub1 kernel: [ 5320.433620] md2: WARNING: sdb7 appears to be on the same physical disk as sdb8.

May 19 09:21:45 ub1 kernel: [ 5320.433628] True protection against single-disk failure might be compromised.

May 19 09:21:45 ub1 kernel: [ 5320.433772] raid1: raid set md2 active with 2 out of 2 mirrors

May 19 09:21:45 ub1 kernel: [ 5320.433913] md: resync of RAID array md2

May 19 09:21:45 ub1 kernel: [ 5320.433926] md: minimum _guaranteed_ speed: 1000 KB/sec/disk.

May 19 09:21:45 ub1 kernel: [ 5320.433934] md: using maximum available idle IO bandwidth (but not more than 200000 KB/sec) for resync.

May 19 09:21:45 ub1 kernel: [ 5320.433954] md: using 128k window, over a total of 497856 blocks.

Create the File System ext 3.
You have to place a file system on your RAID device. The journaling system ext3 is placed on the device in this example.

# mke2fs -j /dev/md1

mke2fs 1.40.8 (13-Mar-2008)

Filesystem label=

OS type: Linux

Block size=1024 (log=0)

Fragment size=1024 (log=0)

124928 inodes, 497856 blocks

24892 blocks (5.00%) reserved for the super user

First data block=1

Maximum filesystem blocks=67633152

61 block groups

8192 blocks per group, 8192 fragments per group

2048 inodes per group

Superblock backups stored on blocks:

8193, 24577, 40961, 57345, 73729, 204801, 221185, 401409

Writing inode tables: done

Creating journal (8192 blocks): done

Writing superblocks and filesystem accounting information: done

This filesystem will be automatically checked every 35 mounts or

180 days, whichever comes first. Use tune2fs -c or -i to override.

Mount the RAID on the /raid Partition

In order to use the RAID array you will need to mount it on the file system. For testing purposes you can create a mount point and test. To make a permanent mount point you will need to edit /etc/fstab.

# mount /dev/md1 /raid

# df
The df command will verify that it has mounted.

Filesystem 1K-blocks Used Available Use% Mounted on

/dev/sda2 5809368 2699256 2817328 49% /

varrun 1037732 104 1037628 1% /var/run

varlock 1037732 0 1037732 0% /var/lock

udev 1037732 80 1037652 1% /dev

devshm 1037732 12 1037720 1% /dev/shm

/dev/sda1 474440 49252 400691 11% /boot

/dev/sda4 474367664 1738024 448722912 1% /home

/dev/md1 482090 10544 446654 3% /raid

You should be able to create files on the new partition. If this works then you may edit the /etc/fstab and add a line that looks like this:

/dev/md1 /raid defaults 0 2

Be sure to test and be prepared to enter single user mode to fix any problems with the new RAID device.

Create a Failed RAID Disk

In order to test your RAID 1 you can fail a disk, remove it and reinstall it. This is an important feature to practice.

# mdadm /dev/md1 -f /dev/sdb8
This will deliberately make the /dev/sdb8 faulty.

mdadm: set /dev/sdb8 faulty in /dev/md1

root@ub1:/etc/network# cat /proc/mdstat

Personalities : [raid0] [raid1]

md2 : active raid1 sdb8[2](F) sdb7[0]

497856 blocks [2/1] [U_]

md0 : active raid0 sdb6[1] sdb5[0]

995712 blocks 64k chunks

unused devices:

Hot Remove the Failed Disk
You can remove the faulty disk from the RAID array.

# mdadm /dev/md1 -r /dev/sdb8

mdadm: hot removed /dev/sdb8

Verify the Process

You should be able to see the process as it is working.

# cat /proc/mdstat

Personalities : [raid0] [raid1]

md2 : active raid1 sdb7[0]

497856 blocks [2/1] [U_]

md0 : active raid0 sdb6[1] sdb5[0]

995712 blocks 64k chunks

unused devices:

Add a Replacement Drive HOT

This will allow you to add a device into the array to replace the bad one.
# mdadm /dev/md1 -a /dev/sdb8

mdadm: re-added /dev/sdb8

Verify the Process.

# cat /proc/mdstat

Personalities : [raid0] [raid1]

md2 : active raid1 sdb8[2] sdb7[0]

497856 blocks [2/1] [U_]

[=====>……………] recovery = 26.8% (134464/497856) finish=0.2min speed=26892K/sec

md0 : active raid0 sdb6[1] sdb5[0]

995712 blocks 64k chunks

unused devices:

How to setup RAID 0 on Ubuntu Linux?

Estimated Reading Time: 3 minutes

RAID 0 will create striping to increase read/write speeds as the data can be read and written on separate disks at the same time. This level of RAID is what you want to use if you need to increase the speed of disk access.You will need to create RAID aware partitions on your drives before you can create RAID and you will need to install mdadm on Ubuntu.

raid0
These commands must be done as root or you must add the sudo command in front of each command.

# mdadm –create /dev/md0 –level=0 –raid-devices=2 /dev/sdb5 /dev/sdb6

–create
This will create a RAID array. The device that you will use for the first RAID array is /dev/md0.

–level=0
The level option determines what RAID level you will use for the RAID.

–raid-devices=2 /dev/sdb5 /dev/sdb6
Note: for illustration or practice this shows two partitions on the same drive. This is NOT what you want to do, partitions must be on separate drives. However, this will provide you with a practice scenario. You must list the number of devices in the RAID array and you must list the devices that you have partitioned with fdisk. The example shows two RAID partitions.
mdadm: array /dev/md0 started.

Check the development of the RAID.

# cat /proc/mdstat

Personalities : [raid0]

md0 : active raid0 sdb6[1] sdb5[0]

995712 blocks 64k chunks
unused devices:

# tail /var/log/messages
You can also verify that RAID is being built in /var/log/messages.

May 19 09:08:51 ub1 kernel: [ 4548.276806] raid0: looking at sdb5

May 19 09:08:51 ub1 kernel: [ 4548.276809] raid0: comparing sdb5(497856) with sdb6(497856)

May 19 09:08:51 ub1 kernel: [ 4548.276813] raid0: EQUAL

May 19 09:08:51 ub1 kernel: [ 4548.276815] raid0: FINAL 1 zones

May 19 09:08:51 ub1 kernel: [ 4548.276822] raid0: done.

May 19 09:08:51 ub1 kernel: [ 4548.276826] raid0 : md_size is 995712 blocks.

May 19 09:08:51 ub1 kernel: [ 4548.276829] raid0 : conf->hash_spacing is 995712 blocks.

May 19 09:08:51 ub1 kernel: [ 4548.276831] raid0 : nb_zone is 1.

May 19 09:08:51 ub1 kernel: [ 4548.276834] raid0 : Allocating 4 bytes for hash.

Create the ext 3 File System
You have to place a file system on your RAID device. The journaling system ext3 is placed on the device in this example.

# mke2fs -j /dev/md0

mke2fs 1.40.8 (13-Mar-2008)

Filesystem label=

OS type: Linux

Block size=4096 (log=2)

Fragment size=4096 (log=2)

62464 inodes, 248928 blocks

12446 blocks (5.00%) reserved for the super user

First data block=0

Maximum filesystem blocks=255852544

8 block groups

32768 blocks per group, 32768 fragments per group

7808 inodes per group

Superblock backups stored on blocks:

32768, 98304, 163840, 229376

Writing inode tables: done

Creating journal (4096 blocks): done

Writing superblocks and filesystem accounting information: done

This filesystem will be automatically checked every 39 mounts or

180 days, whichever comes first. Use tune2fs -c or -i to override.

Create a Place to Mount the RAID on the File System

In order to use the RAID array you will need to mount it on the file system. For testing purposes you can create a mount point and test. To make a permanent mount point you will need to edit /etc/fstab.

# mkdir /raid

Mount the RAID Array

# mount /dev/md0 /raid

You should be able to create files on the new partition. If this works then you may edit the /etc/fstab and add a line that looks like this:

/dev/md0 /raid defaults 0 2

Be sure to test and be prepared to enter single user mode to fix any problems with the new RAID device.

Hope you find this article helpful.

How do I bind NIC interrupts to selected CPU?

Estimated Reading Time: 2 minutes

I read this interesting mailing thread few weeks back. I won’t be late to share this with open source enthusiast like you. Here goes the story:

nic2

I have a 4 Quad server, am trying to bind NIC eth0 interrupt(s) to CPU4 and CPU5. As of now, my eth0 is found bind to all the 8’s.
#grep eth0 /proc/interrupts | awk ‘{print $NF}’ | sort

eth0-0
eth0-1
eth0-2
eth0-3
eth0-4
eth0-5
eth0-6
eth0-7

How to move ahead?

Solution: Follow these steps to get it done.

As I am using Broadcom card(bnx2), I am going to run this command and reboot my machine.

Open the terminal:

echo “options bnx2 disable_msi=1” > /etc/modprobe.d/bnx2.conf

then reboot, after you’ll only see one irq for eth0.

Next, run this command:

echo cpumask > /proc/irq/IRQ-OF-ETH0-0/smp_affinity

I believe the mask for cpu4 is 10 and cpu5 is 20.
(don’t forget to disable irqbalance)

you can only bind the irqs for one nic to one core at a time.

or you could do something fancy/silly with isolcpus and….

isolcpus all but 4/5 so that all irqs will be scheduled on 4/5. this will
mean that the kernel can only schedule tasks on cpu4/5.

Hope it helps !!!
then use cpusets/taskset/tuna to move all the processes off cpu 4/5… and
you’ll have to use taskset/cpuset/tuna for every task to ensure its not
using cpu4/5

Puppet Module for JBOSS

Estimated Reading Time: 5 minutes

Recently one of my colleague called me up with a problem statement where he was finding difficulty configuring JBOSS through puppet. I tried to help him through one of VMware Workstation box on my Dell Inspiron.

puppetlabs_1304099092_11

I tried to google but couldn’t find the working example. I tried my hands of my own and YES…I did it finally.

I am sharing the overall idea how to deploy and configure JBOSS through Puppet.

Let’s say you have the following steps which you manually perform for installing JBOSS on your Linux machine:

1.$ su -c “yum install java-1.6.0-openjdk-devel”

2.$ java –version

3.wget http://download.jboss.org/jbossas/7.1/jboss-as-7.1.1.Final/jboss-as-7.1.1.Final.zip

4.$ unzip jboss-as-7.1.1.Final.zip -d /usr/share

5.$ adduser jboss

6.$ chown -fR jboss.jboss /usr/share/jboss-as-7.1.1.Final/

7.$ su jboss

8.$ cd /usr/share/jboss-as-7.1.1.Final/bin/

9.$ ./add-user.sh

You should see the following message on the console after executing the command:

What type of user do you wish to add?

a) Management User (mgmt-users.properties)

b) Application User (application-users.properties)

(a): a

We select “a”, next you should see the following message:

Enter the details of the new user to add.

Realm (ManagementRealm) :

Username : jboss

Password :

Re-enter Password :

* hit enter for Realm to use default, then provide a username and password

We select the default value for the Realm (ManagementRealm), by hitting enter, and select “jboss” as our username. By default, we supply “jb0ss” as our password, of course, you can provide any password you prefer here.

Step 4: Start the JBoss AS 7 server:

Once the appropriate JBoss users are created, we are now ready to start our new JBoss AS 7 server. With JBoss AS 7, a new standalone and domain model has been introduced. In this tutorial, we focus on starting up a standalone server. The domain server will be part of a future tutorial.

Startup a JBoss 7, standalone instance:

A standalone instance of JBoss 7 can be starting by executing:

$ ./standalone.sh -Djboss.bind.address=0.0.0.0 -Djboss.bind.address.management=0.0.0.

We can automate those steps for client through Puppet.Let’s start writing puppet init.pp from scratch. I will be delivering step by step of init.pp to achieve every components of init.pp.

Line 1 – 4

The Line 1 to 4 does nothing but downloading JBOSS to /usr/share directory. What we are going to do is put the downloaded jboss-as-7.1.1.Final on /var/lib/puppet/files directory on puppet master and push it to the puppet-client at /usr/share/jboss-as directory.

Here is the below init.pp

The above init.pp define a class jboss-custom, takes JBOSS-as-7.1.1-Final from the puppet-master /var/lib/puppet/files/ and push it to the puppet-client.

Que: How does it know which directory to pull the files from?

Answer: Under /etc/puppet/fileserver.conf, we define those path and permission as shown below:

Shall we start?

Ensure that you have put JBOSS-as-7.1.1-Final under /var/lib/puppet/files directory with permission:

#chown –R puppet:puppet /var/lib/puppet

The permission is very important and shouldn’t be skipped.

Now run the command from the puppet-client to check if it runs without any issue:

Wow !!! Our first program went well and the server has pushed the file to the puppet-client successfully.

Line:5 to 9

The easiest way of performing the overall step is writing a shell script which will run on the remote machine:

Go to /var/lib/puppet/files and create a script called jbossdeploy.sh

#!/bin/bash

groupadd jbossas

useradd -g jbossas -p deQcvEr1PRPSM jbossas

chown -fR jbossas:jbossas /usr/share/jboss-as-7.1.1.Final/

cd /usr/share/jboss-as/

#!/usr/bin/expect

spawn ./add-user.sh

expect “(a):”

send “a”

expect “Realm (ManagementRealm):”

send “ManagementRealm”

expect “Username:”

send “jbossas”

expect “Password:”

send “jbossas”

expect “Re-enter Password:”

send “jbossas”

cd /usr/share/jboss-as-7.1.1.Final/bin

./standalone.sh -Djboss.bind.address=0.0.0.0 -Djboss.bind.address.management=0.0.0.0&

The above script will create jboss user and group, run add-user.sh command under /usr/share/jboss-as/jboss-as-7.1.1.Final/bin directory. I have used expect library (ensure it is already installed) in perl.

Lets modify the init.pp so as to accommodate this script execution as shown:

class jboss-custom {

file {‘/usr/share/jboss-as/jboss-7.1.1.Final’:

owner => ‘root’,

group => ‘root’,

mode => ‘0440’,

source => ‘puppet://puppet-server.test.com/files/jboss-as-7.1.1.Final’

}

file { ‘/usr/share/jboss-as/jbossdeploy.sh’:

source => ‘puppet://puppet-server.test.com/files/jbossdeploy.sh’

}

exec { “/usr/share/jboss-as/jbossdeploy.sh”:}

}

If you run now the following it goes all fine and start the JBOSS application server.

Lets test it.

puppet agent –test –verbose –server puppet-server.test.com

info: Caching catalog for puppet-client.test.com

info: Applying configuration version ‘1345944985’

notice: /File[/usr/share/jboss-as/jbossdeploy.sh]/content:

— /usr/share/jboss-as/jbossdeploy.sh 2012-08-29 17:37:01.365003616 -0400

+++ /tmp/puppet-file20120829-18751-81r0p8-0 2012-08-29 17:44:40.993732919 -0 400

@@ -1,12 +1,10 @@

#!/bin/bash

groupadd jbossas

useradd -g jbossas -p deQcvEr1PRPSM jbossas

chown -fR jbossas:jbossas /usr/share/jboss-as-7.1.1.Final/

-cd /usr/share/jboss-as/jboss-as-7.1.1.Final/bin

+cd /usr/share/jboss-as/

#!/usr/bin/expect

-/usr/bin/expect << EOD

-spawn sh add-user.sh

+spawn ./add-user.sh

expect “(a):”

send “a”

expect “Realm (ManagementRealm):”

@@ -17,7 +15,6 @@

send “jbossas”

expect “Re-enter Password:”

send “jbossas”

-EOD

cd /usr/share/jboss-as-7.1.1.Final/bin

./standalone.sh -Djboss.bind.address=0.0.0.0 -Djboss.bind.address.management=0. 0.0.0&

info: FileBucket adding {md5}afc9bd6b8229da628396b90f2759f41f

info: /File[/usr/share/jboss-as/jbossdeploy.sh]: Filebucketed /usr/share/jboss-a s/jbossdeploy.sh to puppet with sum afc9bd6b8229da628396b90f2759f41f

notice: /File[/usr/share/jboss-as/jbossdeploy.sh]/content: content changed ‘{md5 }afc9bd6b8229da628396b90f2759f41f’ to ‘{md5}140ab2a8605d1164793c2175aa972675’

notice: /Stage[main]/Jboss-custom/Exec[/usr/share/jboss-as/jbossdeploy.sh]/retur ns: executed successfully

notice: /File[/usr/share/jboss-as/jboss-7.1.1.Final]/ensure: created

notice: Finished catalog run in 0.79 seconds

[root@puppet-client ~]#

Automated shutdown of ESXi and VMs through Powershell

Estimated Reading Time: 3 minutes

My friend was running a cybercafe center. He has dozens of ESXi running 100 of VMs. He was using Dell Multi-UPS Management console to handle his small computer center. The very often power cut was ruining his business. He called up me to see there is a possible solution to handle graceful shutdown of VMs and then ESXi through the available tools and technique. This solution really helped us to handle the situation gracefully.

I just noted down the steps roughly and will not be late to share it with you:

1. Download the Dell Multi-UPS Management Console from http://www.dell.com/support/home/us/en/19/Drivers/DriversDetails?driverId=R318771

2. Execute DELL_MULTI-DEVICE_A00_R318771.exe and run the setup to install on Windows 2008 R2.

3. Once installed, try accessing it through:

http://127.0.0.1:4679/default.html

4. The default credential for the MUMC login is admin/admin

5.A quick scan is automatically performed and you will get to see the overall power devices available.

6. Click on Settings > Systems and enable shutdown as shown below

7. Click on Shutdown option and configure as shown below

The above shown is a sample entry. Change it as per your required IPs.

In our case, that’s :

Power Source: 192.168.1.99

Load Segment: Master Output

Login/password: admin/admin

Shutdown timer : 180 sec

Shutdown duration: 120 sec

Type: Script

Location: C:\scripts/Ajeet.bat

8. Save the settings

9. Now let’s create a required scripts one by one for shutting down ESXi and VMs.

Script 1:

location: C:\scripts\Ajeet.bat

Start shutdownvms.bat

Script 2:

location: C:\scripts\shutdownvms.bat

———- Start——–

PowerShell.exe C:\Scripts\shutdownallvms.ps1

———End————–

Script 3:

Location:C:\scripts\shutdownallvms.ps1

========= START =========

Add-Snappin “Vmware. Automation.Core”

Connect-VIServer IPADDRESS

# Get All the ESX Hosts

$ESXSRV = Get-VMHost

# For each of the VMs on the ESX hosts

Foreach ($VM in ($ESXSRV | Get-VM)){

# Shutdown the guest cleanly

$VM | Shutdown-VMGuest -Confirm:$false

}

# Set the amount of time to wait before assuming the remaining powered on guests are stuck

$waittime = 200 #Seconds

$Time = (Get-Date).TimeofDay

do {

# Wait for the VMs to be Shutdown cleanly

sleep 1.0

$timeleft = $waittime – ($Newtime.seconds)

$numvms = ($ESXSRV | Get-VM | Where { $_.PowerState -eq “poweredOn” }).Count

Write “Waiting for shutdown of $numvms VMs or until $timeleft seconds”

$Newtime = (Get-Date).TimeofDay – $Time

} until ((@($ESXSRV | Get-VM | Where { $_.PowerState -eq “poweredOn” }).Count) -eq 0 -or ($Newtime).Seconds -ge $waittime)

# Shutdown the ESX Hosts

$ESXSRV | Foreach {Get-View $_.ID} | Foreach {$_.ShutdownHost_Task($TRUE)}

Write-Host “Shutdown Complete”

===== END ========

10. Now its time to test drive the scripts. Just try running those script manually and confirm if those works or not.

11. Now time to test drive with UPS System menu option. Click on Shutdown option of MUMC and Click on Test Shutdown. The Script should work and power-on VMs and ESXi gradually.

12. Try creating the power-off situation and let UPS perform this activity of running the scripts to shutdown ESXi and VMs.

It really went flawless saving couple of rupees for my friend. He treated me the next day for the overall efforts and we rocked the party too 🙂

Installing Java Development Toolkit (JDK) on Linux

Estimated Reading Time: 4 minutes

Installing Java Development toolkit on Linux is always matter of difficulty since Oracle provides both RPM and tarballs in their official website. Installing RPM is pretty simple but installing through tarballs need sufficient environmental variable declaration which is little different from Windows Operating System.

javalinux

Follow the steps mentioned below to install java-1.6.

1.Download jdk-6u16-linux-x64.bin (From the URL mentioned in the prerequisites section) to /opt directory.

2.Change the permission of jdk-6u16-linux-x64.bin file to 755.

[root@bang opt]# chmod 755 jdk-6u16-linux-x64.bin

3.Execute the jdk-6u16-linux-x64.bin file to start installation.

[root@bang opt]# ./jdk-6u16-linux-x64.bin

Sun Microsystems, Inc. Binary Code License Agreement

for the JAVA 2 PLATFORM STANDARD EDITION DEVELOPMENT KIT 6.0

SUN MICROSYSTEMS, INC. (“SUN”) IS WILLING TO LICENSE THE

SOFTWARE IDENTIFIED BELOW TO YOU ONLY UPON THE CONDITION

THAT YOU ACCEPT ALL OF THE TERMS CONTAINED IN THIS BINARY

CODE LICENSE AGREEMENT AND SUPPLEMENTAL LICENSE TERMS

(COLLECTIVELY “AGREEMENT”). PLEASE READ THE AGREEMENT

CAREFULLY. BY DOWNLOADING OR INSTALLING THIS SOFTWARE, YOU

ACCEPT THE TERMS OF THE AGREEMENT. INDICATE ACCEPTANCE BY

SELECTING THE “ACCEPT” BUTTON AT THE BOTTOM OF THE

AGREEMENT. IF YOU ARE NOT WILLING TO BE BOUND BY ALL THE

TERMS, SELECT THE “DECLINE” BUTTON AT THE BOTTOM OF THE

AGREEMENT AND THE DOWNLOAD OR INSTALL PROCESS WILL NOT

CONTINUE.

1. DEFINITIONS. “Software” means the identified above in

binary form, any other machine readable materials

(including, but not limited to, libraries, source files,

header files, and data files), any updates or error

corrections provided by Sun, and any user manuals,

programming guides and other documentation provided to you

by Sun under this Agreement. “General Purpose Desktop

Computers and Servers” means computers, including desktop,

laptop and tablet computers, or servers, used for general

computing functions under end user control (such as but not

specifically limited to email, general purpose Internet

browsing, and office suite productivity tools). The use of

Software in systems and solutions that provide dedicated

functionality (other than as mentioned above) or designed

for use in embedded or function-specific software

applications, for example but not limited to: Software

embedded in or bundled with industrial control systems,

wireless mobile telephones, wireless handheld devices,

kiosks, TV/STB, Blu -ray Disc devices, telematics and

network control switching equipment, printers and storage

management systems, and other related systems are excluded

from this definition and not licensed under this Agreement.

“Programs” means Java technology applets and applications

intended to run on the Java Platform Standard Edition (Java

SE) ptaform on Java-enabled General Purpose Desktop

Computers and Servers.

4.Press the Tab key until you reach the End of License Agreement Screen

5.Type yes and hit Enter key

6.It will installs jdk-1.6.0_16

F. Source Code. Software may contain source code that,

unless expressly licensed for other purposes, is provided

solely for reference purposes pursuant to the terms of this

Agreement. Source code may not be redistributed unless

expressly provided for in this Agreement.

G. Third Party Code. Additional copyright notices and

license terms applicable to portions of the Software are set

forth in the THIRDPARTYLICENSEREADME.txt file. In addition

to any terms and conditions of any third party

opensource/freeware license identified in the

THIRDPARTYLICENSEREADME.txt file, the disclaimer of warranty

and limitation of liability provisions in paragraphs 5 and 6

of the Binary Code License Agreement shall apply to all

Software in this distribution.

H. Termination for Infringement. Either party may

terminate this Agreement immediately should any Software

become, or in either party’s opinion be likely to become,

the subject of a claim of infringement of any intellectual

property right.

I. Installation and Auto-Update. The Software’s

installation and auto-update processes transmit a limited

amount of data to Sun (or its service provider) about those

specific processes to help Sun understand and optimize them.

Sun does not associate the data with personally identifiable

information. You can find more information about the data

Sun collects at http://java.com/data/.

For inquiries please contact: Sun Microsystems, Inc., 4150

Network Circle, Santa Clara, California 95054, U.S.A.

(LFI#143333/Form ID#011801)

Do you agree to the above license terms? [yes or no]

yes

Unpacking…

Checksumming…

0

0

Extracting…

UnZipSFX 5.50 of 17 February 2002, by Info-ZIP (Zip-Bugs@lists.wku.edu).

inflating: jdk-6u16-linux-x64-rpm

Preparing… #################################### [100%]

1:jdk #################################### [100%]

Done.

#

7.Add below lines in /etc/profile file to define JAVA_HOME and its PATH

export JAVA_HOME=/usr/java/jdk-1.6.0_16

export PATH=$PATH:$HOME/bin:$JAVA_HOME/bin:

8. Verify the java installation by running the following command in the new window.`

[root@bang opt]# java -version

java version “1.6.0_16”

Java(TM) SE Runtime Environment (build 1.6.0_16-b01)

Java HotSpot(TM) 64-Bit Server VM (build 14.2-b01, mixed mode)

[root@bangvbapp opt]#