Monitor Dell servers on Debian Squeeze with Nagios

Im just writing up this post because the dellomsa packages arent working with the new Debian Squeeze 6.0.

I had problems with the omreport command not giving me info of ex memory/psu/cpu. (omreport chassis info said No sensors found etc)

I used some hours to try to get it working with a newer dellomsa but that didnt work either.
Then i found some official Dell Ubuntu packages, which i found working excellent on Debian Squeeze as well:
dpkg -P dellomsa #Make sure dellomsa isnt installed.
echo 'deb http://linux.dell.com/repo/community/deb/latest /' | sudo tee -a /etc/apt/sources.list.d/linux.dell.com.sources.list
apt-get update
apt-get install srvadmin-base smbios-utils

You will also need the libsmbios2_2.2.13-0ubuntu4_amd64.deb from Ubuntu Lucid to get smbios stuff working.
dpkg -i libsmbios2_2.2.13-0ubuntu4_amd64.deb
/etc/init.d/dataeng start #if this starts, omreport works!

Now you have the newer Debian Squeeze Dell stuff working.

We have deployed our hwmonitoring of our Dell servers with check_openmanage and Nagios
Read more about the check_openmanage on the check_openmanage site (this is a great plugin btw!)

Resources:
http://folk.uio.no/trondham/software/check_openmanage.html
http://linux.dell.com/repo/community/deb/latest/

Tags: , , , , ,

Backporting varnish from debian unstable/testing to etch

Recently i wanted to upgrade Varnish to latest stable – 2.0.4 on our debian etch boxes. Do you think debian has the latest version in the etch repos? Ofcourse not.. However, backporting turned out to be allmost too easy :) Here is how I did it;

1. Edit your /etc/apt/sources.list deb-src entry to reflect which debian version you want to backport packages from. My entry looks like this:
deb-src http://ftp.no.debian.org/debian/ unstable main non-free contrib
2. Download the source code from the repo:
apt-get source varnish
3. Make sure you have all the deps needed:
apt-get build-dep varnish
4. (might not be necessary) I was missing the fakeroot package:
apt-get install fakeroot
5. Build the package(s):
cd varnish-2.0.4
dpkg-buildpackage -rfakeroot -uc -b

6. And finally install your newly created packages:
cd ..
dpkg -i *.deb

Reference(s):
http://www.debian.org/doc/manuals/apt-howto/ch-sourcehandling.en.html

Tags: , , , , , ,

Making Debian Etch boot with UUID, not hardcoded blockdevicename

You got additional disk controllers in your server and the etch kernel is fucking up your devicenames from time to time.
Suddenly when you reboot the etch kernel decides to make /dev/sda become /dev/sdf etc.
Ubuntu has learned from this and they use UUID.
This is a unique ID the filesystem becomes, and when you boot, you really want to catch the right device for the root filesystem, or get a crappy Busybox shell.

This is how you implement it on Debian Etch (and possibly Lenny, if its not default there either):

fittafeis:~# /lib/udev/vol_id /dev/sda1
ID_FS_USAGE=filesystem
ID_FS_TYPE=ext3
ID_FS_VERSION=1.0
ID_FS_UUID=3abbdf59-4a9e-4be1-ac56-f474609e637c
ID_FS_LABEL=
ID_FS_LABEL_SAFE=

Now edit this line in /boot/grub/menu.lst:

From: # kopt=root=/dev/sda1 ro
To: # kopt=root=UUID=3abbdf59-4a9e-4be1-ac56-f474609e637c ro

Then run update-grub and it will update all your kernel entries using UUID :)

Tags: , , ,

Mysql authentication backwards compability

Had an issue today where an old mysql-client on debian sarge failed to connect to a newer mysql-server on debian etch.

I was happy to find out that it’s easily fixed :

update mysql.user set password=OLD_PASSWORD('supahsecretpassword') where user='theuser' and host='some-ip-address';

then just :

flush privileges;

and you’re gooooood to goooooooooo :)

Tags: , ,

Setting up postgres loadbalancing on debian

There’s a few ways to do it, but I personally found pgpool2 to be very nice and easy to set up.
Here’s a quick and dirty guide to get it up and running.

First of all, you want your database backends to look exactly the same.
So dump the database and restore it on your second server.
Make sure your postgres backends allows your pgpool host to connect to them.

Pick a host where you wanna run pgtool2.

apt-get install pgpool2

It will automatically install a config file under /etc/pgpool.conf and start up pgpool2. Not very useful yet, since you need to do some configuring, so just kill it and open the config file for editing.
Note: pgpool has several config options that you can play with. I’m not gonna go through them here, but just show how to get the basics up and running.

First of all, you need to config access to the master/system dbserver:

# system DB info
system_db_hostname = 'test-dbserver1'
system_db_port = 5432
system_db_dbname = 'pgpool'
system_db_schema = 'pgpool_catalog'
system_db_user = 'someuser'
system_db_password = 'md5hashedvalue'

To find your md5 hashed password value, you can just use the command pg_md5 :

pg_md5 yourpasswordhere

After that you should configure info about your backend servers :

backend_hostname0 = 'test-dbserver1'
backend_port0 = 5432
backend_weight0 = 1
backend_hostname1 = 'test-dbserver2'
backend_port1 = 5432
backend_weight1 = 1

….and that’s about it. Now fire up pgpool2 on your host by typing :

pgpool -n -d

This will start pgpool in the foreground and display all debug info about whats going on. Handy when checking if things work properly. If they do, you can restart it by typing :

pgpool -f /etc/pgpool.conf

If you want to turn on replication, just change the parameter to:

replication_mode = true

..and restart pgpool.

Tags: , , ,

Debian Lenny, Xen and live migration using iscsi storage backend

After alot of google-ing for iscsi powered live migration i found references to these two block-iscsi scripts:

http://kinkrsoftware.nl/contrib/xen/block-iscsi
http://www.suse.de/~garloff/linux/xen/scripts/block-iscsi

However none of them worked “out of the box” with Debian Lenny. After alot of messing around with adding debug output in the scripts i found the script in the first link to work when i changed the path to iscsiadm from /usr/sbin to /usr/bin (since the open-iscsi lenny package installs iscsiadm to /usr/bin) and when using the following in my VMs .cfg:

disk = [ 'iscsi:iqn.2006-01.com.openfiler:tsn.64b04dcbd835,hda1,w' ]

Dont forget to stick the block-iscsi script in /etc/xen/scripts and chmod +x ! :)

Tags: , , ,

Debian sarge (3.1) repositories gone

Yes so the sarge repos is gone from all official debian mirrors.

Still got an old sarge box and want to install some packages?

Add this line to your /etc/apt/sources.list:

deb http://archive.debian.org/debian/ sarge main

And remove all the others.

Now do:
apt-get update

Now you should be able to install packages for your sarge. (But you really should upgrade due to big security risks!)

Tags: , , ,

I just installed a new debian etch box with dhcp, changed the /etc/network/interfaces to a static setup and now /etc/init.d/network restart does not work….

gah… We use FAI to auto install our debian servers. It installs the system with dhcp and when it reboots for the first time the box comes perfectly online. However, I want the box to have a static address and so far this is a manual operations in our setup. So i change my /etc/network/interfaces file to reflect a static setup, something like:

host:~# cat /etc/network/interfaces
# The loopback network interface
auto lo
iface lo inet loopback

# The primary network interface
allow-hotplug eth0
iface eth0 inet static
address 192.168.1.250
netmask 255.255.255.0
network 192.168.1.0
broadcast 192.168.1.255
gateway 192.168.1.1

I just copy/pasted this setup from another box on the same network and changed the “address” field. And then i issued:

host:~# /etc/init.d/networking restart
Reconfiguring network interfaces...done.

Hmmm nothing happens… Lets take a peak in the init.d script and see what the hell it does:

host:~# cat /etc/init.d/networking
#!/bin/sh -e
### BEGIN INIT INFO
# Provides: networking
# Required-Start: mountkernfs ifupdown $local_fs
# Required-Stop: ifupdown $local_fs
# Default-Start: S
# Default-Stop: 0 6
# Short-Description: Raise network interfaces.
### END INIT INFO

PATH="/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin"

[ -x /sbin/ifup ] || exit 0

. /lib/lsb/init-functions

spoofprotect_rp_filter() {
[ -e /proc/sys/net/ipv4/conf/all/rp_filter ] || return 1
RC=0
for f in /proc/sys/net/ipv4/conf/*/rp_filter; do
echo 1 > $f || RC=1
done
return $RC
}

spoofprotect() {
log_action_begin_msg "Setting up IP spoofing protection"
if spoofprotect_rp_filter; then
log_action_end_msg 0 "rp_filter"
else
log_action_end_msg 1
fi
}

ip_forward() {
log_action_begin_msg "Enabling packet forwarding"
if echo 1 > /proc/sys/net/ipv4/ip_forward; then
log_action_end_msg 0
else
log_action_end_msg 1
fi
}

syncookies() {
log_action_begin_msg "Enabling TCP SYN cookies"
if echo 1 > /proc/sys/net/ipv4/tcp_syncookies; then
log_action_end_msg 0
else
log_action_end_msg 1
fi
}

doopt() {
optname=$1
default=$2
opt=`grep "^$optname=" /etc/network/options || true`
if [ -z "$opt" ]; then
opt="$optname=$default"
fi
optval=${opt#$optname=}
if [ "$optval" = "yes" ]; then
eval $optname
fi
}

process_options() {
[ -e /etc/network/options ] || return 0
log_warning_msg "/etc/network/options is deprecated (see README.Debian of netbase)."
doopt spoofprotect yes
doopt syncookies no
doopt ip_forward no
}

case "$1" in
start)
process_options
log_action_begin_msg "Configuring network interfaces"
if ifup -a; then
log_action_end_msg $?
else
log_action_end_msg $?
fi
;;

stop)
if sed -n 's/^[^ ]* \([^ ]*\) \([^ ]*\) .*$/\2/p' /proc/mounts |
grep -qE '^(nfs[1234]?|smbfs|ncp|ncpfs|coda|cifs)$'; then
log_warning_msg "not deconfiguring network interfaces: network shares still mounted."
exit 0
fi

log_action_begin_msg "Deconfiguring network interfaces"
if ifdown -a --exclude=lo; then
log_action_end_msg $?
else
log_action_end_msg $?
fi
;;

force-reload|restart)
process_options
log_action_begin_msg "Reconfiguring network interfaces"
ifdown -a --exclude=lo || true
if ifup -a --exclude=lo; then
log_action_end_msg $?
else
log_action_end_msg $?
fi
;;

*)
echo "Usage: /etc/init.d/networking {start|stop|restart|force-reload}"
exit 1
;;
esac

exit 0

The interesting part is the restart|reload function:

log_action_begin_msg "Reconfiguring network interfaces"
ifdown -a --exclude=lo || true
if ifup -a --exclude=lo; then

Oki so lets try to issue these commands manually and see what happens:

host:~# ifdown eth0
ifdown: interface eth0 not configured

…. *sigh*. Now what, I just configured it… Its time to look at some other boxes where this doesnt happen. And I find that some boxes does not have the “allow-hotplug eth0″, but instead just “auto eth0″. So I changed my setup to the following:

host:~# cat /etc/network/interfaces
# The loopback network interface
auto lo
iface lo inet loopback

# The primary network interface
auto eth0
iface eth0 inet static
address 192.168.1.250
netmask 255.255.255.0
network 192.168.1.0
broadcast 192.168.1.255
gateway 192.168.1.1

And what do you know…. IT WORKED!!

Tags: , , , , ,