Build a Debian/Ganeti System on

With much help from

And And is also helpful.

Boot into MegaRaid BIOS and configure the four drives as one RAID5 for the 1RU and RAID6 for the 2RUs.

Install Debian

  • Boot Debian CD/ISO
  • Choose Install
  • Choose English, UK (so you can get UCT)
  • Choose American English
  • Name the host
  • Choose root password
  • Choose user name and password
  • Partition
    • Choose Manual Partitioning
    • Select the drive
    • Create new empty partition table
    • Select Free Space
    • Create new partition, primary, 1GB, begining, bios
    • Done
    • Select Free Space
    • Create new partition, primary, 256MGB, begining, /boot, ext4, bootable
    • Done
    • Select Free Space again
    • Create a new partition
    • Accept whatever size is shown (the rest of the disk)
    • Primary, physical volume for LVM
    • Done
  • Configure LVM
    • Configure LVM accepting write changes to disks
    • Create volume group
      • Volume group name: ganeti
      • Devices for the new volume group: select only the LVM partition
    • Create Logical Volume: on ganeti, root, 16G
    • Create Logical Volume: on ganeti, swap, 16G
    • Create Logical Volume: on ganeti, var, 16G
    • Edit the Logical Volumes to be ext4 /, swap, and ext4 /var
    • Finish partitioning and write changes
  • Finish partitioning and write changes to disk
  • Be sure it will not boot CD-ROM, and Reboot from the installed system

Finish Debian Installation

Clean up from CDROM sources

vi /etc/apt/sources.list

and delete the two CDROM entries at the top

Install homey things (it's not a computer without emacs:)

apt-get update
apt-get upgrade
apt-get install emacs23-nox
apt-get install rsync
apt-get install gcc
apt-get install bridge-utils vlan
apt-get install sudo
apt-get install unbound
usermod -G sudo -a randy

Fix hostname

echo > /etc/hostname
hostname `cat /etc/hostname`

Fix /etc/unbound/unbound.conf

        access-control: allow
        access-control: allow
        access-control: refuse
        access-control: ::1 allow
        access-control: ::ffff: allow
        access-control: 2001:deb::/48
        access-control: ::0/0 refuse

Unattended Upgrades, syslog-NG, etc.

Install Unattended Upgrading

Install syslog-ng

apt-get install syslog-ng

Hack /etc/logrotate.d/syslog-ng

*** /etc/logrotate.d/syslog-ng~ 2013-03-20 17:30:26.000000000 +0000
--- /etc/logrotate.d/syslog-ng  2014-07-31 14:00:08.148813531 +0100
*** 29,34 ****
--- 29,36 ----
+       mailfirst
+       mail

Fix Exim

dpkg-reconfigure exim4-config

Use internet mail
System name is
Only listen on local interfaces
Relay mail for noone

Debian Ganeti Specific Configuration

Edit /etc/hosts to have the real address of the host, e.g.       localhost    vm0    vm1    vm2   gnt0

Fix /etc/network/interfaces

Make eth0 hang off of whatever your bridge will be called

# This file describes the network interfaces available on your system
# and how to activate them. For more information, see interfaces(5).

# The loopback network interface
auto lo
iface lo inet loopback

# public LAN
auto eth0
iface eth0 inet manual

auto br-public
iface br-public inet manual
      bridge_ports eth0
      bridge_stp off
      bridge_fd 0
      bridge_maxwait 0

# internal LAN
auto eth1
iface eth1 inet manual

iface eth1.222 inet manual
        vlan_raw_device eth1

iface eth1.444 inet manual
        vlan_raw_device eth1

auto br-internal
iface br-internal inet static
      bridge_ports eth1.222
      bridge_stp off
      bridge_fd 0
      bridge_maxwait 0

auto br-cluster
iface br-cluster inet static
      bridge_ports eth1.444
      bridge_stp off
      bridge_fd 0
      bridge_maxwait 0

#iface eth0 inet6 auto

Check /etc.resolv.comf

In theory, this looks like

                            |         this host
                  |        eth0       |
                  |                   |
                  |eth0.255   eth0.101|
                     |           |
                   br-svc      br-rep
                     |           |
         VMs --------+           +------> to other ganeti hosts

Install Ganeti

Set up to get Ganeti from backports on all three servers, vm0, vm1, and vm2

cat > /etc/apt/sources.list.d/wheezy-backports.list
deb wheezy-backports main

And then install it on all three servers

apt-get update
apt-get install ganeti/wheezy-backports

Fix up drbd

echo "options drbd minor_count=128 usermode_helper=/bin/true" > /etc/modprobe.d/drbd.conf
rmmod drbd      # ignore any error
modprobe drbd

Initialize and Build Ganeti Cluster

gnt-cluster init \
  --master-netdev=br-internal \
  --enabled-hypervisors=kvm \
  -H kvm:kernel_path="",initrd_path="" \
  --vg-name=ganeti \
  -N link=br-cluster \
  -s \

If it barfs, try

gnt-cluster destroy --yes-do-it

then re-run gnt-cluster init ...

If you get into gnutls problems check apt-get upgrade and dist-upgrade

Check That Cluster is Happy

Fix VNC passwording

echo 'fnillies' > /etc/ganeti/vnc-cluster-password
gnt-cluster modify -H kvm:vnc_password_file=/etc/ganeti/vnc-cluster-password

Test that the cluster was built happily

gnt-cluster verify

If you get

Fri Mar 14 05:20:55 2014   - ERROR: node volume ganeti/root is unknown
Fri Mar 14 05:20:55 2014   - ERROR: node volume ganeti/swap is unknown
Fri Mar 14 05:20:55 2014   - ERROR: node volume ganeti/var is unknown

That's OK. To make it go away

gnt-cluster modify --reserved-lvs=ganeti/root,ganeti/swap,ganeti/var

Now add vm1 and vm2 to the cluster by the following on vm0

gnt-node add -s
gnt-node add -s

which should result in gnt-node add -s
-- WARNING -- 
Performing this operation is going to replace the ssh daemon keypair
on the target machine ( with the ones of the current one
and grant full intra-cluster ssh root access to/from it

The authenticity of host ' (' can't be established.
ECDSA key fingerprint is 2f:48:91:f7:f2:dd:c4:95:da:30:d2:f4:92:6a:53:ec.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added '' (ECDSA) to the list of known hosts.
Restarting OpenBSD Secure Shell server: sshd.
Fri Aug 22 05:37:53 2014  - INFO: Node will be a master candidate
Last modified 5 years ago Last modified on Mar 20, 2015, 9:50:43 PM