Installation of the Virtual Machines and copy the production server

A list:

• Find server infrastructure.  Eventually, the service will run on DMU’s converged private cloud but for now I’m making do with a DL360 G6 which will hopefully have two 4 Core CPUS and 16-32GB RAM.  Disaster recovery will be covered through thorough backup up of what is at the moment a small amount of data.  Users have the ability to upload up to 4.5GB in to a blog at a time…
• Install CentOS 6.3
  • 400GB, 50GB for the OS and the rest for VMs and storage in /usr1 including install ISOs
  • yum groupinstall “Virtualisation” “Virtualization Client” “Virtualization Platform” “Virtualization Tools” “Web Server” “PHP Support”
  • Setup networking on the host to include br0
    • added this to /etc/sysctl.d/libvirtd (might be crazy)
      • net.bridge.bridge-nf-call-ip6tables = 0
      • net.bridge.bridge-nf-call-iptables = 0
      • net.bridge.bridge-nf-call-arptables = 0
    • wondering about Jumbo frames over br0 for storage???
    • Add tap0 to br0 for better networking :

    • yum install tunctl

      # /etc/rc.local /usr/sbin/tunctl -b
      /sbin/ifconfig tap0 up
      /usr/sbin/brctl addif br0 tap0

  • dd if=/dev/zero of=/usr1/local/storage/commonstg1.img bs=1M seek=204800 count=0 #200GB sparse iSCSI storage image
  • vi /etc/tgt/targets.conf and append

    • <target iqn.2012-09.uk.ac.dmu.blue:commonstg1>
       initiator-address 146.227.XX.XX
       <direct-store /usr1/local/storage/commonstg1.img>
       lun 1
       </direct-store>
      </target>

• Creating a 20GB (generous today) VM for MySQL
  • virt-install –connect qemu:///system -n CommonsDBVM -r 2048 –vcpus=2 –disk path=/usr1/local/vm/CommonsDBVM.img,size=20 -c /usr/local/iso/CentOS-6.3-x86_64-netinstall.iso –vnc –noautoconsole –os-type linux –os-variant rhel6 –accelerate –network=bridge:br0 –hvm
  • Set up networking (manually)
  • yum update, install rsync wget, groupinstall “MySQL Database client” “MySQL Database server”
  • Change SSHD config to allow passphrases but not passwords (having applied public keys to authorized_keys)
  • (make a decision about SELinux)
  • grab a database backup (and Commons WP backup) for testing the install
    • set MySQL’s admin password
    • change iptables to reflect MySQL comms with Common’s web server
    • mysql -u root -p < backup.sql
    • grant ownership to database for user remembering remote hostname
• Creating a 20GB (generous today) VM for Commons Web (storage will be iSCSI on VMs’ host for now)
  • yum update, install rsync wget, groupinstall “Web Server”, php-pecl-apc
  • Change SSHD config to allow passphrases but not passwords (having applied public keys to authorized_keys)
  • (make a decision about SELinux)
  • iscsiadm –mode discovery –type sendtargets –portal 146.227.XX.XX
  • iscsiadm –mode node –targetname iqn.2012-09.uk.ac.dmu.blue:commonstg1 –portal 146.227.XX.XX –login
  • tail -f /var/log/messages (to verify and discover your new disk)
  • service iscsi restart (for a laugh)
  • fdisk -l /dev/yourNewDisk (sda for me)
    • create one partition the size of the disk and set to LVM
  • pvcreate /dev/sda1
  • vgcreate vg_commonswebdata /dev/sda1
  • lvcreate -L 199G -nusr1lv vg_commonswebdata
  • mkfs.ext4 -L /usr1 /dev/vg_commonswebdata/usr1lv
  • tune2fs -c -1 -i -1 -m 1 /dev/vg_commonswebdata/usr1lv
  • Add entry to /etc/fstab

    • LABEL=/usr1 /usr1  ext4    _netdev 0 0

  • Reboot the VM with fingers crossed the storage comes up and gets mounted.  For me, wowzers, it’s there.
• Back on the host list the VMs and make them boot at boot:

• # virsh list
   Id Name State
  ----------------------------------------------------
   1 CommonsDBVM running
   2 CommonsWebVM running

  [root@winsor ~]# virsh autostart CommonsDBVM
  Domain CommonsDBVM marked as autostarted

  [root@winsor ~]# virsh autostart CommonsWebVM
  Domain CommonsWebVM marked as autostarted

• Reboot the host and check the VMs come up with the iSCSI storage mounted.  For me “Wuhoo!”
• Back on CommonsWeb:
  • yum install php-mysql mysql php-ldap and check connection to remote database
  • Configured Apache to reflect current Commons website
  • Changed /etc/hosts to include IP/hostnames for testing
  • restored WP backup (correlating to database restore)
• Mustn’t forget back ups of host and VMs.  We use Amanda which is super good for Linux and web servers.  Large media files and large VM images are a problem but for now we’re living with it.  On the VMs’ host I exclude ISOs, VMs and the storage as I’m backing up the VMs’ file systems.  This avoids backing up large images because of a small change to a VM and keeps data related to the host in one place.
• Tune PHP and MySQL
  • MySQL first:

  • [mysqld]
    datadir=/var/lib/mysql
    socket=/var/lib/mysql/mysql.sock
    user=mysql
    # Disabling symbolic-links is recommended to prevent assorted security risks
    symbolic-links=0

    query_cache_limit=1M
    query_cache_size=32M
    query_cache_type=1
    max_connections=3000
    max_user_connections=600
    interactive_timeout=100
    wait_timeout=100
    connect_timeout=10
    thread_cache_size=128
    key_buffer=256M # 64M for 1GB, 128M for 2GB, 256 for 4GB
    join_buffer_size=4M # 1M for 1GB, 2M for 2GB, 4M for 4GB
    max_allowed_packet=32M
    table_cache=1024
    sort_buffer_size=4M # 1M for 1GB, 2M for 2GB, 4M for 4GB
    read_buffer_size=4M # 1M for 1GB, 2M for 2GB, 4M for 4GB
    read_rnd_buffer_size=3072K # 768K for 1GB, 1536K for 2GB, 3072K for 4GB
    max_connect_errors=10
    thread_concurrency=4
    myisam_sort_buffer_size=128M # 32M for 1GB, 64M for 2GB, 128 for 4GB
    skip-locking

    [mysqld_safe]
    log-error=/var/log/mysqld.log
    pid-file=/var/run/mysqld/mysqld.pid

    [isamchk]
    key_buffer=256M # 64M for 1GB, 128M for 2GB, 256M for 4GB
    sort_buffer=256M # 64M for 1GB, 128M for 2GB, 256M for 4GB
    read_buffer=64M # 16M for 1GB, 32M for 2GB, 64M for 4GB
    write_buffer=64M # 16M for 1GB, 32M for 2GB, 64M for 4GB

    [myisamchk]
    key_buffer=256M # 64M for 1GB, 128M for 2GB, 256M for 4GB
    sort_buffer=256M # 64M for 1GB, 128M for 2GB, 256M for 4GB
    read_buffer=64M # 16M for 1GB, 32M for 2GB, 64M for 4GB
    write_buffer=64M # 16M for 1GB, 32M for 2GB, 64M for 4GB

Hmmm…I seem to have a working copy of our website spread across two VMs.