Project:Infrastructure/fbsd notes

From Gentoo Wiki
Jump to:navigation Jump to:search

Setup notes from fbsd.amd64.dev.gentoo.org

LVM (ganeti host#1)

  • The disk image is (21GiB+64KiB)
  • 21508MiB is the next 4MB boundary after the size of the disk image.
  • lvcreate -n fbsd.amd64.os -L 21508m -m 1 vg /dev/sda4 /dev/sdb4 --type raid1
  • lvcreate -n fbsd.amd64.scratch -L 150g -m 1 -i 2 vg --type raid10 /dev/sdc1 /dev/sdd1 /dev/sde1 /dev/sdf1

Populate disk (ganeti host#1)

  • qemu-img convert -p -O host_device FreeBSD-11.1-RELEASE-amd64.qcow2 /dev/vg/fbsd.amd64.os

Ganeti (ganeti host#1)

  • You cannot adopt into DRBD disk mode, we will convert to DRBD later
  • gnt-instance add -t plain -o noop --disk=0:adopt=fbsd.amd64.os --disk=1:adopt=fbsd.amd64.scratch --net=0:network=vlan103 -g supermicro --ignore-ipolicy --debug -B vcpus=4,minmem=1G,maxmem=8G,memory=4G -n $GNT_NODE1 --no-start --no-name-check --no-ip-check fbsd.amd64.dev.gentoo.org

Access Spice console (ganeti host#1)

  • ssh -D9500 $GNT_NODE
  • SOCKS_SERVER=127.0.0.1:9500 socksify spicy --host=$GNT_NODE1 --port=$VM_PORT

Fixup parts of the OS (VM, via Spice)

  • Ctrl-C to break waiting for DHCP (no DHCPv4 in our env)
  • Set root password: passwd
  • Enable serial console: echo '-Dh' >/boot.config
  • Disable IPv4 DHCP: sed -i -e '/^ifconfig_DEFAULT/s,DHCP ,,' /etc/rc.conf
  • Enable SSH: echo sshd_enable=YES /etc/rc.conf
  • Grab IPv6 address: ifconfig
  • Allow SSH-as-root-for-now: sed -i -e '/^#?PermitRootLogin/s,no,yes,g' /etc/ssh/sshd_config
  • Restart SSH: service sshd restart
  • Reboot to verify: reboot

Verify serial (ganeti host#1)

  • gnt-instance console $VM
  • (Ctrl-] to break.

Verify SSH

  • ssh -6 root@$IPV6_ADDRESS

Fixup to DRBD (ganeti host#1)

  • gnt-instance stop fbsd
  • gnt-instance modify -t drbd -n $GNT_NODE2 --no-wait-for-sync fbsd


Convert slave disks to RAID (ganeti host#2)

  • By doing this now, the DRBD conversion above will complete MUCH faster.
  • Ensure the data is on the correct set of disks to start
  • Move OS to first ssd from spinning disk: pvmove -i 5 -n vg/${ID}.disk0_data ${SRC_SPIN1} ${DEST_SSD1}
  • Move scratch to a spinning disk: TODO
  • Convert OS to RAID1, using 2nd SSD: lvconvert --type raid1 -m 1 vg/${ID}.disk0_data ${DEST_SSD2}
  • Convert Scratch to RAID10, 2-way mirror, 2 stripes = 4 disks: -m 1 -i 2
  • lvconvert --type raid10 -m 1 -i 2 vg/${ID}.disk1_data ${DEST_SPIN2} ${DEST_SPIN3} ${DEST_SPIN4}

Fixup to DRBD (ganeti host#1)

  • Wait for DRBD completion on the OS disk at least
  • gnt-job list
  • gnt-job watch $JOBID
  • gnt-instance start fbsd

Bring up ZFS (VM)

  • echo zfs_enable=YES >>/etc/rc.conf
  • service zfs start
  • zpool create pool0 /dev/vtbd1