Jammy (22.04) UEFI#

This guide can be used to install Ubuntu onto a single disk with or without ZFS encryption.

The end result will be a pristine Ubuntu install with no GUI or anything other than the base system. You'll be able to install ubuntu-desktop, ubuntu-server-minimal or whatever takes your fancy afterwards.

It assumes the following:

Your system uses UEFI to boot
Your system is x86_64
You're mildly comfortable with ZFS, EFI and discovering system facts on your own (lsblk, dmesg, gdisk, ...)

Download the latest Ubuntu Desktop Jammy (22.04) Live image, write it to a USB drive and boot your system in EFI mode. You can use the server installation media if you want, although instructions are provided for installation using the desktop installer live session.

Configure Live Environment#

Open a root shell#

Open a terminal on the live installer session, then:

sudo -i

Confirm EFI support:

# dmesg | grep -i efivars
[    0.301784] Registered efivars operations

Source `/etc/os-release`#

The file /etc/os-release defines variables that describe the running distribution. In particular, the $ID variable defined within can be used as a short name for the filesystem that will hold this installation.

source /etc/os-release
export ID

Install helpers#

apt update
apt install debootstrap gdisk zfsutils-linux

Generate `/etc/hostid`#

zgenhostid -f 0x00bab10c

Define disk variables#

For convenience and to reduce the likelihood of errors, set environment variables that refer to the devices that will be configured during the setup.

For many users, it is most convenient to place boot files (i.e., ZFSBootMenu and any loader responsible for launching it) on the the same disk that will hold the ZFS pool. However, some users may wish to dedicate an entire disk to the ZFS pool or create a multi-disk pool. A USB flash drive provides a convenient location for the boot partition. Fortunately, this alternative configuration is easily realized by simply defining a few environment variables differently.

Verify your target disk devices with lsblk. /dev/sda, /dev/sdb and /dev/nvme0n1 used below are examples.

First, define variables that refer to the disk and partition number that will hold boot files:

export BOOT_DISK="/dev/sda"
export BOOT_PART="1"
export BOOT_DEVICE="${BOOT_DISK}${BOOT_PART}"

export BOOT_DISK="/dev/nvme0n1"
export BOOT_PART="1"
export BOOT_DEVICE="${BOOT_DISK}p${BOOT_PART}"

export BOOT_DISK="/dev/sdb"
export BOOT_PART="1"
export BOOT_DEVICE="${BOOT_DISK}${BOOT_PART}"

Next, define variables that refer to the disk and partition number that will hold the ZFS pool:

export POOL_DISK="/dev/sda"
export POOL_PART="2"
export POOL_DEVICE="${POOL_DISK}${POOL_PART}"

export POOL_DISK="/dev/nvme0n1"
export POOL_PART="2"
export POOL_DEVICE="${POOL_DISK}p${POOL_PART}"

export POOL_DISK="/dev/sda"
export POOL_PART="1"
export POOL_DEVICE="${POOL_DISK}${POOL_PART}"

Disk preparation#

Wipe partitions#

zpool labelclear -f "$POOL_DISK"

wipefs -a "$POOL_DISK"
wipefs -a "$BOOT_DISK"

sgdisk --zap-all "$POOL_DISK"
sgdisk --zap-all "$BOOT_DISK"

Create EFI boot partition#

sgdisk -n "${BOOT_PART}:1m:+512m" -t "${BOOT_PART}:ef00" "$BOOT_DISK"

Create zpool partition#

sgdisk -n "${POOL_PART}:0:-10m" -t "${POOL_PART}:bf00" "$POOL_DISK"

ZFS pool creation#

Create the zpool#

zpool create -f -o ashift=12 \
 -O compression=lz4 \
 -O acltype=posixacl \
 -O xattr=sa \
 -O relatime=on \
 -o autotrim=on \
 -o compatibility=openzfs-2.1-linux \
 -m none zroot "$POOL_DEVICE"

echo 'SomeKeyphrase' > /etc/zfs/zroot.key
chmod 000 /etc/zfs/zroot.key

zpool create -f -o ashift=12 \
 -O compression=lz4 \
 -O acltype=posixacl \
 -O xattr=sa \
 -O relatime=on \
 -O encryption=aes-256-gcm \
 -O keylocation=file:///etc/zfs/zroot.key \
 -O keyformat=passphrase \
 -o autotrim=on \
 -o compatibility=openzfs-2.1-linux \
 -m none zroot "$POOL_DEVICE"

Note

It's out of the scope of this guide to cover all of the pool creation options used - feel free to tailor them to suit your system. However, the following options need to be addressed:

encryption=aes-256-gcm - You can adjust the algorithm as you see fit, but this will likely be the most performant on modern x86_64 hardware.
keylocation=file:///etc/zfs/zroot.key - This sets our pool encryption passphrase to the file /etc/zfs/zroot.key, which we created in a previous step. This file will live inside your initramfs stored on the ZFS boot environment.
keyformat=passphrase - By setting the format to passphrase, we can now force a prompt for this in zfsbootmenu. It's critical that your passphrase be something you can type on your keyboard, since you will need to type it in to unlock the pool on boot.

Note

The option -o compatibility=openzfs-2.1-linux ensures that the pool is created only with feature flags supported by the current ZFSBootMenu binary release. If you plan on building a custom ZFSBootMenu image that you will keep synchronized with your host, the compatibility option may be omitted.

Binary releases of ZFSBootMenu are generally built with the latest stable version of ZFS. Future releases of ZFSBootMenu may therefore support newer feature sets. Check project release notes prior to updating or removing compatibility options and upgrading your system pool.

Create initial file systems#

zfs create -o mountpoint=none zroot/ROOT
zfs create -o mountpoint=/ -o canmount=noauto zroot/ROOT/${ID}
zfs create -o mountpoint=/home zroot/home

zpool set bootfs=zroot/ROOT/${ID} zroot

Note

It is important to set the property canmount=noauto on any file systems with mountpoint=/ (that is, on any additional boot environments you create). Without this property, the OS will attempt to automount all ZFS file systems and fail when multiple file systems attempt to mount at /; this will prevent your system from booting. Automatic mounting of / is not required because the root file system is explicitly mounted in the boot process.

Also note that, unlike many ZFS properties, canmount is not inheritable. Therefore, setting canmount=noauto on zroot/ROOT is not sufficient, as any subsequent boot environments you create will default to canmount=on. It is necessary to explicitly set the canmount=noauto on every boot environment you create.

Export, then re-import with a temporary mountpoint of `/mnt`#

zpool export zroot
zpool import -N -R /mnt zroot
zfs mount zroot/ROOT/${ID}
zfs mount zroot/home

zpool export zroot
zpool import -N -R /mnt zroot
zfs load-key -L prompt zroot

zfs mount zroot/ROOT/${ID}
zfs mount zroot/home

Verify that everything is mounted correctly#

# mount | grep mnt
zroot/ROOT/ubuntu on /mnt type zfs (rw,relatime,xattr,posixacl)
zroot/home on /mnt/home type zfs (rw,relatime,xattr,posixacl)

Update device symlinks#

udevadm trigger

Install Ubuntu#

debootstrap jammy /mnt

Copy files into the new install#

cp /etc/hostid /mnt/etc
cp /etc/resolv.conf /mnt/etc

cp /etc/hostid /mnt/etc/hostid
cp /etc/resolv.conf /mnt/etc/
mkdir /mnt/etc/zfs
cp /etc/zfs/zroot.key /mnt/etc/zfs

Chroot into the new OS#

mount -t proc proc /mnt/proc
mount -t sysfs sys /mnt/sys
mount -B /dev /mnt/dev
mount -t devpts pts /mnt/dev/pts
chroot /mnt /bin/bash

Basic Ubuntu Configuration#

Set a hostname#

echo 'YOURHOSTNAME' > /etc/hostname
echo -e '127.0.1.1\tYOURHOSTNAME' >> /etc/hosts

Set a root password#

passwd

Configure apt. Use other mirrors if you prefer.

cat <<EOF > /etc/apt/sources.list
# Uncomment the deb-src entries if you need source packages

deb http://archive.ubuntu.com/ubuntu/ jammy main restricted universe multiverse
# deb-src http://archive.ubuntu.com/ubuntu/ jammy main restricted universe multiverse

deb http://archive.ubuntu.com/ubuntu/ jammy-updates main restricted universe multiverse
# deb-src http://archive.ubuntu.com/ubuntu/ jammy-updates main restricted universe multiverse

deb http://archive.ubuntu.com/ubuntu/ jammy-security main restricted universe multiverse
# deb-src http://archive.ubuntu.com/ubuntu/ jammy-security main restricted universe multiverse

deb http://archive.ubuntu.com/ubuntu/ jammy-backports main restricted universe multiverse
# deb-src http://archive.ubuntu.com/ubuntu/ jammy-backports main restricted universe multiverse

deb http://archive.canonical.com/ubuntu/ jammy partner
# deb-src http://archive.canonical.com/ubuntu/ jammy partner
EOF

Update the repository cache and system#

apt update
apt upgrade

Install additional base packages#

apt install --no-install-recommends linux-generic locales keyboard-configuration console-setup

Note

The --no-install-recommends flag is used here to avoid installing recommended, but not strictly needed, packages (including grub2).

Configure packages to customize local and console properties#

dpkg-reconfigure locales tzdata keyboard-configuration console-setup

Note

You should always enable the en_US.UTF-8 locale because some programs require it.

ZFS Configuration#

Install required packages#

apt install dosfstools zfs-initramfs zfsutils-linux

Enable systemd ZFS services#

systemctl enable zfs.target
systemctl enable zfs-import-cache
systemctl enable zfs-mount
systemctl enable zfs-import.target

Configure `initramfs-tools`#

No required steps

echo "UMASK=0077" > /etc/initramfs-tools/conf.d/umask.conf

Note

Because the encryption key is stored in /etc/zfs directory, it will automatically be copied into the system initramfs.

Rebuild the initramfs#

update-initramfs -c -k all

Install and configure ZFSBootMenu#

Set ZFSBootMenu properties on datasets#

Assign command-line arguments to be used when booting the final kernel. Because ZFS properties are inherited, assign the common properties to the ROOT dataset so all children will inherit common arguments by default.

zfs set org.zfsbootmenu:commandline="quiet loglevel=4" zroot/ROOT

zfs set org.zfsbootmenu:commandline="quiet loglevel=4" zroot/ROOT

Setup key caching in ZFSBootMenu.

zfs set org.zfsbootmenu:keysource="zroot/ROOT/${ID}" zroot

Create a `vfat` filesystem#

mkfs.vfat -F32 "$BOOT_DEVICE"

Create an fstab entry and mount#

cat << EOF >> /etc/fstab
$( blkid | grep "$BOOT_DEVICE" | cut -d ' ' -f 2 ) /boot/efi vfat defaults 0 0
EOF

mkdir -p /boot/efi
mount /boot/efi

Install ZFSBootMenu#

apt install curl

Fetch a prebuilt ZFSBootMenu EFI executable, saving it to the EFI system partition:

mkdir -p /boot/efi/EFI/ZBM
curl -o /boot/efi/EFI/ZBM/VMLINUZ.EFI -L https://get.zfsbootmenu.org/efi
cp /boot/efi/EFI/ZBM/VMLINUZ.EFI /boot/efi/EFI/ZBM/VMLINUZ-BACKUP.EFI

Install all packages required to build a ZFSBootMenu image on Ubuntu:

apt install \
  libsort-versions-perl \
  libboolean-perl \
  libyaml-pp-perl \
  git \
  fzf \
  make \
  mbuffer \
  kexec-tools \
  dracut-core \
  efibootmgr \
  bsdextrautils

Note

Choose 'No' when asked if kexec-tools should handle reboots.

Download the latest ZFSBootMenu source and install on the host:

mkdir -p /usr/local/src/zfsbootmenu
cd /usr/local/src/zfsbootmenu
curl -L https://get.zfsbootmenu.org/source | tar -zxv --strip-components=1 -f -
make core dracut

Configure generate-zbm(5) by ensuring that the following keys appear in /etc/zfsbootmenu/config.yaml:

Global:
  ManageImages: true
  BootMountPoint: /boot/efi
Components:
   Enabled: false
EFI:
  ImageDir: /boot/efi/EFI/zbm
  Versions: false
  Enabled: true
Kernel:
  CommandLine: quiet loglevel=0

Create a ZFSBootMenu image:

generate-zbm

Configure EFI boot entries#

mount -t efivarfs efivarfs /sys/firmware/efi/efivars

apt install efibootmgr

efibootmgr -c -d "$BOOT_DISK" -p "$BOOT_PART" \
  -L "ZFSBootMenu (Backup)" \
  -l '\EFI\ZBM\VMLINUZ-BACKUP.EFI'

efibootmgr -c -d "$BOOT_DISK" -p "$BOOT_PART" \
  -L "ZFSBootMenu" \
  -l '\EFI\ZBM\VMLINUZ.EFI'

apt install refind

refind-install
rm /boot/refind_linux.conf

cat << EOF > /boot/efi/EFI/ZBM/refind_linux.conf
"Boot default"  "quiet loglevel=0 zbm.skip"
"Boot to menu"  "quiet loglevel=0 zbm.show"
EOF

Prepare for first boot#

Exit the chroot, unmount everything#

exit

umount -n -R /mnt

Export the zpool and reboot#

zpool export zroot
reboot

Jammy (22.04) UEFI

Contents

Jammy (22.04) UEFI#

Configure Live Environment#

Open a root shell#

Source /etc/os-release#

Install helpers#

Generate /etc/hostid#

Define disk variables#

Disk preparation#

Wipe partitions#

Create EFI boot partition#

Create zpool partition#

ZFS pool creation#

Create the zpool#

Create initial file systems#

Export, then re-import with a temporary mountpoint of /mnt#

Verify that everything is mounted correctly#

Update device symlinks#

Install Ubuntu#

Copy files into the new install#

Chroot into the new OS#

Basic Ubuntu Configuration#

Set a hostname#

Set a root password#

Update the repository cache and system#

Install additional base packages#

Configure packages to customize local and console properties#

ZFS Configuration#

Install required packages#

Enable systemd ZFS services#

Configure initramfs-tools#

Rebuild the initramfs#

Install and configure ZFSBootMenu#

Set ZFSBootMenu properties on datasets#

Create a vfat filesystem#

Create an fstab entry and mount#

Install ZFSBootMenu#

Configure EFI boot entries#

Prepare for first boot#

Exit the chroot, unmount everything#

Export the zpool and reboot#

Source `/etc/os-release`#

Generate `/etc/hostid`#

Export, then re-import with a temporary mountpoint of `/mnt`#

Configure `initramfs-tools`#

Create a `vfat` filesystem#