Workstation 38 UEFI

This guide can be used to install Fedora onto a single disk with or without ZFS encryption.

It assumes the following:

Your system uses UEFI to boot
Your system is x86_64
You're mildly comfortable with ZFS, EFI and discovering system facts on your own (lsblk, dmesg, gdisk, ...)

Download Fedora Workstation Live , write it to a USB drive and boot your system in EFI mode.

Confirm EFI support:

# dmesg | grep -i efivars
[    0.301784] Registered efivars operations

Configure Live Environment

Switch to a root account

sudo -i

Source `/etc/os-release`

The file /etc/os-release defines variables that describe the running distribution. In particular, the $ID variable defined within can be used as a short name for the filesystem that will hold this installation.

source /etc/os-release
export ID

Install updated ZFS packages

rpm -e --nodeps zfs-fuse
dnf install -y https://zfsonlinux.org/fedora/zfs-release-2-3$(rpm --eval "%{dist}").noarch.rpm
dnf install -y https://dl.fedoraproject.org/pub/fedora/linux/releases/${VERSION_ID}/Everything/x86_64/os/Packages/k/kernel-devel-$(uname -r).rpm
dnf install -y zfs
modprobe zfs

Generate `/etc/hostid`

zgenhostid -f 0x00bab10c

Define disk variables

For convenience and to reduce the likelihood of errors, set environment variables that refer to the devices that will be configured during the setup.

For many users, it is most convenient to place boot files (i.e., ZFSBootMenu and any loader responsible for launching it) on the the same disk that will hold the ZFS pool. However, some users may wish to dedicate an entire disk to the ZFS pool or create a multi-disk pool. A USB flash drive provides a convenient location for the boot partition. Fortunately, this alternative configuration is easily realized by simply defining a few environment variables differently.

Verify your target disk devices with lsblk. /dev/sda, /dev/sdb and /dev/nvme0n1 used below are examples.

First, define variables that refer to the disk and partition number that will hold boot files:

export BOOT_DISK="/dev/sda"
export BOOT_PART="1"
export BOOT_DEVICE="${BOOT_DISK}${BOOT_PART}"

export BOOT_DISK="/dev/nvme0n1"
export BOOT_PART="1"
export BOOT_DEVICE="${BOOT_DISK}p${BOOT_PART}"

export BOOT_DISK="/dev/sdb"
export BOOT_PART="1"
export BOOT_DEVICE="${BOOT_DISK}${BOOT_PART}"

Next, define variables that refer to the disk and partition number that will hold the ZFS pool:

export POOL_DISK="/dev/sda"
export POOL_PART="2"
export POOL_DEVICE="${POOL_DISK}${POOL_PART}"

export POOL_DISK="/dev/nvme0n1"
export POOL_PART="2"
export POOL_DEVICE="${POOL_DISK}p${POOL_PART}"

export POOL_DISK="/dev/sda"
export POOL_PART="1"
export POOL_DEVICE="${POOL_DISK}${POOL_PART}"

Disk preparation

Wipe partitions

wipefs -a "$POOL_DISK"
wipefs -a "$BOOT_DISK"

sgdisk --zap-all "$POOL_DISK"
sgdisk --zap-all "$BOOT_DISK"

Create EFI boot partition

sgdisk -n "${BOOT_PART}:1m:+512m" -t "${BOOT_PART}:ef00" "$BOOT_DISK"

Create zpool partition

sgdisk -n "${POOL_PART}:0:-10m" -t "${POOL_PART}:bf00" "$POOL_DISK"

ZFS pool creation

Create the zpool

zpool create -f -o ashift=12 \
 -O compression=lz4 \
 -O acltype=posixacl \
 -O xattr=sa \
 -O relatime=on \
 -o autotrim=on \
 -o compatibility=openzfs-2.1-linux \
 -m none zroot "$POOL_DEVICE"

echo 'SomeKeyphrase' > /etc/zfs/zroot.key
chmod 000 /etc/zfs/zroot.key

zpool create -f -o ashift=12 \
 -O compression=lz4 \
 -O acltype=posixacl \
 -O xattr=sa \
 -O relatime=on \
 -O encryption=aes-256-gcm \
 -O keylocation=file:///etc/zfs/zroot.key \
 -O keyformat=passphrase \
 -o autotrim=on \
 -o compatibility=openzfs-2.1-linux \
 -m none zroot "$POOL_DEVICE"

Note

It's out of the scope of this guide to cover all of the pool creation options used - feel free to tailor them to suit your system. However, the following options need to be addressed:

encryption=aes-256-gcm - You can adjust the algorithm as you see fit, but this will likely be the most performant on modern x86_64 hardware.
keylocation=file:///etc/zfs/zroot.key - This sets our pool encryption passphrase to the file /etc/zfs/zroot.key, which we created in a previous step. This file will live inside your initramfs stored on the ZFS boot environment.
keyformat=passphrase - By setting the format to passphrase, we can now force a prompt for this in zfsbootmenu. It's critical that your passphrase be something you can type on your keyboard, since you will need to type it in to unlock the pool on boot.

Note

The option -o compatibility=openzfs-2.1-linux ensures that the pool is created only with feature flags supported by the current ZFSBootMenu binary release. If you plan on building a custom ZFSBootMenu image that you will keep synchronized with your host, the compatibility option may be omitted.

Binary releases of ZFSBootMenu are generally built with the latest stable version of ZFS. Future releases of ZFSBootMenu may therefore support newer feature sets. Check project release notes prior to updating or removing compatibility options and upgrading your system pool.

Create initial file systems

zfs create -o mountpoint=none zroot/ROOT
zfs create -o mountpoint=/ -o canmount=noauto zroot/ROOT/${ID}
zfs create -o mountpoint=/home zroot/home

zpool set bootfs=zroot/ROOT/${ID} zroot

Note

It is important to set the property canmount=noauto on any file systems with mountpoint=/ (that is, on any additional boot environments you create). Without this property, the OS will attempt to automount all ZFS file systems and fail when multiple file systems attempt to mount at /; this will prevent your system from booting. Automatic mounting of / is not required because the root file system is explicitly mounted in the boot process.

Also note that, unlike many ZFS properties, canmount is not inheritable. Therefore, setting canmount=noauto on zroot/ROOT is not sufficient, as any subsequent boot environments you create will default to canmount=on. It is necessary to explicitly set the canmount=noauto on every boot environment you create.

Export, then re-import with a temporary mountpoint of `/mnt`

zpool export zroot
zpool import -N -R /mnt zroot
zfs mount zroot/ROOT/${ID}
zfs mount zroot/home

zpool export zroot
zpool import -N -R /mnt zroot
zfs load-key -L prompt zroot

zfs mount zroot/ROOT/${ID}
zfs mount zroot/home

Verify that everything is mounted correctly

# mount | grep mnt
zroot/ROOT/fedora on /mnt type zfs (rw,relatime,xattr,posixacl)
zroot/home on /mnt/home type zfs (rw,relatime,xattr,posixacl)

Update device symlinks

udevadm trigger

Install Fedora

mkdir /run/install
mount /dev/mapper/live-base /run/install

rsync -pogAXtlHrDx \
 --stats \
 --exclude=/boot/efi/* \
 --exclude=/etc/machine-id \
 --info=progress2 \
 /run/install/ /mnt

Copy files into the new install

mv /mnt/etc/resolv.conf /mnt/etc/resolv.conf.orig
cp -L /etc/resolv.conf /mnt/etc
cp /etc/hostid /mnt/etc

mv /mnt/etc/resolv.conf /mnt/etc/resolv.conf.orig
cp /etc/hostid /mnt/etc
cp -L /etc/resolv.conf /mnt/etc
mkdir -p /mnt/etc/zfs
cp /etc/zfs/zroot.key /mnt/etc/zfs

Chroot into the new OS

mount -t proc proc /mnt/proc
mount -t sysfs sys /mnt/sys
mount -B /dev /mnt/dev
mount -t devpts pts /mnt/dev/pts
chroot /mnt /bin/bash

ZFS Configuration

Configure Dracut to load ZFS support

cat << EOF > /etc/dracut.conf.d/zol.conf
nofsck="yes"
add_dracutmodules+=" zfs "
omit_dracutmodules+=" btrfs "
EOF

cat << EOF > /etc/dracut.conf.d/zol.conf
nofsck="yes"
add_dracutmodules+=" zfs "
omit_dracutmodules+=" btrfs "
install_items+=" /etc/zfs/zroot.key "
EOF

Install required packages

source /etc/os-release

rpm -e --nodeps zfs-fuse

dnf install -y https://dl.fedoraproject.org/pub/fedora/linux/releases/${VERSION_ID}/Everything/x86_64/os/Packages/k/kernel-devel-$(uname -r).rpm

dnf --releasever=${VERSION_ID} install -y \
  https://zfsonlinux.org/fedora/zfs-release-2-3$(rpm --eval "%{dist}").noarch.rpm

dnf install -y zfs zfs-dracut

Install and configure ZFSBootMenu

Set ZFSBootMenu properties on datasets

Assign command-line arguments to be used when booting the final kernel. Because ZFS properties are inherited, assign the common properties to the ROOT dataset so all children will inherit common arguments by default.

zfs set org.zfsbootmenu:commandline="quiet loglevel=4 rhgb" zroot/ROOT

Assign command-line arguments to be used when booting the final kernel. Because ZFS properties are inherited, assign the common properties to the ROOT dataset so all children will inherit common arguments by default.

zfs set org.zfsbootmenu:commandline="quiet loglevel=4 rhgb" zroot/ROOT

Setup key caching in ZFSBootMenu.

zfs set org.zfsbootmenu:keysource="zroot/ROOT/${ID}" zroot

Create a `vfat` filesystem

mkfs.vfat -F32 "$BOOT_DEVICE"

Create an fstab entry and mount

cat << EOF >> /etc/fstab
$( blkid | grep "$BOOT_DEVICE" | cut -d ' ' -f 2 ) /boot/efi vfat defaults 0 0
EOF

mkdir -p /boot/efi
mount /boot/efi

Install ZFSBootMenu

Fetch a prebuilt ZFSBootMenu EFI executable, saving it to the EFI system partition:

mkdir -p /boot/efi/EFI/ZBM
curl -o /boot/efi/EFI/ZBM/VMLINUZ.EFI -L https://get.zfsbootmenu.org/efi
cp /boot/efi/EFI/ZBM/VMLINUZ.EFI /boot/efi/EFI/ZBM/VMLINUZ-BACKUP.EFI

Install all packages required to build a ZFSBootMenu image on Fedora:

dnf install -y \
  systemd-boot-unsigned \
  perl-YAML-PP \
  perl-Sort-Versions \
  perl-boolean \
  git \
  fzf \
  mbuffer \
  kexec-tools

Download the latest ZFSBootMenu source and install on the host:

mkdir -p /usr/local/src/zfsbootmenu
cd /usr/local/src/zfsbootmenu
curl -L https://get.zfsbootmenu.org/source | tar -zxv --strip-components=1 -f -
make core dracut

Configure generate-zbm(5) by ensuring that the following keys appear in /etc/zfsbootmenu/config.yaml:

Global:
  ManageImages: true
  BootMountPoint: /boot/efi
Components:
   Enabled: false
EFI:
  ImageDir: /boot/efi/EFI/zbm
  Versions: false
  Enabled: true
Kernel:
  CommandLine: quiet loglevel=0

Create a ZFSBootMenu image:

generate-zbm

Configure EFI boot entries

mount -t efivarfs efivarfs /sys/firmware/efi/efivars

efibootmgr -c -d "$BOOT_DISK" -p "$BOOT_PART" \
  -L "ZFSBootMenu (Backup)" \
  -l \\EFI\\ZBM\\VMLINUZ-BACKUP.EFI

efibootmgr -c -d "$BOOT_DISK" -p "$BOOT_PART" \
  -L "ZFSBootMenu" \
  -l \\EFI\\ZBM\\VMLINUZ.EFI

dnf install -y refind

refind-install
rm /boot/refind_linux.conf

cat << EOF > /boot/efi/EFI/ZBM/refind_linux.conf
"Boot default"  "quiet loglevel=0 zbm.skip"
"Boot to menu"  "quiet loglevel=0 zbm.show"
EOF

Reset resolv.conf

mv /etc/resolv.conf.orig /etc/resolv.conf

Prepare for first boot

Exit the chroot, unmount everything

exit

umount -n -R /mnt

Export the zpool and reboot

zpool export zroot
reboot

Workstation 38 UEFI

Configure Live Environment

Switch to a root account

Source /etc/os-release

Install updated ZFS packages

Generate /etc/hostid

Define disk variables

Disk preparation

Wipe partitions

Create EFI boot partition

Create zpool partition

ZFS pool creation

Create the zpool

Create initial file systems

Export, then re-import with a temporary mountpoint of /mnt

Verify that everything is mounted correctly

Update device symlinks

Install Fedora

Copy files into the new install

Chroot into the new OS

ZFS Configuration

Configure Dracut to load ZFS support

Install required packages

Install and configure ZFSBootMenu

Set ZFSBootMenu properties on datasets

Create a vfat filesystem

Create an fstab entry and mount

Install ZFSBootMenu

Configure EFI boot entries

Reset resolv.conf

Prepare for first boot

Exit the chroot, unmount everything

Export the zpool and reboot

Source `/etc/os-release`

Generate `/etc/hostid`

Export, then re-import with a temporary mountpoint of `/mnt`

Create a `vfat` filesystem