Remote Access to ZFSBootMenu
Having SSH access to ZFSBootMenu can be critical because it allows some measure of recovery over a remote connection. If your boot environments reside in encrypted filesystems, SSH access is necessary if you ever intend to reboot a machine when you are not physically present. Because ZFSBootMenu supports Dracut and mkinitcpio, any mechanism that can provide remote access to a Dracut or mkinitcpio initramfs will work.
Dracut
The dracut-crypt-ssh provides a straightforward approach to
configuring and launching an SSH server in Dracut images. The module is packaged in Void and does not rely on
systemd
within the initramfs. If you run a distribution that does not package dracut-crypt-ssh
, you will need to
track down its dependencies. The dracut-network
module and dropbear
are required to provide network access and
an SSH server, respectively; other prerequisites are probably already installed on your system.
Simplified Installation Instructions
The dracut-crypt-ssh
package comes with a few helper utilities in the module/60crypt-ssh/helper
directory that
are designed to simplify providing passwords and snooping console output so that you can interact with unlock processes
that are already running in the initramfs. These components are not required for ZFSBootMenu and do not provide a lot of
value. If you have no problems installing the package as intended, it is OK to leave the helpers installed. If your
distribution has trouble compiling the helpers, just copy the contents of the 60crypt-ssh
directory, less the
helper
directory and Makefile
, to the modules directory for Dracut. This will most likely be
/usr/lib/dracut/modules.d/60crypt-ssh
.
If you do not install the contents of helper
, you may wish to edit the module-setup.sh
script provided by the
package to remove references to installing the helper. At the time of writing, these references consist of the last four
lines (five, if you count the harmless comment) of the install()
functioned. Removing these lines should not be
critical, as Dracut should happily continue the initramfs creation process even if those installation commands fail.
If you use Dracut to produce the initramfs images in your boot environment, you may wish to disable the crypt-ssh
module in those images. Just add
omit_dracutmodules+=" crypt-ssh "
to a configuration file in /etc/dracut.conf.d
. The configuration file must have a .conf
extension to be
recognized; see dracut.conf(5) for more information.
Configuring Dropbear in ZFSBootMenu
By default, dracut-crypt-ssh
will generate random host keys for your ZFSBootMenu initramfs. This is undesirable
because SSH will complain about unknown keys every time you reboot. If you wish, you can configure the module to copy
your regular host keys into the image. However, there are two problems with this:
The ZFSBootMenu image will generally be installed on a filesystem with no access permissions, allowing anybody to read your private host keys; and
The
dropbearconvert
program may be incapable of converting modern OpenSSH host keys into the required dropbear format.
To create dedicated host keys in the proper format, decide on a location, for example /etc/dropbear
, and create the
new keys:
mkdir -p /etc/dropbear
ssh-keygen -t rsa -m PEM -f /etc/dropbear/ssh_host_rsa_key
ssh-keygen -t ecdsa -m PEM -f /etc/dropbear/ssh_host_ecdsa_key
The module expects to install RSA and ECDSA keys, so both types are created here.
Note
When prompted for a passphrase when creating each host key, leave it blank. A non-empty password will prevent dropbear from reading a key.
To inform dracut-network
that it must bring up a network interface, pass the kernel command-line parameters
ip=dhcp
and rd.neednet=1
to your ZFSBootMenu image. If you use another boot loader to start ZFSBootMenu, e.g.
rEFInd or syslinux, this can be accomplished by configuring that loader. However, it may be more convenient to add these
parameters directly to the ZFSBootMenu image:
mkdir -p /etc/cmdline.d
echo "ip=dhcp rd.neednet=1" > /etc/cmdline.d/dracut-network.conf
It is possible to specify a static IP configuration by replacing dhcp
with a properly formatted configuration
string. Consult the dracut documentation for details about
static IP configuration.
There are methods besides writing to /etc/cmdline.d
or configuring another boot loader to specify kernel
command-line arguments that will configure networking in Dracut. However, Dracut uses the /etc/cmdline.d
directory
to store "fake" arguments, which it processes directly rather than handing to the kernel. In my tests, using other
methods (like adding these arguments to the kernel_cmdline
Dracut option for a UEFI bundle) can cause the
ip=dhcp
argument to appear more than once on the kernel command-line, which may cause dracut-network
to fail
catastrophically and refuse to boot. Writing a configuration file in /etc/cmdline.d
is a reliable way to ensure that
ip=dhcp
appears exactly once to dracut-network
.
With critical pieces in place, ZFSBootMenu can be configured to bundle dracut-crypt-ssh
in its images. Create the
Dracut configuration file /etc/zfsbootmenu/dracut.conf.d/dropbear.conf
with the following contents:
# Enable dropbear ssh server and pull in network configuration args
add_dracutmodules+=" crypt-ssh "
install_optional_items+=" /etc/cmdline.d/dracut-network.conf "
# Copy system keys for consistent access
dropbear_rsa_key=/etc/dropbear/ssh_host_rsa_key
dropbear_ecdsa_key=/etc/dropbear/ssh_host_ecdsa_key
# User zbmuser is the authorized unlocker here
dropbear_acl=/home/zbmuser/.ssh/authorized_keys
The last line is optional and assumes the user zbmuser
should provide an authorized_keys
file that will
determine remote access to the ZFSBootMenu image. The dracut-crypt-ssh
module does not allow for password
authentication over SSH; instead, key-based authentication is forced. By default, the list of authorized keys is taken
from /root/.ssh/authorized_keys
on the host. If you would prefer to use the authorized_keys
file from another
user on your system, copy the above example and replace zbmuser
with the name of the user whose authorized_keys
you wish to include.
Note
The default configuration will start dropbear on TCP port 222. This can be overridden with the dropbear_port
configuration option. Generally, you do not want the server listening on the default port 22. Clients that expect to
find your normal host keys when connecting to an SSH server on port 22 will refuse to connect when they find different
keys provided by dropbear.
Unless you've taken steps not described here, the network-enabled ZFSBootMenu image will not advertise itself via
dynamic DNS or mDNS. You will need to know the IP address of the ZFSBootMenu host to connect. Thus, you should either
configure a static IP address in /etc/cmdline.d/dracut-network.conf
or configure your DHCP server to reserve a known
address for the MAC address of the network interface you configure for dracut-crypt-ssh
.
mkinitcpio
ZFSBootMenu also supports the mkinitcpio initramfs generator used by Arch Linux.
ZFSBootMenu Configuration Changes
Since version 2.0.0, ZFSBootMenu will install a standard
mkinitcpio.conf in the /etc/zfsbootmenu
configuration directory. This file
is generally the same as a standard mkinitcpio.conf
, except some additional declarations may be added to control
aspects of the zfsbootmenu
mkinitcpio module. The configuration file includes extensive inline documentation in the
form of comments; configuration options specific to ZFSBootMenu are also described in the
zfsbootmenu(7) manual page.
ZFSBootMenu still expects to use dracut by default. To override this behavior and instead use mkinitcpio, edit
/etc/zfsbootmenu/config.yaml
and add the following options:
Global:
InitCPIO: true
## NOTE: The following three lines are OPTIONAL
InitCPIOHookDirs:
- /etc/zfsbootmenu/initcpio
- /usr/lib/initcpio
Note
In the examples below, a couple of mkinitcpio modules will be installed to /etc/zfsbootmenu/initcpio
to keep them
isolated from system-installed modules. To accommodate this non-standard installation, InitCPIOHookDirs
must be
defined in /etc/zfsbootmenu/config.yaml
. Furthermore, because overriding the hook directory causes mkinitcpio to
ignore its default module path, the default /usr/lib/initcpio
must be manually specified. If all hooks are
installed in /usr/lib/initcpio
or /etc/initcpio
, the ZFSBootMenu configuration does not need to specify
InitCPIOHookDirs
.
Without further changes, running generate-zbm
should now produce a ZBM image based on mkinitcpio rather than dracut,
although it will lack networking and remote-access capabilities. (By default, generate-zbm
instructs mkinitcpio to
use the configuration at /etc/zfsbootmenu/config.yaml
, although this can be changed in the generate-zbm
configuration file.) For these features, some additional mkinitcpio modules and configuration changes are necessary.
Because further configuration will require additional mkinitcpio modules, and these must be run before the
zfsbootmenu
module in the initramfs, edit /etc/zfsbootmenu/mkinitcpio.conf
and remove any zfsbootmenu
entry in the HOOKS
definition. As the standard configuration file notes, the zfsbootmenu
module is required for
ZFSBootMenu to function, but generate-zbm
will forcefully at this at the end of the module list. Thus, the simplest
way to ensure that additions to the HOOKS
array occur before the zfsbootmenu
module is to omit the latter from
the configuration. The standard HOOKS
line in /etc/zfsbootmenu/mkinitcpio.conf
should therefore be something
like:
HOOKS=(base udev autodetect modconf block filesystems keyboard)
Basic Network Access
Network access in a mkinitcpio image can be realized in one of several ways. In Arch Linux, for example, the
mkinitcpio-nfs-utils package provides a
net module that allows the initramfs to parse ip=
directives from the kernel command line. When a static IP configuration is sufficient, the
mkinitcpio-rclocal module allows user scripts to be injected at
several points in the initramfs boot process and provides a simple mechanism for configuring a network interface.
When installing mkinitcpio modules that are not provided by a system package manager, it may be preferable to keep them
isolated from the ordinary module tree. Because this module will only be required in ZBM images, placing extra modules
in /etc/zfsbootmenu/initcpio
is convenient:
curl -L https://github.com/ahesford/mkinitcpio-rclocal/archive/master.tar.gz | tar -zxvf - -C /tmp
mkdir -p /etc/zfsbootmenu/initcpio/{install,hooks}
cp /tmp/mkinitcpio-rclocal-master/rclocal_hook /etc/zfsbootmenu/initcpio/hooks/rclocal
cp /tmp/mkinitcpio-rclocal-master/rclocal_install /etc/zfsbootmenu/initcpio/install/rclocal
rm -r /tmp/mkinitcpio-rclocal-master
Next, create an rc.local
script that can be run within the mkinitcpio image to configure the eth0
interface:
cat > /etc/zfsbootmenu/initcpio/rc.local <<RCEOF
#!/bin/sh
# Don't attempt to configure an interface that does not exist
ip link show dev eth0 >/dev/null 2>&1 || exit
# Bring up the interface
ip link set dev eth0 up
# Configure a static address for this host
ip addr add 192.168.1.2/24 brd + dev eth0
ip route add default via 192.168.1.1
# Add some name servers
cat > /etc/resolv.conf <<-EOF
nameserver 1.1.1.1
nameserver 8.8.8.8
EOF
RCEOF
Note
If your Ethernet interface is called something other than eth0
or your static IP configuration is different,
adjust the script as needed.
To ensure that the rclocal
module is installed and run in the ZBM image, either append rclocal
to the array
defined on the HOOKS
line in /etc/zfsbootmenu/mkinitcpio.conf
or run
sed -e '/HOOKS=/a HOOKS+=(rclocal)' -i /etc/zfsbootmenu/mkinitcpio.conf
The rclocal
module should be told where it can find the rc.local
script to install and run by running:
echo 'rclocal_hook=/etc/zfsbootmenu/initcpio/rc.local' >> /etc/zfsbootmenu/mkinitcpio.conf
Finally, make sure to include the ip
executable in your initramfs image by manually adding ip
to the
BINARIES
array in /etc/zfsbootmenu/mkinitcpio.conf
or by running
sed -e '/BINARIES=/a BINARIES+=(ip)' -i /etc/zfsbootmenu/mkinitcpio.conf
Dropbear
Arch Linux provides a mkinitcpio-dropbear package that provides a straightforward method for installing, configuring and running the dropbear SSH server inside a mkinitcpio image. This package is based on a project of the same name by an Arch Linux developer. A fork of the mkinitcpio-dropbear project contains a few minor improvements in runtime configuration and key management. If these improvements are not needed, using the upstream project is perfectly acceptable.
Once again, the mkinitcpio module must first be downloaded and installed:
curl -L https://github.com/ahesford/mkinitcpio-dropbear/archive/master.tar.gz | tar -zxvf - -C /tmp
mkdir -p /etc/zfsbootmenu/initcpio/{install,hooks}
cp /tmp/mkinitcpio-dropbear-master/rclocal_hook /etc/zfsbootmenu/initcpio/hooks/dropbear
cp /tmp/mkinitcpio-dropbear-master/rclocal_install /etc/zfsbootmenu/initcpio/install/dropbear
rm -r /tmp/mkinitcpio-dropbear-master
The upstream dropbear
module will attempt to copy host OpenSSH keys into /etc/dropbear
if possible; otherwise,
it will generate random host keys. Both options are undesirable. Copying host keys will leave these protected files
directly accessible to anybody able to read a ZFSBootMenu image, which is probably every user on the system. Generating
unique keys with each run inhibits your ability to detect interlopers when you connect to your bootloader via SSH. My
fork will, by default, respect any existing dropbear keys available as /etc/dropbear/dropbear_*_host_key
. Therefore,
make some new host keys for use in your ZFSBootMenu image:
mkdir -p /etc/dropbear
for keytype in rsa ecdsa ed25519; do
dropbearkey -t "${keytype}" -f "/etc/dropbear/dropbear_${keytype}_host_key"
done
The module also requires, at /etc/dropbear/root_key
, a set of authorized SSH keys that will be given access to the
root
account in the image. On a single-user system, it is sufficient to do:
ln -s ${HOME}/.ssh/authorized_keys /etc/dropbear/root_key
assuming that ${HOME}
points to the home directory of the user who should be given access to ZFSBootMenu.
Finally, enable the dropbear
module in /etc/zfsbootmenu/mkinitcpio.conf
by manually appending dropbear
to
the HOOKS
array, or by running:
sed -e '/HOOKS.*rclocal/a HOOKS+=(dropbear)' -i /etc/zfsbootmenu/mkinitcpio.conf
Final Steps
With the above configuration complete, running generate-zbm
should produce a ZFSBootMenu image that contains the
necessary components to enable an SSH server in your bootloader. This can be verified with the lsinitrd
tool
provided by dracut or the lsinitcpio
tool provided by mkinitcpio. (The lsinitcpio
tool is not able to inspect
UEFI bundles, but lsinitrd
can.) In the file listing, you should see keys in /etc/dropbear
, the dropbear
and
ip
executables, and the file root/.ssh/authorized_keys
.
After rebooting, ZFSBootMenu should configure the network interface, launch an SSH server and accept connections on TCP
port 22 by default. If your SSH client complains because it finds ZFSBootMenu keys when it expects to find your normal
host keys, you may wish to reconfigure dropbear to listen on a non-standard port. My fork of mkinitcpio-dropbear
supports this by writing a dropbear_listen
definition to /etc/dropbear/dropbear.conf
:
echo 'dropbear_listen=2222' > /etc/dropbear/dropbear.conf
After writing this file (adjust 2222
to whatever port you prefer), re-run generate-zbm
, reboot and confirm that
dropbear listens where expected.
Accessing ZFSBootMenu Remotely
When you connect to ZFSBootMenu via SSH, you will be presented a simple shell prompt. Launch zfsbootmenu
to start
the menu interface over the remote connection:
zfsbootmenu
You may then use the menu as if you were connected locally.
Note
recent versions of ZFSBootMenu automatically set the TERM
environment variable to linux
. If you are running an
older version, your SSH client may have provided a more specific terminal definition that will not be recognized by
the restricted environment provided by ZFSBootMenu. Under these circumstances, you may need to run:
export TERM=linux
from the login shell to ensure that basic terminal functionality works as expected.
If you followed the Void Linux ZFSBootMenu install guide and configured
rEFInd to launch ZFSBootMenu, you may need to remove the zbm.skip
argument from the default menu entry if you would
like remote access and you have no encrypted boot environments. Otherwise, rEFInd will attempt to bypass the ZFSBootMenu
countdown and your default boot environment will be started immediately if possible. In this case, either set
zbm.timeout
to a suitably long delay (e.g., 60 sec) to give yourself time to connect and launch ZFSBootMenu
remotely before the automatic boot can proceed, or use zbm.show
by default to prevent automatic boot and force the
local instance to show the interactive menu immediately.
Note
To provide some safety against multi-user conflicts, only one ZFSBootMenu instance is allowed to run at any given
time. If you have encrypted boot environments, this will generally not present an issue, because the local instance
will always block awaiting passphrase entry before launching the menu instance. Otherwise, the later instance of
ZFSBootMenu will wait patiently for the earlier instance to terminate before continuing. If you are certain that the
currently running instance is not being actively used, you can interrupt the wait loop by pressing [ESC]
and then
run:
rm /zfsbootmenu/active
to eliminate the indicator of the other running instance. You may then run zfsbootmenu
again to launch the menu.