We want to replace a disk of a ZFS pool, either because it is faulty or because we want to extend the pool.
If the latter is the case, the first thing we have to check is that the ZFS pool has autoextend configured.
zpool get autoexpand <pool>
If that is not the case we have to enable it before replacing any disks:
zpool set autoexpand=on <pool>
The next step is to mark the disk we want to replace as offline in order to stop ZFS creating IO on it:
zpool offline <pool> <old-disk>
We proceed by physically removing the old disk and physically adding the new disk to the server. This can be done with a running system.
If you have to replace a boot disk, it is easiest to copy the partition layout of an online disk of that pool.
sgdisk <online-disk-of-pool> -R <new-disk>
Followed by a randomization of GUIDs:
sgdisk -G <new-disk>
If you added disks of larger size it makes sense to extend the respective ZFS partition with a tool of your liking (sgdisk, gdisk, etc.).
Finally, we instruct ZFS to replace the offline disk with the new disk (basically deteaching and attaching to the pool in one step)
zpool replace <pool> -f /dev/disk/by-id/<old-zfs-part> /dev/disk/by-id/<new-zfs-part>
Wait for resilver. This is ist for a non-boot disk. For a boot disk you have to install the respective bootloader. These steps depend on your bootloader. For Proxmox with uefi boot via systemd-boot the steps look as follows:
proxmox-boot-tool format /dev/disk/by-id/<new-efi-part>
proxmox-boot-tool init /dev/disk/by-id/<new-efi-part>
proxmox-boot-tool status
proxmox-boot-tool refresh
proxmox-boot-tool clean
proxmox-boot-tool status