Skip to content

OS updates

Updated

Virtainer Free ships as one immutable host image. Updating is not a package upgrade: you point the host at a different image and reboot into it.

That is what makes the update reversible. The previous image stays on disk, so going back is a reboot rather than a restore.

Each host image updates the operating system and Virtainer together. Read the release notes for supported behavior and changes before choosing a version. Existing VMs keep their nested virtualization behaviour; new VMs default to off and can turn it on.

  1. Open OS Updates. The page shows the image the host booted from, labelled Booted image. The reference names the version, so this is where you confirm what you are actually running before you change it.

  2. Pick the target version. The page lists the builds published to the repository this host reads from, with the date each was published and its size where the registry reports them. Pick a newer one to move forward, or an older one deliberately.

    Every entry is a fixed build, and a build is not the same thing as a release: the list shows whatever that repository currently publishes. A normal installation sees one repository and has no channel control. A host deliberately opted into the release-candidate channel gets a channel selector as well, and switching channel changes which builds appear in the list.

  3. Choose how to apply it.

    • Switch & reboot stages the new image and reboots into it. The dialog asks you to type a confirmation phrase before it will run, so a stray click cannot reboot the host.
    • Stage only prepares it and leaves the host running as it is. Reboot later, on your own schedule, to activate it.
  4. Follow the operation. The host reports the change as three steps: Pull & stage image, Reboot host, and Verify booted image. The last one confirms the host really came back on the image you asked for.

Virtainer Free 1.0.0 is affected. The fix ships in 1.1.0, so every 1.1.0 build and later is not.

On a host running 1.0.0, an update can pull the new image, create the deployment, reboot, and come back on the image it already had, reporting no error. The machine looks unchanged rather than broken, which is why this is worth a section of its own.

The update can fail before its new boot entry is saved. The host then restarts into the existing version even though the image download completed.

The console’s explanation is a guess, and in this case it is wrong. It reports that the host came back on a different image than the target, and suggests the most likely cause is that the new deployment failed to boot and the system rolled back. Nothing ever booted, so that is not what happened. Treat the wording as “the update did not land” and confirm which of the two it was before acting:

  • The host log carries Remounting /boot read-write: Invalid argument from the image layer.
  • The boot entries under /boot/loader/entries/ keep their old timestamps.
  • The boot list shows one reboot rather than a boot attempt and a rollback.

For a host still on 1.0.0, apply the workaround before its first update. At the host’s console or over SSH, run:

Terminal window
sudo systemctl mask boot.automount

Then retry the update and verify Booted image afterwards. This workaround prevents the known boot-directory conflict; it is needed only for the first successful move from 1.0.0 to 1.1.0 or later. Later versions contain the fix.

A host that is rebooting cannot be reached, and the page notices: it keeps checking and recovers on its own once Virtainer Free answers again. If the host has not answered for eight minutes the page says so and stops claiming to know, so reload it once the host is reachable to see the verified result. A host that comes back on an image other than the one you chose is reported as that, not as a success. Read when an update does nothing before you accept the reason the console offers for that outcome, because on 1.0.0 the console is guessing.

Every entry in the list names one fixed image, because picking an entry means deciding exactly what the host will run.

The registry also publishes a latest tag. It is a name for the most recently published image, not an image of its own, and it exists so that install and switch commands typed by hand do not need a new version number every time. It is deliberately never offered in the console, and two properties of it are worth knowing before you use it in a script:

  • It tracks the last publish, not the newest version. Re-publishing an older build moves it backwards, and anything installing by latest then receives that older build.
  • Each repository has its own latest. The stable repository’s and the release-candidate repository’s are independent names pointing at independent images.

There are two different operations here, and only one of them is a reboot.

Boot the image that is already installed. The previous deployment is still on disk, so going back to it is a single reboot that depends on no backup. This belongs to the host image mechanism rather than to Virtainer Free, and the console offers no button for it. Run it on the host, over SSH or at its local console:

Terminal window
sudo bootc status # "Booted" and "Rollback": where you are and where you can go
sudo bootc rollback --apply # make the previous image the boot entry and reboot now

Without --apply only the boot order changes, and it takes effect at the next reboot. Afterwards, Booted image on the OS Updates page should show the version you rolled back to. A few things to know first:

  • It goes back to the previous image only, not to any older one. For an earlier version, switch to it from the version list, which makes it the new current image.
  • It rolls back the system and the program, not /var. Machine disks, snapshots and the control-plane database stay as they are. An older version still starts on a database a newer one changed, but it cannot see or use data that only the newer version’s features produced.
  • If you had used Stage only and not yet rebooted, the rollback discards the staged image.
  • The host never updates itself, so a rollback is not quietly undone. It stays on the image you chose until you switch again.

Switch to an older version. This is an ordinary update in the other direction: pick the older build in the list and switch to it, staging first if you want the choice to sit there until you decide. Because it is the same operation, it carries the same failure mode, including the one described above (see “When an update does nothing”). A host running 1.0.0 can discard a switch back to an older version exactly as readily as it discards a switch forward.

A host reboot stops every machine. When the host comes back, machines that were running are started again, and their addresses are preserved. See After a host reboot for the details and the failure cases.

Nothing about your machines lives in the host image. Configuration and data persist across the switch; only the host system is replaced.

LVM volume groups after upgrading to Free 1.3

Section titled “LVM volume groups after upgrading to Free 1.3”

From Free 1.3, each disk reports its own serial to the guest instead of a serial shared by every disk (see Volumes). The change happens once, at each VM’s first boot after the host upgrade, and most guests do not notice it.

Who is affected. Guests on a host installed from a Virtainer image whose LVM keeps a devices file, the default on RHEL 9 and later, AlmaLinux and Rocky Linux 9 and later, CentOS Stream 9 and later, and Fedora 36 and later. Their devices file recorded the old shared serial, vhost_user_blk, for every physical volume created before the upgrade, and no disk reports it any more. After the upgrade LVM logs device is not in devices file, vgs lists nothing, and mounts on those volume groups fail; a guest without nofail on such a mount stops in emergency mode. The disks and their data are intact.

Filesystems mounted by UUID or label, guests whose LVM does not use a devices file, and Windows guests in the cases tested are not affected. To check a Linux guest before the upgrade, run lvmconfig --typeconfig full devices/use_devicesfile (1 means LVM uses a devices file) and look for IDNAME=vhost_user_blk in /etc/lvm/devices/system.devices.

Recovery, in each affected guest after the host upgrade:

Terminal window
sudo lvmdevices --update --refresh # rewrite the entries with each disk's new serial
sudo vgchange -ay # activate the volume groups again
sudo mount -a # or reboot

Plain lvmdevices --update does not find the disks; it needs --refresh. LVM keeps the previous devices file under /etc/lvm/devices/backup/. Once repaired, the volume groups survive reboots, and later boots, attachments and moves of a volume to another VM do not change the serial again.

On an older lvm2 without --refresh, re-add each disk by hand. This path is not tested:

Terminal window
sudo lvmdevices --check # lists the stale entries ("device not found") with their PVIDs
sudo pvs --devicesfile "" -o pv_name,pv_uuid,vg_name # shows which /dev/vdX carries each physical volume
sudo lvmdevices --delpvid <PVID> # remove each stale entry
sudo lvmdevices --adddev /dev/vdX # add the disk again with its current serial
sudo lvmdevices --check # no stale entries should remain
sudo vgchange -ay
sudo mount -a

The PVID in the devices file is the pv_uuid from pvs with the hyphens removed.

A root volume group on a Virtainer disk. The guest cannot find its root filesystem, so repair it from rescue media. This path is not tested:

  1. Activate and mount the guest’s root: vgchange -ay --devicesfile "", then mount the root logical volume on /mnt and bind-mount /dev, /proc, /sys and /run into it.
  2. chroot /mnt, then mount /boot (and /boot/efi if it is separate), for example with mount /boot.
  3. Inside the chroot, run lvmdevices --update --refresh, then dracut -f --regenerate-all, so the guest’s initramfs carries the updated devices file.
  4. Exit the chroot, unmount everything under /mnt, and reboot the guest from its own disk.

Running these outside the chroot changes the rescue system instead of the guest.

If the host was not installed from a Virtainer image, the page says so and the switch is unavailable. Update it the way you update the rest of that system.

The image reference behind the version you pick must be reachable over a valid TLS connection. There is no option to disable certificate checking. A registry using plain HTTP or a self-signed certificate has to be trusted by the host image itself.