appliance-tools

#17 Populate fstab(5) and bootloader configuration for Btrfs subvolumes
Merged by ngompa. Opened by ngompa.
populate-fstab-and-bootloader-config  into  master

In order for an image to be setup fully on boot, we need to populate
fstab(5) configuration and ensure the bootloader has the right
references for where the Btrfs filesystems are located on the disk.

rebased onto dfbbb52befa027221b23cdd21852306f3240a56c

@dcavalca @pwhalen Please test this and see if this makes a bootable image.

rebased onto 87eaedbb02ad4e3dc5701e49382678650d0a76cb

 
[davide@localhost ~]$ sudo /usr/bin/appliance-creator -c koji-image-f33-build-50158384-x86_64.ks --logfile appliance.log --cache  koji-appliance -o app-output --format raw --name Fedora-Xfce-armhfp-33-20200825.n.1 --version 33 --release 20200825.n.1
Create subvolume '/var/tmp/imgcreate-1ck_k6rv/install_rootbtrfs.007/root'
Create subvolume '/var/tmp/imgcreate-1ck_k6rv/install_rootbtrfs.007/home'
Traceback (most recent call last):
  File "/usr/bin/appliance-creator", line 193, in <module>
    sys.exit(main())
  File "/usr/bin/appliance-creator", line 153, in main
    creator.mount("NONE", options.cachedir)
  File "/usr/lib/python3.9/site-packages/imgcreate/creator.py", line 573, in mount
    self.__write_fstab()
  File "/usr/lib/python3.9/site-packages/imgcreate/creator.py", line 458, in __write_fstab
    fstab.write(self._get_fstab())
  File "/usr/lib/python3.9/site-packages/appcreate/appliance.py", line 91, in _get_fstab
    if not p['UUID'] is None:
TypeError: 'NoneType' object is not subscriptable
[davide@localhost ~]$

I think we need to assign p = p1 here as well, or the test in the block below can fail because it will still be None

With that fix it builds, but fails right at the end with

Created symlink /etc/systemd/system/multi-user.target.wants/initial-setup.service → /usr/lib/systemd/system/initial-setup.service.
Traceback (most recent call last):
  File "/usr/bin/appliance-creator", line 193, in <module>
    sys.exit(main())
  File "/usr/bin/appliance-creator", line 155, in main
    creator.configure()
  File "/usr/lib/python3.9/site-packages/imgcreate/creator.py", line 799, in configure
    self._create_bootconfig()
  File "/usr/lib/python3.9/site-packages/appcreate/appliance.py", line 610, in _create_bootconfig
    self._create_grub_config()
  File "/usr/lib/python3.9/site-packages/appcreate/appliance.py", line 330, in _create_grub_config
    (bootdevnum, rootdevnum, rootdev, prefix) = self._get_grub_boot_config()
  File "/usr/lib/python3.9/site-packages/appcreate/appliance.py", line 320, in _get_grub_boot_config
    rootdevnum = s['num'] - 1
TypeError: unsupported operand type(s) for -: 'NoneType' and 'int'
lazy umount succeeded on /var/tmp/imgcreate-5epmbj4d/install_root/var/cache/dnf
[davide@localhost ~]$

This comparison always fails because:

p['device'] = /dev/loop33
s['device'] = /dev/mapper/loop3p3

which results in s['num'] never getting initialized.
Edited by dcavalca

Hmm, actually, I think I need the parent loop also skipped in this case...

rebased onto 99fd67be1b17877854d578503a8ee902a7f14236

@dcavalca I think I fixed the patch to do the right thing here...

rebased onto 7d432723ebfc6e9da115b11216ca2a88b6f98135

rebased onto 7d432723ebfc6e9da115b11216ca2a88b6f98135

Testing the latest:

Writing kickstart file.
Writing GRUB Legacy config.
Traceback (most recent call last):
File "/usr/bin/appliance-creator", line 193, in
sys.exit(main())
File "/usr/bin/appliance-creator", line 155, in main
creator.configure()
File "/usr/lib/python3.8/site-packages/imgcreate/creator.py", line 799, in configure
self._create_bootconfig()
File "/usr/lib/python3.8/site-packages/appcreate/appliance.py", line 611, in _create_bootconfig
self._create_grub_config()
File "/usr/lib/python3.8/site-packages/appcreate/appliance.py", line 331, in _create_grub_config
(bootdevnum, rootdevnum, rootdev, prefix) = self._get_grub_boot_config()
File "/usr/lib/python3.8/site-packages/appcreate/appliance.py", line 321, in _get_grub_boot_config
rootdevnum = s['num'] - 1
TypeError: unsupported operand type(s) for -: 'NoneType' and 'int'

rebased onto 8598a734aac39591812cb4b3b0169e7b83c45bbc

@dcavalca @pwhalen Can you take a look again?

Fails early with

Creating mount point /var/tmp/imgcreate-u2bjy8sg/install_rootbtrfs.007
Mounting /dev/loop03 at /var/tmp/imgcreate-u2bjy8sg/install_rootbtrfs.007
Traceback (most recent call last):
  File "/usr/bin/appliance-creator", line 193, in <module>
    sys.exit(main())
  File "/usr/bin/appliance-creator", line 153, in main
    creator.mount("NONE", options.cachedir)
  File "/usr/lib/python3.9/site-packages/imgcreate/creator.py", line 537, in mount
    self._mount_instroot(base_on)
  File "/usr/lib/python3.9/site-packages/appcreate/appliance.py", line 250, in _mount_instroot
    self.__instloop.mount()
  File "/usr/lib/python3.9/site-packages/appcreate/partitionedfs.py", line 389, in mount
    self.setup_subvolumes()
  File "/usr/lib/python3.9/site-packages/appcreate/partitionedfs.py", line 309, in setup_subvolumes
    s['num'] = int(s['device'].rsplit('p', 1))
TypeError: int() argument must be a string, a bytes-like object or a number, not 'list'
Unmounting directory /var/tmp/imgcreate-u2bjy8sg/install_root/home
umount: /var/tmp/imgcreate-u2bjy8sg/install_root/home: no mount point specified.
Unmounting directory /var/tmp/imgcreate-u2bjy8sg/install_root/home failed, using lazy umount
Unmounting directory /var/tmp/imgcreate-u2bjy8sg/install_root/home failed, using lazy umount
umount: /var/tmp/imgcreate-u2bjy8sg/install_root/home: no mount point specified.
Unmounting directory /var/tmp/imgcreate-u2bjy8sg/install_root/
umount: /var/tmp/imgcreate-u2bjy8sg/install_root/: not mounted.
Unmounting directory /var/tmp/imgcreate-u2bjy8sg/install_root/ failed, using lazy umount
Unmounting directory /var/tmp/imgcreate-u2bjy8sg/install_root/ failed, using lazy umount
umount: /var/tmp/imgcreate-u2bjy8sg/install_root/: not mounted.
Removing compat symlinks
Unmapping /dev/loop0
Exception ignored in: <function ImageCreator.__del__ at 0x7fdbd9772310>
Traceback (most recent call last):
  File "/usr/lib/python3.9/site-packages/imgcreate/creator.py", line 114, in __del__
    self.cleanup()
  File "/usr/lib/python3.9/site-packages/imgcreate/creator.py", line 613, in cleanup
    self.unmount()
  File "/usr/lib/python3.9/site-packages/imgcreate/creator.py", line 587, in unmount
    self._unmount_instroot()
  File "/usr/lib/python3.9/site-packages/appcreate/appliance.py", line 638, in _unmount_instroot
    self.__instloop.cleanup()
  File "/usr/lib/python3.9/site-packages/appcreate/partitionedfs.py", line 257, in cleanup
    self.__unmap_partitions()
  File "/usr/lib/python3.9/site-packages/appcreate/partitionedfs.py", line 225, in __unmap_partitions
    raise MountError("Failed to unmap partitions for '%s'" %
imgcreate.errors.MountError: Failed to unmap partitions for '/dev/loop0'
[davide@localhost ~]$

rebased onto 38567258ed975d5c3a91960bb57b6942c9927f55

@dcavalca @pwhalen So I was dumb and forgot to type in the index thingy, I just pushed an updated patch that has that.

Fails in the same spot still, but differently so:

Creating mount point /var/tmp/imgcreate-4muoovjo/install_rootbtrfs.007
Mounting /dev/loop33 at /var/tmp/imgcreate-4muoovjo/install_rootbtrfs.007
Traceback (most recent call last):
  File "/usr/bin/appliance-creator", line 193, in <module>
    sys.exit(main())
  File "/usr/bin/appliance-creator", line 153, in main
    creator.mount("NONE", options.cachedir)
  File "/usr/lib/python3.9/site-packages/imgcreate/creator.py", line 537, in mount
    self._mount_instroot(base_on)
  File "/usr/lib/python3.9/site-packages/appcreate/appliance.py", line 250, in _mount_instroot
    self.__instloop.mount()
  File "/usr/lib/python3.9/site-packages/appcreate/partitionedfs.py", line 389, in mount
    self.setup_subvolumes()
  File "/usr/lib/python3.9/site-packages/appcreate/partitionedfs.py", line 309, in setup_subvolumes
    s['num'] = int(s['device'].rsplit('p', 1)[0])
ValueError: invalid literal for int() with base 10: '/dev/mapper/loop3'
Unmounting directory /var/tmp/imgcreate-4muoovjo/install_root/home
umount: /var/tmp/imgcreate-4muoovjo/install_root/home: no mount point specified.
Unmounting directory /var/tmp/imgcreate-4muoovjo/install_root/home failed, using lazy umount
Unmounting directory /var/tmp/imgcreate-4muoovjo/install_root/home failed, using lazy umount
umount: /var/tmp/imgcreate-4muoovjo/install_root/home: no mount point specified.
Unmounting directory /var/tmp/imgcreate-4muoovjo/install_root/
umount: /var/tmp/imgcreate-4muoovjo/install_root/: not mounted.
Unmounting directory /var/tmp/imgcreate-4muoovjo/install_root/ failed, using lazy umount
Unmounting directory /var/tmp/imgcreate-4muoovjo/install_root/ failed, using lazy umount
umount: /var/tmp/imgcreate-4muoovjo/install_root/: not mounted.
Removing compat symlinks
Unmapping /dev/loop3
Exception ignored in: <function ImageCreator.__del__ at 0x7f15eeb8c310>
Traceback (most recent call last):
  File "/usr/lib/python3.9/site-packages/imgcreate/creator.py", line 114, in __del__
    self.cleanup()
  File "/usr/lib/python3.9/site-packages/imgcreate/creator.py", line 613, in cleanup
    self.unmount()
  File "/usr/lib/python3.9/site-packages/imgcreate/creator.py", line 587, in unmount
    self._unmount_instroot()
  File "/usr/lib/python3.9/site-packages/appcreate/appliance.py", line 638, in _unmount_instroot
    self.__instloop.cleanup()
  File "/usr/lib/python3.9/site-packages/appcreate/partitionedfs.py", line 257, in cleanup
    self.__unmap_partitions()
  File "/usr/lib/python3.9/site-packages/appcreate/partitionedfs.py", line 225, in __unmap_partitions
    raise MountError("Failed to unmap partitions for '%s'" %
imgcreate.errors.MountError: Failed to unmap partitions for '/dev/loop3'
[davide@localhost ~]$

rebased onto 27119bc104b3a7d5835a570987f9f1f1d7b011c3

@dcavalca @pwhalen I just fixed it. Whoops! That's me not understanding rsplit()...

Ok it now builds and completes, the partition layout is correct, but I can't seem to be able to get it to spit out a bootloader config when building for x86_64, and I can't build for armhf due to https://pagure.io/fedora-btrfs/project/issue/11

As noted above, builds fine. Attempting to boot failed. Checking the kernel parameters in the extlinux.conf:

"append ro root=UUID_SUB=6063995c-7541-4f1c-8e22-eb52892c7122 rhgb quiet LANG=en_US.UTF-8 rootflags=subvol=root cma=192MB"

Using UUID_SUB rather than UUID, blkid output:
/dev/sdb3: LABEL="_btrfs.007" UUID="ee019b9b-5e48-4574-a85e-17eda0378bc3" UUID_SUB="6063995c-7541-4f1c-8e22-eb52892c7122" BLOCK_SIZE="4096" TYPE="btrfs" PARTUUID="539d5ee5-03"
Edited by pwhalen

rebased onto 4d7c81f7b4ff4ca7b2a84632f2f0b89d5af58bbf

1 new commit added

  • appcreate/partitionedfs: Ensure __getuuid() actually gets the UUID

@pwhalen I believe I've fixed the parameter with this refreshed PR. Could you please try again?

Pull-Request has been merged by ngompa

In todays Rawhide images created with appliance-tools-011.0-1.fc34:

The good:
cat extlinux.conf:
..
append ro root=UUID=8a8b2744-fabb-43fc-bf31-766214ba7afd rhgb quiet LANG=en_US.UTF-8

The bad:
cat etc/fstab
UUID=UUID=8a8b2744-fabb-43fc-bf31-766214ba7afd / ext4 defaults,noatime 0 0
UUID=UUID=78e0388c-8f65-414f-ab33-2919e4714fb8 /boot ext4 defaults,noatime 0 0
UUID=UUID=E803-6385 /boot/efi vfat defaults,noatime 0 0

The above is from Fedora-Minimal-armhfp-Rawhide-20200827.n.0.

Oh man. :sob:

Fixed that in 441205f4224bf2305917833139490b63bc1825b1 and released 011.1.
Metadata
None
No Tags
Changes Summary 2
+39 -7
file changed
appcreate/appliance.py
+12 -5
file changed
appcreate/partitionedfs.py
Powered by Pagure 1.0.0
Documentation • File an Issue • About • SSH Hostkey/Fingerprint
© Red Hat, Inc. and others.