#11 add a 'missing release blocking images' check
Closed by ausil. Opened by adamwill.
adamwill/compose-utils blocking-images  into  master

no initial comment

Couple of things to note: this is a simple implementation which is Fedora-specific. I wasn't sure if compose-utils is used for RHEL. If it is, obviously if we're gonna do this here we need to make the blocking list configurable somehow. Also, I'm not totally sure this is the best way to do this, but I had it more or less written already and a PR is as good a way as any to discuss the best way to do it. It's not really part of the changelog vs. the previous compose, so doing it here is a bit 'wrong', but it slots in nicely since we already did the work of getting the image IDs from the compose. Per https://pagure.io/pungi/issue/249 , it may actually be an idea to make Pungi itself do most or even all of this work. I will look into doing that next week, perhaps - it's not something I can bash out on a Friday afternoon :)

This, again, is a port of a feature currently in check-compose,
more or less. check-compose has an 'expected images' check,
which covers rather more than the 'release blocking' images,
but it's a fairly arbitrary list and I figured for something
that isn't just my pet project, sticking to a properly defined
list would be better.

It's quite simple: it checks the images in the compose against
a built-in list of 'release blocking' images, and logs any that
are missing.

Unlike the 'image diff' stuff from the last commit we cannot
store productmd-style image dicts in the JSON because we don't
have any - the images are missing, they have no dicts. All
we can reasonably store is the image 'identifiers' we use to
try and find them.

Using the 'image ID'-style tuples for matching images makes the list very simple and makes it easy to produce the text output, at the cost of being rather inflexible. Another option (the way we do it in the openQA scheduler) is to use dicts for matching images, which is more flexible but a bit more verbose and would require more work to generate the human-readable identifier text for the missing images.

In an ideal world, the list of release blocking images would
be available in some easily parseable format that matches the
productmd metadata (like, say, a simple set of image identifier
tuples!), and either the code could check it live each time or
(if you don't want a network dependency) we could have a simple
utility for synchronizing this tool's list which could be run
every checkin or something.

In the real world, the 'official' list of release blocking
deliverables is this ridiculous wiki page:

https://fedoraproject.org/wiki/Fedora_Program_Management/ReleaseBlocking/Fedora24

The list in this commit is almost the same, only I left out
the Cloud netinst because I'm 99% sure that being 'release
blocking' is a mistake.

I am quite good at parsing wikis and I could write a thing
which would turn that page into this list. I am not going to
because that would be incredibly dumb. I'd suggest we do two
things to clean this up, but I'm easy on the order:

i) merge this
ii) produce an image list in a sane format (JSON or whatever)
and a nice simple script for turning it into a static HTML page
for human consumption
iii) find an appropriate official server to host both things on,
then kill the ridiculous wiki page and make this code use that
list.

Currently this package is used only by Fedora. However, it is open-sourced code originally used for RHEL. At some point I would like to consolidate the tooling, but that is a bit far away at this point.

Even then, I don't think this would be required for RHEL. As far as I know, all images that are produced as part of a compose are release blocking there: if it isn't blocking, it's not done in the compose.

That said, I don't oppose doing this here, I would just prefer to do it in a way that would not be tied to exactly one user.

What would you think about perhaps doing this here as a short-term thing then throwing it away when we're done with https://pagure.io/pungi/issue/249 ?

I can quite easily do a short-term bodge for making it configurable if we decide to go with that approach.

For the RHEL case, well, they could still use this code, unless they also make all deliverables failable - so long as it's possible for a compose to be FINISHED_INCOMPLETE they could use this code. It'd simply be the case that their list of release-blocking deliverables would be...all of them. :) Again when #249 is done this could be thrown away.

I am going to close this PR, I believe we will have everything in place when we upgrade koji to 1.12.0 to be able to mark in the pungi config all release blocking deliverables as non failable. which will mean that if a compose completes as FINISHED or FINISHED_INCOMPLETE that we will have all blocking artifacts.

Pull-Request has been closed by ausil