Couple of things to note: this is a simple implementation which is Fedora-specific. I wasn't sure if compose-utils is used for RHEL. If it is, obviously if we're gonna do this here we need to make the blocking list configurable somehow. Also, I'm not totally sure this is the best way to do this, but I had it more or less written already and a PR is as good a way as any to discuss the best way to do it. It's not really part of the changelog vs. the previous compose, so doing it here is a bit 'wrong', but it slots in nicely since we already did the work of getting the image IDs from the compose. Per https://pagure.io/pungi/issue/249 , it may actually be an idea to make Pungi itself do most or even all of this work. I will look into doing that next week, perhaps - it's not something I can bash out on a Friday afternoon :)
compose-utils
This, again, is a port of a feature currently in check-compose, more or less. check-compose has an 'expected images' check, which covers rather more than the 'release blocking' images, but it's a fairly arbitrary list and I figured for something that isn't just my pet project, sticking to a properly defined list would be better.
It's quite simple: it checks the images in the compose against a built-in list of 'release blocking' images, and logs any that are missing.
Unlike the 'image diff' stuff from the last commit we cannot store productmd-style image dicts in the JSON because we don't have any - the images are missing, they have no dicts. All we can reasonably store is the image 'identifiers' we use to try and find them.
Using the 'image ID'-style tuples for matching images makes the list very simple and makes it easy to produce the text output, at the cost of being rather inflexible. Another option (the way we do it in the openQA scheduler) is to use dicts for matching images, which is more flexible but a bit more verbose and would require more work to generate the human-readable identifier text for the missing images.
In an ideal world, the list of release blocking images would be available in some easily parseable format that matches the productmd metadata (like, say, a simple set of image identifier tuples!), and either the code could check it live each time or (if you don't want a network dependency) we could have a simple utility for synchronizing this tool's list which could be run every checkin or something.
In the real world, the 'official' list of release blocking deliverables is this ridiculous wiki page:
https://fedoraproject.org/wiki/Fedora_Program_Management/ReleaseBlocking/Fedora24
The list in this commit is almost the same, only I left out the Cloud netinst because I'm 99% sure that being 'release blocking' is a mistake.
I am quite good at parsing wikis and I could write a thing which would turn that page into this list. I am not going to because that would be incredibly dumb. I'd suggest we do two things to clean this up, but I'm easy on the order:
i) merge this ii) produce an image list in a sane format (JSON or whatever) and a nice simple script for turning it into a static HTML page for human consumption iii) find an appropriate official server to host both things on, then kill the ridiculous wiki page and make this code use that list.
Currently this package is used only by Fedora. However, it is open-sourced code originally used for RHEL. At some point I would like to consolidate the tooling, but that is a bit far away at this point.
Even then, I don't think this would be required for RHEL. As far as I know, all images that are produced as part of a compose are release blocking there: if it isn't blocking, it's not done in the compose.
That said, I don't oppose doing this here, I would just prefer to do it in a way that would not be tied to exactly one user.
What would you think about perhaps doing this here as a short-term thing then throwing it away when we're done with https://pagure.io/pungi/issue/249 ?
I can quite easily do a short-term bodge for making it configurable if we decide to go with that approach.
For the RHEL case, well, they could still use this code, unless they also make all deliverables failable - so long as it's possible for a compose to be FINISHED_INCOMPLETE they could use this code. It'd simply be the case that their list of release-blocking deliverables would be...all of them. :) Again when #249 is done this could be thrown away.
FINISHED_INCOMPLETE
I am going to close this PR, I believe we will have everything in place when we upgrade koji to 1.12.0 to be able to mark in the pungi config all release blocking deliverables as non failable. which will mean that if a compose completes as FINISHED or FINISHED_INCOMPLETE that we will have all blocking artifacts.
Pull-Request has been closed by ausil