#987 [openssl] upcoming critical openssl vulnerability | rhbz#2137661
Closed by blockerbot. Opened by blockerbot.

Bug details: ** https://bugzilla.redhat.com/show_bug.cgi?id=2137661 **
Information from BlockerBugs App:
2137661

Current vote summary

Commented but haven't voted yet: kparal, adamwill

The votes have been last counted at 2022-10-27 18:07 UTC and the last processed comment was #comment-823220

To learn how to vote, see:
https://pagure.io/fedora-qa/blocker-review
A quick example: BetaBlocker +1 (where the tracker name is one of BetaBlocker/FinalBlocker/BetaFE/FinalFE/0Day/PreviousRelease and the vote is one of +1/0/-1)


I posted this to the BZ, but it probably also belongs here:

Without knowing the extent of the problem, I'd be hesitant to delay for it. If we had shipped today, we'd be awaiting an errata just the same. The relevant release criterion is: "The release must contain no known security bugs of 'important' or higher impact according to the Red Hat severity classification scale which cannot be satisfactorily resolved by a package update (e.g. issues during installation)." But since we cannot know by the upcoming Go/No-Go whether this issue would impact installation, I think we just have to plan for a quick security bug release.

Alternately, if we can get enough of a disclosure from upstream that says "This will probably have impact on your installer", without going into detail, I'd probably bow to their wisdom and block based on this criterion. Without that hint, however, I think we have to operate under the assumption that it's fixable as an update post-release.

FinalBlocker -1
(This vote is subject to change if the above condition is met)

FinalBlocker +1

This will be OpenSSL's first "CRITICAL" vulnerability since 2016. Examples of "CRITICAL" vulnerabilities include "significant disclosure of the contents of server memory (potentially revealing user details), vulnerabilities which can be easily exploited remotely to compromise server private keys or where remote code execution is considered likely in common situations"

https://www.openssl.org/policies/general/security-policy.html

Just for bureaucracy purposes, here's a link to our relevant release criterion:
https://fedoraproject.org/wiki/Fedora_37_Final_Release_Criteria#Security_bugs

Although I had "lean hard on the GO button" on my agenda for tomorrow, RH product security tells me that they advise making sure our installation media and various images are updated. So, sadly...

FinalBlocker +1

Is there a possibility of doing a less-than-full-week slip, here?

FinalBlocker +1

In addition to the reasoning Matthew cited, the Infrastructure Team will want to make quick updates to our infrastructure, and the release day is a really bad time to try to do emergency patching.

Although I had "lean hard on the GO button" on my agenda for tomorrow, RH product security tells me that they advise making sure our installation media and various images are updated. So, sadly...

FinalBlocker +1

Is there a possibility of doing a less-than-full-week slip, here?

Given how central OpenSSL is to so much of the distro, I'd really prefer that if we slip for this, we take the time to run a complete regression suite before shipping. I'm somewhat concerned that if the disclosure next Tuesday happens very late in the day (or ends up delayed for other reasons), we may not be able to produce a release candidate with sufficient time to test it before next Thursday's Go/No-Go meeting.

We've already slipped a couple times; I don't know if we want to potentially slip two more weeks.

Nobody has yet argued that this problem cannot be "satisfactorily resolved by a package update" and therefore it does not meet any blocker criterion.

FinalBlocker -1

My best guess is that this would somehow affect the security of anaconda's netinstall (does it use OpenSSL via curl?), but without more details it does not fit the criterion.

Nobody has yet argued that this problem cannot be "satisfactorily resolved by a package update" and therefore it does not meet any blocker criterion.

It's rather hard to argue that when we don't know what the problem is. (Although I will note that the "e.g." in the release criteria really is just an example, it is not a comprehensive list; we should also block if the issue affects anything people would be likely to do from a live desktop.) But if "RH product security tells me that they advise making sure our installation media and various images are updated", I am kind of inclined to trust that, so my initial read on this would be +1. I'm gonna hold my vote in case we have more info by the meeting tomorrow, though.

Note that if we slip for this, there is kind of an argument to do a mini-refresh before release: also pull in kernel 6.0, Plasma 5.26.1, possibly GNOME 43.1, Firefox 106.0.1, generally sorta freshen things up. It increases risk of new blockers showing up, but means we wouldn't be shipping with an EOL kernel and we'll have newer versions of everything than the new Ubuntu release (ahem).

Note that if we slip for this, there is kind of an argument to do a mini-refresh before release: also pull in kernel 6.0, Plasma 5.26.1, possibly GNOME 43.1, Firefox 106.0.1, generally sorta freshen things up. It increases risk of new blockers showing up

Might as well eliminate the freeze period, then? :D

But if "RH product security tells me that they advise making sure our installation media and various images are updated", I am kind of inclined to trust that, so my initial read on this would be +1.

Maybe your colleagues at RH would be so kind as to provide more information for the general audience. Trust doesn't simply come by wearing a hat in a primary color.

 Maybe your colleagues at RH would be so kind as to provide more information for the general audience. Trust doesn't simply come by wearing a hat in a primary color.

Please understand that this is not a matter of "kindness".

The information is under embargo — an agreement among vendors to not reveal specifics until a certain date, so that everyone can get their patches ready. If this weren't done, there would be a period where the information to exploit the flaw is widely available without any patches.

Red Hat is part of a group of operating system vendors who participate in this sharing. Specific people on the product security team, as well as individual software engineers who know the code, are brought in as needed. If these people were to break embargo (that is, share details before the agreed-upon date), Red Hat would not get any advance notice next time.

I know it is frustrating to work with imperfect knowledge. I'm doing the best I can here.

Given the severity of the upcoming disclosure and Red Hat's guidance we really should have this on the media, I'm going to go with blocking on this too.

FinalBlocker +1

Note that if we slip for this, there is kind of an argument to do a mini-refresh before release: also pull in kernel 6.0, Plasma 5.26.1, possibly GNOME 43.1, Firefox 106.0.1, generally sorta freshen things up. It increases risk of new blockers showing up, but means we wouldn't be shipping with an EOL kernel and we'll have newer versions of everything than the new Ubuntu release (ahem).

Yeah, I think we should do that too if we can...

(To be clear, I really don't want to delay F37 by two weeks, which is basically what this implies. But if this is as bad as people are implying, then I don't know what else we're supposed to do here.)

so... I have conflicting feelings about this one... but:

FinalBlocker +1

I'll lean on security first, so

FinalBlocker +1

The information is under embargo — an agreement among vendors to not reveal specifics until a certain date, so that everyone can get their patches ready. If this weren't done, there would be a period where the information to exploit the flaw is widely available without any patches.

I understand that and I wasn't expecting any technical details until after the issue is fixed. More something along the line of "it's as bad (or worse) as heartbleed" or "another shellshock". Something that indicates the level of exploitability and scope of affected users/systems.

As @sgallagh put it:

If we had shipped today, we'd be awaiting an errata just the same.

I'll take it a step further: currently running systems are already vulnerable and in need of this fix all the same. We don't go around asking all users to turn off their system for two weeks. They would ask why. And rightly so.

On the other hand, I do consider security important. Important enough to let a release slip for a fortnight. Going by what little we know, which is currently only the definition of CRITICAL by OpenSSL, I'd lean towards blocking.

FinalBlocker +1

(for the chosen ones, this is it https://bugzilla.redhat.com/show_bug.cgi?id=2137723 )

Note that if we slip for this, there is kind of an argument to do a mini-refresh before release: also pull in kernel 6.0, Plasma 5.26.1, possibly GNOME 43.1, Firefox 106.0.1, generally sorta freshen things up. It increases risk of new blockers showing up, but means we wouldn't be shipping with an EOL kernel and we'll have newer versions of everything than the new Ubuntu release (ahem).

What's the point of having a freeze period at all, then?

What's the point of having a freeze period at all, then?

In my take, if we knew about this upcoming disclosure, we'd have moved freeze period to a later point. And it'd still be controlled flow of cherry-picked and tested stuff flowing in.

I'm still not convinced it makes sense to slip the release when we have existing systems that will still need patching. However: if we do slip for this, please consider me firmly -1 on pulling in a whole slate of new versions of major packages.

I feel like I should remind people that, if not for a few genuine blockers, we would have shipped weeks ago. We would then be in no worse of a situation with Fedora 37 than we are for Fedora 35 and 36. If the disclosure had happened while in Freeze, that would change the calculus for me, but as it stands, I think we should proceed with a release (assuming no other blockers pop up before Go/No-Go).

What's amazing to me is that there are 11 +1 votes but still no explanation as to how this bug fits the criterion "cannot be satisfactorily resolved by a package update." The only way I could potentially consider this to fit the criterion were if anaconda netinstall were using it, but nobody has made this argument yet. The bug has not even been announced....

I agree with all of @sgallagh's comment including that my +1 is conditional on it impacting the installer and thus not fixable with an update.

But also this comment highlights the caveat of regression(s) resulting from the fix potentially leading to a three week slip.

I think we have to reevaluate the total vote once we have more information. And if there isn't some decently convincing statement of installer impact by go/no-go, I'm going to change my vote to -1. Not least of which is the pressure of staleness in other packages over a 2-3 week period, and the inclination to grant more freeze exceptions. Instead, release and do proper updates, per usual.

(for the chosen ones, this is it https://bugzilla.redhat.com/show_bug.cgi?id=2137723 )

Yeah. That's helpful for everyone trying to make an informed decision.

I side with @sgallagh on this one:

FinalBlocker -1

Note that if we slip for this, there is kind of an argument to do a mini-refresh before release: also pull in kernel 6.0, Plasma 5.26.1, possibly GNOME 43.1, Firefox 106.0.1, generally sorta freshen things up. It increases risk of new blockers showing up, but means we wouldn't be shipping with an EOL kernel and we'll have newer versions of everything than the new Ubuntu release (ahem).

What's the point of having a freeze period at all, then?

The point of the freeze is to work towards a target, but there does come a point where the freeze has gone on so long it'd be a bit embarrassing to ship the frozen bits and it makes an amount of sense to do a reset. We have actually done this at least once (I think twice) before, when freezes dragged on for a very long time.

@catanzaro I already said it's not just about the installer. Critical security issues in things people would be expected to use on the live image - e.g. web browsers and email clients - are also covered by that. This is not theoretical, we have blocked on such issues before. All sorts of things use openssl. Given that we know there is a critical issue in openssl and the prodsec team's advice is to update images, it seems like it would be a bold decision to just assume it's not going to affect anything we can't fix with an update. Which is what we'd be doing if we rejected this.

@catanzaro I already said it's not just about the installer. Critical security issues in things people would be expected to use on the live image - e.g. web browsers and email clients - are also covered by that.

With respect, this is kind of a weak argument. Web browsers and email clients almost inevitably have a serious security issue crop up within weeks of release. Just like this would have been if we had shipped on time. We don't retroactively update the install media for newer versions of Firefox, we just assume that people will install and upgrade.

We also respin x86 media every two weeks. The current flaw is that we don't do full deliverable composes with that frequency. If we did, these kind of issues would be easier to resolve.

What web browser or email client are you worried about? Regarding Fedora Workstation, Firefox certainly does not use OpenSSL, and we don't ship an email client. And regarding other software, almost all GNOME stuff uses GnuTLS rather than OpenSSL. So these concerns certainly don't apply to Workstation.

We still don't know anything about the vulnerability or have any reason to believe it affects common configurations or even that it affects client connections at all. Without any info, there is just no way to know. If we knew more, then we could make an informed decision, but we don't.

rpm and systemd both use OpenSSL. The latter just switched to it.

Metadata Update from @blockerbot:
- Issue status updated to: Closed (was: Open)

Release F37 is no longer tracked by BlockerBugs, closing this ticket.

Log in to comment on this ticket.

Metadata