FAH should mostly hit deltas which don't seem likely to trigger this right? Though of course that isn't the same as "won't".

I think my vote is to ask infra to turn off http2 server side for a bit, let updates trickle out that disable it client side. Though I saw some discussion about a libcurl fix - if someone has more info can they add it to the libostree issue?

@puiterwijk might have been working on a libcurl fix...any comments Patrick?

@puiterwijk might have been working on a libcurl fix...any comments Patrick?

talked with patrick in IRC, follow discussion in https://github.com/ostreedev/ostree/issues/1362

talked with patrick in IRC, follow discussion in https://github.com/ostreedev/ostree/issues/1362

and here: https://github.com/ostreedev/ostree/issues/878#issuecomment-354803583

Here's how you can hotfix this issue:

ostree admin unlock
rpm -Uvh https://kojipkgs.fedoraproject.org//packages/ostree/2017.15/1.fc27/x86_64/ostree-{,grub2-,libs-}2017.15-1.fc27.x86_64.rpm
systemctl restart rpm-ostreed

Then appy updates as normal, e.g.:

rpm-ostree upgrade

one thing to consider here is that for FAH we produce new media every two weeks. For FAW we just have the ISO that was generated before F27 major release. So basically when doing an upgrade from FAW ISO to latest users will always hit the http2 bug (because the connection resets after a certain number of files have been downloaded). This explains why FAW users are seeing this more than FAH users.

One potentially ugly workaround (to prevent us from having to build new media) is to redirect https://dl.fedoraproject.org/ostree/27/ to a non https location (Since h2 negotiation is done during the TLS handshake) and make that non https location only serve content that is gpg signed. We have signature checking enabled in the image so this should be ok.

@miabbott / @walters / @dustymabe:

Any of you folks want to take this blog post? If not, who do you think would be a good person to do this? If no one else comes to mind I will be the fall back and write one.

AIUI the current status is that Fedora Infrastructure has disabled HTTP2 at entrypoints:

https://infrastructure.fedoraproject.org/cgit/ansible.git/commit/?id=076286e4c401ead7f36187a949245e2fa23251b3
https://infrastructure.fedoraproject.org/cgit/ansible.git/commit/?id=5f4ea314a0b869e537742df632cd930a9a2705d0

I think the agreement was to do that for a short period of time - 2 weeks?

I think the agreement was to do that for a short period of time - 2 weeks?

Right, I wonder if this is even worth a blog post at this point. I think as long as we can fix the FAW ISO install+upgrade path then we don't need to.

Right, I wonder if this is even worth a blog post at this point. I think as long as we can fix the FAW ISO install+upgrade path then we don't need to.

Agreed; with HTTP2 disabled for a while, most users should avoid the error/problem.

Right, I wonder if this is even worth a blog post at this point. I think as long as we can fix the FAW ISO install+upgrade path then we don't need to.

Agreed; with HTTP2 disabled for a while, most users should avoid the error/problem.

In that case is it OK to close this issue?

probably need to open a new ticket to address https://pagure.io/atomic-wg/issue/405#comment-486841 and then close this ticket.

FYI request to have ostree pass version information which will help us workaround issues like this server side in the future: https://github.com/ostreedev/ostree/issues/1405

FYI request to have ostree pass version information which will help us workaround issues like this server side in the future: https://github.com/ostreedev/ostree/issues/1405

That issue is now fixed and ostree will start passing version information in our next two week release (in a few weeks).

That issue is now fixed and ostree will start passing version information in our next two week release (in a few weeks).

ostree made it to stable so we should get the version information in our http requests and can use this data to redirect clients that we need to redirect

probably need to open a new ticket to address https://pagure.io/atomic-wg/issue/405#comment-486841 and then close this ticket.

Are we looking for creating ticket about this comment or fixes made in ostree is sufficent for future workaround for OSTree HTTP2 issue?

I think we are good now. HTTP2 has been disabled in ostree and no one should have issues with media from F28.