Bug 1200446 - opensuse-release introduces fixed-URL repositories, fails without internet access
opensuse-release introduces fixed-URL repositories, fails without internet ac...
Status: IN_PROGRESS
Classification: openSUSE
Product: openSUSE Distribution
Classification: openSUSE
Component: Installation
Leap 15.5
Other openSUSE Leap 15.3
: P2 - High : Normal (vote)
: ---
Assigned To: Lubos Kocman
Jiri Srain
:
Depends on:
Blocks:
  Show dependency treegraph
 
Reported: 2022-06-11 17:39 UTC by Jens-U. Mozdzen
Modified: 2022-10-12 10:27 UTC (History)
11 users (show)

See Also:
Found By: ---
Services Priority:
Business Priority:
Blocker: ---
Marketing QA Status: ---
IT Deployment: ---


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Jens-U. Mozdzen 2022-06-11 17:39:04 UTC
Installing openSUSE 15.3 via AutoYaST fails for systems that have no Internet access:

- client system uses network sources (i. e. SUSE Manager / Cobbler) to access all repositories (oss, non-oss, update/backports, update/sle,...)

- installation of package openSUSE-release is mandatory

- openSUSE-release installs (along other files) four repo files:

/etc/zypp/repos.d/repo-backports-debug-update.repo
/etc/zypp/repos.d/repo-backports-update.repo
/etc/zypp/repos.d/repo-sle-debug-update.repo
/etc/zypp/repos.d/repo-sle-update.repo

- repo-backports-update.repo and repo-sle-update.repo are enabled per these files, and hard-coded reference downloads.opensuse.org:

/etc/zypp/repos.d/repo-backports-update.repo:enabled=1
/etc/zypp/repos.d/repo-backports-update.repo:baseurl=http://download.opensuse.org/update/leap/$releasever/backports/

/etc/zypp/repos.d/repo-sle-update.repo:enabled=1
/etc/zypp/repos.d/repo-sle-update.repo:baseurl=http://download.opensuse.org/update/leap/$releasever/sle/

Because of this, stage two of autoinstallation fails when refreshing libzypp repositories, as the client on purpose has no access to download.opensuse.org.

The contents of both channels are made available to the client via Cobbler-based repository URLs, but the installer needs manual intervention to skip the failing repository refresh. Additionally, the installed system also will fail the same, unless some post-installation task disables these "wrong" repositories.

This problem is blocking automated installation of Leap clients in environments without Internet access, hence "severity: major".
Comment 1 Imobach Gonzalez Sosa 2022-06-13 05:40:08 UTC
Hi Jens,

As you commented, those files are installed by openSUSE-release, so there is not much that AutoYaST can do. However, you could use a chroot-script[1] to disable those repositories so they are not used during the 2nd stage (e.g., changing enabled=1 to enabled=0).

Would it work for you?

Regards,
Imo

[1] https://documentation.suse.com/sles/15-SP3/single-html/SLES-autoyast/#chroot-scripts
Comment 2 Jens-U. Mozdzen 2022-06-14 13:38:55 UTC
Hi Imo,

> not much that AutoYaST can do

I guessed so... but as it is something that affects installation, I put it into this component.

> use a chroot-script[1] to disable those repositories
must have been a hard day, I was under the impression the package only got installed in stage 2 and therefore I'd need to find some "post-module" name to hook in. But it's much more likely that it gets installed in stage 1, so a chroot-script would be easy to use. I've forwarded that to the team and will report back once I have a result.

Generally speaking, I'd appreciate a "solution" where anyone doing such business has a standard way of detecting they're running under auto-installation (AKA "AutoYaST") and would refrain from adding own repositories then - it's job of the AutoYaST profile to do so, like it's already practiced with product and update channels now.

Jiri indicated he has something he could add to a more technical discussion, I'll get that rolling once we have a work-around in place and therefore relieved the current project pressure.

(not clearing needinfo yet.)
Comment 3 Jens-U. Mozdzen 2022-06-15 00:22:14 UTC
Hi Imo,

(In reply to Imobach Gonzalez Sosa from comment #1)
> However, you could use a chroot-script[1] to
> disable those repositories so they are not used during the 2nd stage (e.g.,
> changing enabled=1 to enabled=0).
> 
> Would it work for you?

feedback from my team was positive, the install no longer hit the need for manual intervention.

I'd still be in favor of a solution within the RPM that auto-detects the autoyast situation and acts accordingly without the need for extra hacks.
Comment 4 Imobach Gonzalez Sosa 2022-06-16 05:08:30 UTC
Hi Jens,

I can understand that getting those repositories for free is not exactly nice. But I am not sure whether AutoYaST should be responsible for that and, for sure, I would not expect the RPM to detect the situation. For the time being, I would consider using a chroot as a valid solution.

Jiri, do you have a different POV?

Regards,
Imo
Comment 5 Jens Mozdzen 2022-06-16 14:20:53 UTC
see also https://etherpad.opensuse.org/p/ReleaseEngineering-20220622#L56
Comment 6 Jiri Srain 2022-06-17 13:28:36 UTC
(In reply to Imobach Gonzalez Sosa from comment #4)
> Hi Jens,
> 
> I can understand that getting those repositories for free is not exactly
> nice. But I am not sure whether AutoYaST should be responsible for that and,
> for sure, I would not expect the RPM to detect the situation. For the time
> being, I would consider using a chroot as a valid solution.
> 
> Jiri, do you have a different POV?

I think that the script is the best option without changing the way the repos are introduced - and this would need to go to the openSUSE release manager, however, since you want the repos in most of other cases, I don't know what would be a better solution..

However, the -release package would need to avoid reintroducing them while updating
Comment 7 Imobach Gonzalez Sosa 2022-06-21 05:48:38 UTC
I would say we should turn this bug into a feature for proper discussion. What do you think?
Comment 8 Jens Mozdzen 2022-06-28 15:24:52 UTC
(In reply to Imobach Gonzalez Sosa from comment #7)
> I would say we should turn this bug into a feature for proper discussion.
> What do you think?

Hm, for sure this'd be a new feature.

There's a side effect I didn't stress in my original report, which might make this ticket a valid bug report: AutoYaST seems to have no setting that will make the "unable to refresh repository" message time out - the install, at least for us, forces user intervention to skip the update.

A different notion with regards to the bug title and if needed, I'll open a new ticket :)
Comment 9 Andrei Borzenkov 2022-07-14 06:40:47 UTC
(In reply to Jiri Srain from comment #6)
> since you want the repos in most of other cases, I don't
> know what would be a better solution..
> 
Why are these repos present in openSUSE-release in 15.4 at all? They were introduced in 15.3 because these extra repositories were forgotten at release and "thou shalt not change GA installation media" commandment. Does it mean Leap 15.4 still does not provide those repositories as part of standard installation?

Anyway, zypper service for openSUSE instead of static repo list would allow fixing such "missing" repositories in real time for anyone who receives updates from Internet, and anyone who does not use Internet does not need the repositories in openSUSE-release package anyway.

> However, the -release package would need to avoid reintroducing them while
> updating

The whole point of adding these two repositories to openSUSE-release package was reintroducing them while updating post-install. Now, repo files are marked as %config(noreplace) so RPM will not overwrite them (making it possible to disable); but if you removed these files, they will be installed again on update of openSUSE-release.
Comment 10 David Diaz 2022-07-20 09:10:08 UTC
(In reply to Jens Mozdzen from comment #8)
> (In reply to Imobach Gonzalez Sosa from comment #7)
> > I would say we should turn this bug into a feature for proper discussion.
> > What do you think?
> 
> Hm, for sure this'd be a new feature.
> 
> There's a side effect I didn't stress in my original report, which might
> make this ticket a valid bug report: AutoYaST seems to have no setting that
> will make the "unable to refresh repository" message time out - the install,
> at least for us, forces user intervention to skip the update.
> 
> A different notion with regards to the bug title and if needed, I'll open a
> new ticket :)

Hi Jens!

I think it would be better to open a new ticket if you don't mind. After it, I guess we can close this one as a FEATURE and use the proper channel to start the feature request discussion. Right?

Thanks!
Comment 11 Jens Mozdzen 2022-07-21 08:56:04 UTC
(In reply to David Diaz from comment #10)
> (In reply to Jens Mozdzen from comment #8)
> > (In reply to Imobach Gonzalez Sosa from comment #7)
> > > I would say we should turn this bug into a feature for proper discussion.
> > > What do you think?
> > 
> > Hm, for sure this'd be a new feature.
> > 
> > There's a side effect I didn't stress in my original report, which might
> > make this ticket a valid bug report: AutoYaST seems to have no setting that
> > will make the "unable to refresh repository" message time out - the install,
> > at least for us, forces user intervention to skip the update.
> > 
> > A different notion with regards to the bug title and if needed, I'll open a
> > new ticket :)
> 
> Hi Jens!
> 
> I think it would be better to open a new ticket if you don't mind.

https://bugzilla.suse.com/show_bug.cgi?id=1201751

> After it,
> I guess we can close this one as a FEATURE and use the proper channel to
> start the feature request discussion. Right?

Agreed.
Comment 12 David Diaz 2022-07-21 09:01:04 UTC
(In reply to Jens Mozdzen from comment #11)
> 
> https://bugzilla.suse.com/show_bug.cgi?id=1201751
> 
> > After it,
> > I guess we can close this one as a FEATURE and use the proper channel to
> > start the feature request discussion. Right?
> 
> Agreed.

Thank you!
Comment 13 Luna Jernberg 2022-08-08 13:07:06 UTC
This was discussed in todays 15.5 Features meeting
Comment 14 Lubos Kocman 2022-08-08 13:12:46 UTC
My personal preference would be delivering repos via service like SLES (provides the most flexibility, adding repos post-GA etc). However, I don't belive we would get enginering resource for this, specifically for very likely the last relase of Leap 15.X

Alternative proposal would be moving repo files in a separate package and simply recommending it by the release-package (e.g. existing rpm-repos-openSUSE-Leap)

https://build.opensuse.org/package/show/openSUSE:Factory/rpm-repos-openSUSE

(You can already find it for e.g Leap/DNF containers).

Would the alternative proposal work for you?

Thank you
Comment 15 Josef Reidinger 2022-08-15 08:14:01 UTC
set properly needinfo according to comment#14
Comment 16 Jens Mozdzen 2022-08-15 10:30:57 UTC
(In reply to Lubos Kocman from comment #14)
> Alternative proposal would be moving repo files in a separate package and
> simply recommending it by the release-package (e.g. existing
> rpm-repos-openSUSE-Leap)
> 
> https://build.opensuse.org/package/show/openSUSE:Factory/rpm-repos-openSUSE
> 
> (You can already find it for e.g Leap/DNF containers).
> 
> Would the alternative proposal work for you?

I believe this should work - we need to break the fixed requirement to set these repositories, and making them only "recommended" (and thus at worse defer installation by blocking the RPM via according AutoYaST setting) would solve the problem for us.
Comment 17 Lukas Ocilka 2022-08-15 12:02:38 UTC
Reassigning to Lubos to implement it via that "recommended" dependency.
Comment 18 Lubos Kocman 2022-09-01 11:12:11 UTC
Oky that works for me. I think we should probably do the same for Leap Micro as well.
Comment 19 Lubos Kocman 2022-09-01 11:38:05 UTC
https://build.opensuse.org/request/show/1000660
Comment 20 Ladislav Slezák 2022-09-01 12:33:25 UTC
Using "Recommends" will not solve the problem fully, it will only allow using a workaround and it would introduce just another problems (the repositories might be missing in the system, I guess these should be always present).

I'm not sure if there is a better solution than the already mentioned AutoYaST chroot-script.

One possibility is to adapt the openSUSE-release %post script to disable the repositories if network is not available (or the servers not reachable).

But there is a problem that the network configuration used during AutoYaST installation can be different than the network configuration used in the installed system (this is a special AutoYaST feature added on request). So the network might not be available during openSUSE-release installation but it might be available later. Or the other way round... :-/
Comment 21 Lubos Kocman 2022-09-01 12:59:44 UTC
Max mentioned that he thinks that this issue might not appear in 15.4 nor latest Quaterly Update of 15.3. Seems like we've had issue in skelcd-control which is also being parsed by autoyast.

What we'd like ot see is to double check on current 15.3 and 15 SP4 installation if this is still an issue. If so then Max has concerns why this shouldn't be touched in 15.5 (last release, possible brings new issues).

If we have solution / workaround for this then I'd recommend perhaps to document it. Go with the rpm-repos-openSUSE (which is currently only for DNF) to Factory first. And then take it perhaps for ALP.

We've had rough upgrades with CtLG and this could trigger another set of complains in 15.5 retrospective. I've learned that documenting change in release-notes and System upgrade wiki doesn't mean that people will read it :-) They'll hit issue, we'll forward them to information but it will still be on retro :-)


So please consider my previous SR as rejected, as it doesn't follow factory first. I can try to submit the change to factory (after reworking rpm-repos-openSUSE) and we can re-discuss this once it's accepted. 

But I'm afraid it's only moving the issue arround and in case of Leap triggers possible new issues with zypper dup --releasever 15.5.
Comment 22 Jens Mozdzen 2022-09-01 13:13:18 UTC
Could you introduce an installation-time flag that could be set by the admin (or rather the AutoYaST file, in the reported case) with a "do not add repositories" semantics?

If someone sets this flag, it's that one's responsibility to make sure all required repositories are available.

If that flag were introduced via some wrapper that you ask everyone to use, then this wrapper could also record all blocked "add repository" attempts and report these in the installation logs. This would help in debugging issues if the "admin" (the one setting the "don't add repositories" flag) messed up the extra repository setup. Potentionally the "wrapper" could be "zypper", if that's what every RPM script is using to add repositories.

Yes, indeed, the wrapper would need to be able to distinguish between repositories added by the installation environment, vs. those added by RPM scripts - if the installation environment does use the same mechanism to update the repository list. But even then, it could be handled in a manner that the installation env sets the flag *after* having set up all "manual" repositories, blocking only further updates.
Comment 23 Ladislav Slezák 2022-09-01 13:30:13 UTC
(In reply to Jens Mozdzen from comment #22)
> Could you introduce an installation-time flag that could be set by the admin
> (or rather the AutoYaST file, in the reported case) with a "do not add
> repositories" semantics?

No, the repositories can be defined at many places: the system, RPM packages, AutoYaST profile, repository service, libzypp plugin script, boot parameters, add-on.xml on the medium, control.xml on the medium, SMT/RMT/SCC...

And you actually want that repositories to be present, just in same cases you might need to disable them.

And that disabling can be quite easily implemented by an AutoYaST chroot script, I do not think we need any special support for this relatively rare case.
Comment 24 Lubos Kocman 2022-10-12 10:27:19 UTC
https://github.com/openSUSE/openSUSE-repos

our take on this one, factory already has it 
Leap 15.5 SR https://build.opensuse.org/request/show/1010187

We're pending integration pieces.

Our plan is that new Factory deployment will have it. For existing systems we'll have to announce it news-o-o/factory. And also put it to release notes for Leap 15.5

YaST team, is there anything on your side that would have to change in installer or any other component that you own?