Bug 1142733 - apache2.service: Failed to set up mount namespacing: No such file or directory
apache2.service: Failed to set up mount namespacing: No such file or directory
Status: RESOLVED WONTFIX
Classification: openSUSE
Product: openSUSE Tumbleweed
Classification: openSUSE
Component: Basesystem
Current
Other Other
: P5 - None : Normal (vote)
: ---
Assigned To: systemd maintainers
E-mail List
:
Depends on:
Blocks:
  Show dependency treegraph
 
Reported: 2019-07-25 06:13 UTC by Jiri Slaby
Modified: 2020-09-07 13:20 UTC (History)
4 users (show)

See Also:
Found By: ---
Services Priority:
Business Priority:
Blocker: ---
Marketing QA Status: ---
IT Deployment: ---


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Jiri Slaby 2019-07-25 06:13:07 UTC
This midnight, after/during logrotate:

> Jul 25 00:00:01 anemoi systemd[24180]: apache2.service: Failed to set up mount namespacing: No such file or directory
> Jul 25 00:00:01 anemoi systemd[24180]: apache2.service: Failed at step NAMESPACE spawning /usr/sbin/start_apache2: No such file or directory
> Jul 25 00:00:01 anemoi systemd[1]: Reloading The Apache Webserver.
> Jul 25 00:00:01 anemoi systemd[1]: apache2.service: Control process exited, code=exited, status=226/NAMESPACE
> Jul 25 00:00:01 anemoi systemd[1]: apache2.service: Failed with result 'exit-code'.
> Jul 25 00:00:01 anemoi systemd[1]: Reload failed for The Apache Webserver.


The server stopped working due to this error. 

# systemctl start apache
started the server normally in the morning.

I found a similar bug in debian:
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=919602

I also found this upstream bug, but that should be fixed in 239 already (we have 242):
https://github.com/systemd/systemd/issues/10032
Comment 1 Franck Bui 2019-07-29 09:51:55 UTC
What happens if you try to reload manually apache (systemctl reload apache2) ?
Comment 2 Jiri Slaby 2019-07-31 07:28:19 UTC
(In reply to Franck Bui from comment #1)
> What happens if you try to reload manually apache (systemctl reload apache2)
> ?

Jul 31 09:27:36 anemoi systemd[1]: Reloading The Apache Webserver.
Jul 31 09:27:36 anemoi systemd[1]: Reloaded The Apache Webserver.

It worked.
Comment 3 Franck Bui 2019-07-31 09:47:19 UTC
Do you know how apache reloading is triggered during logrotate ?
Comment 4 Per Jessen 2019-07-31 10:11:36 UTC
Probably unrelated, but I had a similar situation, see bug#996203
Comment 5 Jiri Slaby 2019-07-31 10:15:54 UTC
(In reply to Franck Bui from comment #3)
> Do you know how apache reloading is triggered during logrotate ?

/etc/logrotate.d/apache2 says:
    postrotate
     systemctl reload apache2.service
     sleep 60
    endscript
Comment 6 Franck Bui 2019-08-02 14:05:59 UTC
(In reply to Per Jessen from comment #4)
> Probably unrelated, but I had a similar situation, see bug#996203

Well it looks to me that is the same issue.
Comment 7 Franck Bui 2019-08-02 14:07:32 UTC
How easy is this reproducible for you Jiri ?

Also it might be interesting to reproduce with systemd debug logs enabled, but not sure if it's feasible...
Comment 8 Jiri Slaby 2019-08-07 09:26:53 UTC
(In reply to Franck Bui from comment #7)
> How easy is this reproducible for you Jiri ?

I saw it only once, and not since this report. It must have been some race condition or something peculiar.

> Also it might be interesting to reproduce with systemd debug logs enabled,
> but not sure if it's feasible...

Infeasible, it seems.
Comment 9 Jiri Slaby 2019-09-04 05:36:44 UTC
It's pretty rare, but it happens:
> Sep 04 00:00:02 anemoi systemd[1]: Reloading The Apache Webserver.
> Sep 04 00:00:02 anemoi systemd[18662]: apache2.service: Failed to set up mount namespacing: No such file or directory
> Sep 04 00:00:02 anemoi systemd[18662]: apache2.service: Failed at step NAMESPACE spawning /usr/sbin/start_apache2: No such file or directory
> Sep 04 00:00:02 anemoi systemd[1]: apache2.service: Control process exited, code=exited, status=226/NAMESPACE
> Sep 04 00:00:02 anemoi su[18637]: (to root) root on none
> Sep 04 00:00:02 anemoi systemd[1]: apache2.service: Failed with result 'exit-code'.
> Sep 04 00:00:02 anemoi systemd[1]: Reload failed for The Apache Webserver.
> Sep 04 00:00:02 anemoi logrotate[18612]: Job for apache2.service failed because the control process exited with error code.
> Sep 04 00:00:02 anemoi logrotate[18612]: See "systemctl status apache2.service" and "journalctl -xe" for details.
Comment 10 Franck Bui 2020-07-22 16:44:37 UTC
@Jiri, did you encounter new occurrences of this issue recently, that is with a recent version of systemd ?

I'm asking because I'm out of idea and it would be nice if this could be reproduced on latest versions so we could open a bug against upstream and perhaps get some help.
Comment 11 Jiri Slaby 2020-07-23 08:22:38 UTC
I didn't see it for quite some time. Despite the system is not completely up-to-date:
NAME="openSUSE Tumbleweed"
# VERSION="20200414"

the issue didn't recur lately (famous last few words).
Comment 12 Jiri Slaby 2020-07-23 08:24:16 UTC
$ systemctl --version 
systemd 244 (+suse.138.gf8adabc2b1)
+PAM -AUDIT +SELINUX -IMA +APPARMOR -SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD -IDN2 -IDN +PCRE2 default-hierarchy=hybrid
$ uname -a
Linux anemoi 5.6.2-1.gc8170d6-default #1 SMP Thu Apr 2 06:31:32 UTC 2020 (c8170d6) x86_64 x86_64 x86_64 GNU/Linux
Comment 13 Fabian Vogt 2020-07-23 08:31:18 UTC
I get this error after rm -rf /tmp/systemd-private-*-apache2.service*/, maybe something cleaned /tmp? Newer systemd has some better logging for this:

Jul 23 08:28:52 localhost systemd[1]: Reloading The Apache Webserver.
Jul 23 08:28:52 localhost systemd[2199]: apache2.service: Failed to set up mount namespacing: /run/systemd/unit-root/tmp: No such file or directory
Jul 23 08:28:52 localhost systemd[2199]: apache2.service: Failed at step NAMESPACE spawning /usr/sbin/start_apache2: No such file or directory
Comment 14 Franck Bui 2020-07-23 09:26:54 UTC
(In reply to Fabian Vogt from comment #13)
> I get this error after rm -rf /tmp/systemd-private-*-apache2.service*/,

Hm what did trigger this ?

Did you do that manually or maybe the tmpfiles stuff ?

> maybe something cleaned /tmp? 

yes it's likely that something messed up with the private directory of apache service but 

> Newer systemd has some better logging for this:
> 
> Jul 23 08:28:52 localhost systemd[1]: Reloading The Apache Webserver.
> Jul 23 08:28:52 localhost systemd[2199]: apache2.service: Failed to set up
> mount namespacing: /run/systemd/unit-root/tmp: No such file or directory
> Jul 23 08:28:52 localhost systemd[2199]: apache2.service: Failed at step
> NAMESPACE spawning /usr/sbin/start_apache2: No such file or directory

Interesting, thanks, I'll take a new look into this issue.
Comment 15 Fabian Vogt 2020-07-23 10:06:12 UTC
(In reply to Franck Bui from comment #14)
> (In reply to Fabian Vogt from comment #13)
> > I get this error after rm -rf /tmp/systemd-private-*-apache2.service*/,
> 
> Hm what did trigger this ?
> Did you do that manually or maybe the tmpfiles stuff ?

Yes, manually. I'm unable to reproduce this bug.

> > maybe something cleaned /tmp? 
> 
> yes it's likely that something messed up with the private directory of
> apache service but 
> 
> > Newer systemd has some better logging for this:
> > 
> > Jul 23 08:28:52 localhost systemd[1]: Reloading The Apache Webserver.
> > Jul 23 08:28:52 localhost systemd[2199]: apache2.service: Failed to set up
> > mount namespacing: /run/systemd/unit-root/tmp: No such file or directory
> > Jul 23 08:28:52 localhost systemd[2199]: apache2.service: Failed at step
> > NAMESPACE spawning /usr/sbin/start_apache2: No such file or directory
> 
> Interesting, thanks, I'll take a new look into this issue.

This is only an example, so no need to debug that :-)
Comment 16 Franck Bui 2020-07-23 10:38:55 UTC
(In reply to Jiri Slaby from comment #11)
> I didn't see it for quite some time. Despite the system is not completely
> up-to-date:
> NAME="openSUSE Tumbleweed"
> # VERSION="20200414"
> 
> the issue didn't recur lately (famous last few words).

Ok at least the version you're using has a better logging now as Fabian pointed out.

So I propose to close this bug as "workform", and if you can reproduce again then reopen this bug and provide the improved logs.

Would that work for you ?
Comment 18 Franck Bui 2020-08-04 08:23:09 UTC
OK I don't think there's any point in keeping this bug open unless more info is provided.

Jiri, please feel free to re-open if you can reproduce with the latest version of systemd so we will have hopefully more details on the failure.

Until that I'm closing this bug.

Hope that makes sense.
Comment 19 Swamp Workflow Management 2020-09-01 16:34:44 UTC
SUSE-RU-2020:2411-1: An update that has 6 recommended fixes can now be installed.

Category: recommended (moderate)
Bug References: 1142733,1146991,1158336,1172195,1172824,1173539
CVE References: 
JIRA References: 
Sources used:
SUSE Linux Enterprise Module for Basesystem 15-SP2 (src):    systemd-234-24.58.1
SUSE Linux Enterprise Module for Basesystem 15-SP1 (src):    systemd-234-24.58.1
SUSE Linux Enterprise Installer 15-SP1 (src):    systemd-234-24.58.1

NOTE: This line indicates an update has been released for the listed product(s). At times this might be only a partial fix. If you have questions please reach out to maintenance coordination.
Comment 20 Swamp Workflow Management 2020-09-05 16:20:01 UTC
openSUSE-RU-2020:1340-1: An update that has 6 recommended fixes can now be installed.

Category: recommended (moderate)
Bug References: 1142733,1146991,1158336,1172195,1172824,1173539
CVE References: 
JIRA References: 
Sources used:
openSUSE Leap 15.1 (src):    systemd-234-lp151.26.25.1, systemd-mini-234-lp151.26.25.1
Comment 21 Swamp Workflow Management 2020-09-07 13:20:13 UTC
openSUSE-RU-2020:1355-1: An update that has 6 recommended fixes can now be installed.

Category: recommended (moderate)
Bug References: 1142733,1146991,1158336,1172195,1172824,1173539
CVE References: 
JIRA References: 
Sources used:
openSUSE Leap 15.2 (src):    systemd-234-lp152.31.7.1, systemd-mini-234-lp152.31.7.1