Bug 1061195 - kernel-default-4.14-rc2 with apparmor enabled breaks dnsmasq and others cannot create AF_UNIX sockets
kernel-default-4.14-rc2 with apparmor enabled breaks dnsmasq and others canno...
Status: RESOLVED FIXED
Classification: openSUSE
Product: openSUSE Tumbleweed
Classification: openSUSE
Component: AppArmor
Current
Other Other
: P5 - None : Normal (vote)
: ---
Assigned To: Christian Boltz
E-mail List
:
Depends on:
Blocks:
  Show dependency treegraph
 
Reported: 2017-09-30 19:00 UTC by Vlastimil Babka
Modified: 2017-11-15 15:35 UTC (History)
7 users (show)

See Also:
Found By: ---
Services Priority:
Business Priority:
Blocker: ---
Marketing QA Status: ---
IT Deployment: ---


Attachments
a subset of apparmor lines in audit.log (15.39 KB, text/plain)
2017-10-01 16:45 UTC, Vlastimil Babka
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Vlastimil Babka 2017-09-30 19:00:53 UTC
After upgrade to kernel-default-4.14-rc2 from kernel:HEAD, dnsmasq failed to start:

dnsmasq.service: Main process exited, code=exited, status=3/NOTIMPLEMENTED
running manually:
dnsmasq: cannot open log permission denied: permission denied

strace showed that each attempt for socket(AF_UNIX...) returns EACCES

Interestingly, things worked with 4.14-rc1. During the reboots I also noticed some error messages from nscd that could not open unix sockets.

Running git log on 4.14-rc2 grepping for socket, I noticed commit 651e28c5537a ("apparmor: add base infastructure for socket mediation") which made me suspect apparmor. Indeed, after disabling apparmor, things work again on 4.14-rc2.

So I understand that apparmor learned to restrict more things, and current rules are not prepared for that?
Comment 1 Christian Boltz 2017-10-01 14:56:19 UTC
Can you please check your /var/log/audit/audit.log (or syslog or journal or dmesg, depending on your log configuration) and attach the AppArmor-related messages?
Comment 2 Vlastimil Babka 2017-10-01 16:45:42 UTC
Created attachment 742664 [details]
a subset of apparmor lines in audit.log
Comment 3 Jeff Mahoney 2017-10-02 14:40:29 UTC
We've carried the network mediation patches for a while.  They landed upstream in -rc2, which must've been in a slightly different form.  Hopefully it should be as simple as refreshing our userspace tools to get the updated profiles.  We can't be the only ones seeing this issue.
Comment 4 Christian Boltz 2017-10-02 22:40:03 UTC
I updated to Kernel:HEAD today, and had an interesting evening in #apparmor ;-)

Regarding the now upstreamed network patch, let me just quote John:
    <‎jjohansen‎>‎ cboltz: ah yes, the upstreamed version fixes a couple 
                holes in the old patch suse carried

One of these "holes" were unix events, which explains the denials you noticed (and that I also see now after installing 4.14rc2).

The final solution will be to add some "unix" rules - but that's hard at the moment because 4.14 doesn't log all details needed for unix rules.

Instead, I'll add a temporary patch for abstractions/nameservice that adds
    network unix dgram,
    network unix stream,
(including a TODO note to replace it as soon as support for unix rules was upstreamed, probably 4.15). These rules are broader than needed, but should avoid user-visible breakage - and at least with 4.14, unix rules would get downgraded to network unix anyway ;-)

Do you have any ETA when 4.14 will enter Tumbleweed?

I want to have the updated abstractions/nameservice in place before 4.14 enters Tumbleweed, otherwise *lots of* things will break. To give you an impression what "lots of" means - I had to adjust 40 profiles on my laptop ;-)
Comment 5 Michal Kubeček 2017-10-03 06:54:11 UTC
(In reply to Christian Boltz from comment #4)
> Do you have any ETA when 4.14 will enter Tumbleweed?

New kernel version is usually submitted shortly after upstream release which
in case of 4.14 is most likely going to be in 5 or 6 weeks (we are at rc3 now
and last RC tends to be rc7 or rc8). After that, it depends on how many issues
are found (and how serious). So I would say you can take 5 weeks for granted
but it might be a bit more.
Comment 6 Bernhard Wiedemann 2017-10-04 12:06:33 UTC
This is an autogenerated message for OBS integration:
This bug (1061195) was mentioned in
https://build.opensuse.org/request/show/531184 Factory / apparmor
Comment 8 Doug Smythies 2017-10-07 16:24:06 UTC
Jeff wrote:
> We can't be the only ones seeing this issue.

I am seeing issues, not exactly the same, with Ubuntu.

My test Ubuntu 16.04.3 server fails to start mysql and libvirtd due to many apparmor "DENIED" errors.

My Ubuntu 17.10 development desktop VM fails to acquire an IP address due to dhclient apparmor "DENIED" errors.

The problems came up between kernels 4.14-rc1 and 4.14-rc2, and via kernel bisection was isolated to commit 651e28c5537abb39076d3949fb7618536f1d242e - apparmor: add base infastructure for socket mediation.

I found this thread: https://lkml.org/lkml/2017/10/3/2 , which pointed me here.
The thread debates if this is a kernel regression or not. In my opinion it is a kernel regression. I have passed my information along to the author of the commit.

Jeff wrote:
> We've carried the network mediation patches for a while.
> They landed upstream in -rc2, which must've been in a slightly different form.

I'm told that Ubuntu has carried them for awhile also. And yes, there are significant differences between Ubuntu kernel 4.13.0.12 and what ended up in mainline kernel 4.14-rc2.
Comment 9 Christian Boltz 2017-10-07 17:47:41 UTC
(In reply to Doug Smythies from comment #8)
> I found this thread: https://lkml.org/lkml/2017/10/3/2 , which pointed me
> here.
> The thread debates if this is a kernel regression or not. In my opinion it
> is a kernel regression. I have passed my information along to the author of
> the commit.

I somewhat understand why you (as a user) call this a regression ;-) - but I'd call adding support for new rule types (which weren't mediated before and therefore always allowed) a feature, and it was expected that we'll need to add some rules to the abstractions to avoid user-visible breakage.

That's why I'm currently using a 4.14 rc kernel - basically I'm eating my own dogfood to avoid trouble for everybody else ;-)  The first SR is already on its way to Tumbleweed, weeks before kernel 4.14 is expected there. And I'm also in contact with the libvirt maintainer to make sure the libvirtd profile gets updated (bug 1060860).

I've seen in the lkml discussion that you are using the latest kernel (I'd guess from Kernel:HEAD) on Leap. While this isn't officially supported, I'll see if I can include the updated abstractions with the next maintenance update. Until then, feel free to apply the patch from SR 531184 manually.
Comment 10 Jeff Mahoney 2017-10-07 22:33:23 UTC
Thanks for your effort on this, Christian.

I agree that users running an -rc kernel need to be prepared to deal with issues like this.  That's not to say I don't appreciate users doing exactly that and reporting bugs in the process.  I view this as a profile compatibility issue and one that, as you say, will be fixed before 4.14 hits TW as a real kernel update.
Comment 11 Christian Boltz 2017-10-20 20:49:03 UTC
It turned out that the added "network unix dgram/stream" rules is not really needed. Let me explain ;.-)

In theory apparmor_parser should downgrade the "unix" rules in abstractions/base to "network unix" rules (when using Kernel < 4.15), which allows more than "network unix dgram/stream".

In practise this rule downgrade was broken in apparmor_parser, and will be fixed in AppArmor 2.11.1, 2.10.3 and 2.9.5.

I'll keep the patch adding "network unix dram/stream" to abstractions/nameservice in the Tumbleweed package just to be double sure - but already test locally with 2.11.1 and these rules removed. I'll submit 2.11.1 to Tumbleweed as soon as the currently pending SR 534597 gets accepted.

For Leap, I'll prepare 2.10.3 update packages over the weekend which include the fixed apparmor_parser, and since Leap usually doesn't run with Kernel 4.14 (unless someone uses the KOTD repo), the safety net shouldn't be needed there.
I'll add a pointer to 2.10.3 packages as soon as they are ready, so Leap+KOTD users can test them.
Comment 12 Bernhard Wiedemann 2017-10-25 22:01:20 UTC
This is an autogenerated message for OBS integration:
This bug (1061195) was mentioned in
https://build.opensuse.org/request/show/536621 Factory / apparmor
Comment 13 Christian Boltz 2017-10-29 16:11:21 UTC
The updated package reached Tumbleweed today.

I also submitted an update for Leap 42.2 and 42.3 - SR 537428
Comment 14 Bernhard Wiedemann 2017-10-29 17:02:37 UTC
This is an autogenerated message for OBS integration:
This bug (1061195) was mentioned in
https://build.opensuse.org/request/show/537428 42.2+42.3 / apparmor
Comment 15 Bernhard Wiedemann 2017-10-29 19:00:57 UTC
This is an autogenerated message for OBS integration:
This bug (1061195) was mentioned in
https://build.opensuse.org/request/show/537444 42.2+42.3 / apparmor
Comment 16 Swamp Workflow Management 2017-11-15 14:07:51 UTC
openSUSE-RU-2017:3019-1: An update that has three recommended fixes can now be installed.

Category: recommended (moderate)
Bug References: 1057900,1061195,1062244
CVE References: 
Sources used:
openSUSE Leap 42.3 (src):    apparmor-2.10.3-16.1
openSUSE Leap 42.2 (src):    apparmor-2.10.3-12.6.1