Bug 1170826 - Go packages miss binutils-gold dependency
Go packages miss binutils-gold dependency
Status: NEW
Classification: openSUSE
Product: openSUSE Tumbleweed
Classification: openSUSE
Component: Development
Current
aarch64 Other
: P5 - None : Normal (vote)
: ---
Assigned To: Jeff Kowalczyk
E-mail List
:
Depends on:
Blocks:
  Show dependency treegraph
 
Reported: 2020-04-29 13:15 UTC by Guillaume GARDET
Modified: 2022-06-17 08:02 UTC (History)
7 users (show)

See Also:
Found By: ---
Services Priority:
Business Priority:
Blocker: ---
Marketing QA Status: ---
IT Deployment: ---


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Guillaume GARDET 2020-04-29 13:15:26 UTC
On arm and aarch64, 'go' requires 'binutils-gold' dependency at runtime, but go packages only require it as BuildRequires. So, binutils-gold must be installed manually, which breaks some builds in OBS, such as rclone [0].

Without 'binutils-gold', we ge the following error:
[  138s] /usr/lib64/go/1.11/pkg/tool/linux_arm64/link: running gcc failed: exit status 1
[  138s] collect2: fatal error: cannot find 'ld'

So, we need to add 'Requires: binutils-gold' to all go1.x packages.

[0]: https://build.opensuse.org/package/show/network/rclone
Comment 1 Guillaume GARDET 2020-04-29 13:26:29 UTC
SR to devel projects for go >= 1.10:
* go1.10: https://build.opensuse.org/request/show/798922
* go1.11: https://build.opensuse.org/request/show/798918
* go1.12: https://build.opensuse.org/request/show/798917
* go1.13: https://build.opensuse.org/request/show/798919
* go1.14: https://build.opensuse.org/request/show/798920

It should probably be updated in SLE/Leap as well.
Comment 2 Swamp Workflow Management 2020-04-30 19:00:06 UTC
This is an autogenerated message for OBS integration:
This bug (1170826) was mentioned in
https://build.opensuse.org/request/show/799322 Factory / go1.10
https://build.opensuse.org/request/show/799324 Factory / go1.11
https://build.opensuse.org/request/show/799325 Factory / go1.12
https://build.opensuse.org/request/show/799326 Factory / go1.13
https://build.opensuse.org/request/show/799327 Factory / go1.14
Comment 4 Guillaume GARDET 2020-05-19 06:20:41 UTC
(In reply to Swamp Workflow Management from comment #2)
> This is an autogenerated message for OBS integration:
> This bug (1170826) was mentioned in
> https://build.opensuse.org/request/show/799322 Factory / go1.10
> https://build.opensuse.org/request/show/799324 Factory / go1.11
> https://build.opensuse.org/request/show/799325 Factory / go1.12
> https://build.opensuse.org/request/show/799326 Factory / go1.13
> https://build.opensuse.org/request/show/799327 Factory / go1.14

Those updates should probably go to SLE as well. See https://build.opensuse.org/package/show/network/rclone build now succeeds on Factory/aarch64, whereas it fails on SLE15/aarch64

@Jeff, what do you think?
Comment 5 Swamp Workflow Management 2020-05-25 07:13:34 UTC
openSUSE-RU-2020:0714-1: An update that has one recommended fix can now be installed.

Category: recommended (moderate)
Bug References: 1170826
CVE References: 
Sources used:
openSUSE Leap 15.1 (src):    go1.10-1.10.8-lp151.2.6.1
Comment 6 Swamp Workflow Management 2020-05-28 16:13:43 UTC
openSUSE-RU-2020:0727-1: An update that has one recommended fix can now be installed.

Category: recommended (moderate)
Bug References: 1170826
CVE References: 
Sources used:
openSUSE Leap 15.1 (src):    go1.12-1.12.17-lp151.2.33.2
Comment 7 Swamp Workflow Management 2020-05-30 16:13:17 UTC
openSUSE-RU-2020:0745-1: An update that has one recommended fix can now be installed.

Category: recommended (moderate)
Bug References: 1170826
CVE References: 
Sources used:
openSUSE Leap 15.1 (src):    go1.11-1.11.13-lp151.2.18.1
Comment 8 Swamp Workflow Management 2020-06-04 01:13:30 UTC
openSUSE-RU-2020:0769-1: An update that has one recommended fix can now be installed.

Category: recommended (moderate)
Bug References: 1170826
CVE References: 
Sources used:
openSUSE Backports SLE-15-SP1 (src):    go1.10-1.10.8-bp151.2.3.1
Comment 9 Swamp Workflow Management 2020-06-25 19:13:01 UTC
openSUSE-RU-2020:0867-1: An update that has two recommended fixes can now be installed.

Category: recommended (moderate)
Bug References: 1149259,1170826
CVE References: 
Sources used:
openSUSE Leap 15.1 (src):    go1.13-1.13.11-lp151.2.1
Comment 11 Swamp Workflow Management 2020-07-30 13:16:41 UTC
openSUSE-RU-2020:1113-1: An update that has one recommended fix can now be installed.

Category: recommended (moderate)
Bug References: 1170826
CVE References: 
JIRA References: 
Sources used:
openSUSE Leap 15.2 (src):    go1.12-1.12.17-lp152.3.3.1
Comment 12 Marcus Meissner 2020-08-05 07:20:30 UTC
we are not shipping binutils-gold on SLE yet, so this will be tricky.

How important is it?
Comment 13 Richard Biener 2020-08-05 07:27:45 UTC
(In reply to Guillaume GARDET from comment #0)
> On arm and aarch64, 'go' requires 'binutils-gold' dependency at runtime, but
> go packages only require it as BuildRequires. So, binutils-gold must be
> installed manually, which breaks some builds in OBS, such as rclone [0].
> 
> Without 'binutils-gold', we ge the following error:
> [  138s] /usr/lib64/go/1.11/pkg/tool/linux_arm64/link: running gcc failed:
> exit status 1
> [  138s] collect2: fatal error: cannot find 'ld'

This is not a good failure description since 'ld' is provided by
plain binutils.  What exact gold feature do they require that is not
provided by GNU ld?

> So, we need to add 'Requires: binutils-gold' to all go1.x packages.
> 
> [0]: https://build.opensuse.org/package/show/network/rclone
Comment 15 Jeff Kowalczyk 2020-08-13 15:22:39 UTC
It does seem that go1.x on ARM depends on the gold linker for stated reasons of GNU linker behavior with COPY relocations. The error "collect2: fatal error: cannot find 'ld'" likely results from the flag "-fuse-ld=gold" being automatically applied:

go1.14 $ grep gold -B 19 -A 3 src/cmd/link/internal/ld/lib.go
if ctxt.IsELF && ctxt.DynlinkingGo() {
    // We force all symbol resolution to be done at program startup
    // because lazy PLT resolution can use large amounts of stack at
    // times we cannot allow it to do so.
    argv = append(argv, "-Wl,-znow")

    // Do not let the host linker generate COPY relocations. These
    // can move symbols out of sections that rely on stable offsets
    // from the beginning of the section (like sym.STYPE).
    argv = append(argv, "-Wl,-znocopyreloc")

    if ctxt.Arch.InFamily(sys.ARM, sys.ARM64) && objabi.GOOS == "linux" {
        // On ARM, the GNU linker will generate COPY relocations
        // even with -znocopyreloc set.
        // https://sourceware.org/bugzilla/show_bug.cgi?id=19962
        //
        // On ARM64, the GNU linker will fail instead of
        // generating COPY relocations.
        //
        // In both cases, switch to gold.
        argv = append(argv, "-fuse-ld=gold")

        // If gold is not installed, gcc will silently switch
        // back to ld.bfd. So we parse the version information
        // and provide a useful error if gold is missing.
        cmd := exec.Command(*flagExtld, "-fuse-ld=gold", "-Wl,--version")
        if out, err := cmd.CombinedOutput(); err == nil {
            if !bytes.Contains(out, []byte("GNU gold")) {
                log.Fatalf("ARM external linker must be gold (issue #15696), but is not: %s", out)
            }
        }
    }

https://github.com/golang/go/blob/release-branch.go1.14/src/cmd/link/internal/ld/lib.go#L1305


Notably, Android was removed in late 2019 from the platforms where "-fuse-ld=gold" is automatically applied:

https://github.com/golang/go/commit/d3c2b1f17600fadeebf62d65c85baf3bef879e2b
"The NDK is switching to ldd, and will stop including the gold linker."



There is active development on the internal Go linker, which may influence what degree of accommodations should be made for gold:

https://golang.org/doc/go1.15#linker  (go1.15 was released 2020-08-11)

"... This release includes substantial improvements to the Go linker, which reduce linker resource usage (both time and memory) and improve code robustness/maintainability.

For a representative set of large Go programs, linking is 20% faster and requires 30% less memory on average, for ELF-based OSes (Linux, FreeBSD, NetBSD, OpenBSD, Dragonfly, and Solaris) running on amd64 architectures, with more modest improvements for other architecture/OS combinations.

The key contributors to better linker performance are a newly redesigned object file format, and a revamping of internal phases to increase concurrency (for example, applying relocations to symbols in parallel). Object files in Go 1.15 are slightly larger than their 1.14 equivalents.

These changes are part of a multi-release project to modernize the Go linker, meaning that there will be additional linker improvements expected in future releases.

The linker now defaults to internal linking mode for -buildmode=pie on linux/amd64 and linux/arm64, so these configurations no longer require a C linker. External linking mode (which was the default in Go 1.14 for -buildmode=pie) can still be requested with -ldflags=-linkmode=external flag.



Building a better Go linker (2019)
https://golang.org/s/better-linker
Comment 16 Jeff Kowalczyk 2020-08-13 16:42:10 UTC
I have added a comment to the open Go upstream issue tracking the dependency on gold: https://github.com/golang/go/issues/22040#issuecomment-673580134
asking for feedback on whether the proposed binutils patch proposed in https://sourceware.org/bugzilla/show_bug.cgi?id=19962 would address the issues with ld on arm and aarch64:

--- a/bfd/elf32-arm.c	
+++ a/bfd/elf32-arm.c	
@@ -14112,11 +14112,15 @@ elf32_arm_adjust_dynamic_symbol (struct bfd_link_info * info,
   s = bfd_get_linker_section (dynobj, ".dynbss");
   BFD_ASSERT (s != NULL);
 
-  /* We must generate a R_ARM_COPY reloc to tell the dynamic linker to
-     copy the initial value out of the dynamic object and into the
-     runtime process image.  We need to remember the offset into the
+  /* If allowed, we must generate a R_ARM_COPY reloc to tell the dynamic
+     linker to copy the initial value out of the dynamic object and into
+     the runtime process image.  We need to remember the offset into the
      .rel(a).bss section we are going to use.  */
-  if ((h->root.u.def.section->flags & SEC_ALLOC) != 0 && h->size != 0)
+  if (info->nocopyreloc == 0
+      && (h->root.u.def.section->flags & SEC_ALLOC) != 0
+      /* PR 16177: A copy is only needed if the input section is readonly.  */
+      && (h->root.u.def.section->flags & SEC_READONLY) == 0
+      && h->size != 0)
     {
       asection *srel;
Comment 17 Jeff Kowalczyk 2020-08-14 03:46:00 UTC
go upstream replies that no, the proposed patch is only for 32-bit arm (I should have noticed that before asking) and would likely not resolve go's issue with ld on aarch64.

Leaving a note for myself on how to track changes in binutils upstream:the proposed patch was added to binutils 2016-04-19 soon after the bug report:

https://sourceware.org/git/?p=binutils-gdb.git;a=blobdiff;f=bfd/elf32-arm.c;h=ba89aa6c2621480e36ed219a845dee822b926fb5;hp=6e27155d5d30fdb5127c75448c722c7b0a03f4bb;hb=5522f910cb5;hpb=2deb93c7a7708ed164e4b10afb9d6a885d4615c6

That commit is present in:

binutils-gdb $ git branch --contains 5522f910cb539905d6adfdceab208ddfa5e84557
  binutils-2_32-branch
* master

I will also do some checking to better understand and document how the conditional check in Go code:

   if ctxt.Arch.InFamily(sys.ARM, sys.ARM64)

maps to our armv7l and aarch64 at runtime.
Comment 18 Richard Biener 2020-08-24 11:05:33 UTC
The upstream binutils bugreport was for ARM only, not for AARCH64.  I've amended
the bugreport.
Comment 19 Marcus Meissner 2020-09-04 14:52:51 UTC
I will remove the buildrequires from go1.14 now on SLE (wrap into a openSUSE conditional).
Comment 20 Guillaume GARDET 2020-09-04 15:00:55 UTC
(In reply to Marcus Meissner from comment #19)
> I will remove the buildrequires from go1.14 now on SLE (wrap into a openSUSE
> conditional).

That's not a good idea to diverge SLE and openSUSE.
Comment 21 Marcus Meissner 2020-09-04 15:26:04 UTC
actual change I did:

%ifarch %arm aarch64
%if 0%{?is_opensuse}
Requires:       binutils-gold
%else
Recommends:     binutils-gold
%endif
%endif


I am facing the issue that we do not want to ship binutils-gold to our customers in SLE currently, but I do need to shiop this go1.14 security update.

I am open to ideas :/
Comment 22 Guillaume GARDET 2020-09-04 15:28:24 UTC
Ok, that's a minor diff. With the Jump project, this diff will be effective on Tumbleweed only (for the openSUSE part).
Comment 23 Marcus Meissner 2020-09-04 15:30:59 UTC
I am hoping that either GO or binutils team can resolve this in their regular packages at some point.
Comment 24 Jeff Kowalczyk 2020-09-04 15:45:13 UTC
Based on the Go source excerpt from https://bugzilla.opensuse.org/show_bug.cgi?id=1170826#c15 , I do not think any go build command will currently run on ARM if the gold linker is not available.
Comment 25 Swamp Workflow Management 2020-09-07 19:13:34 UTC
SUSE-SU-2020:2562-1: An update that solves three vulnerabilities and has four fixes is now available.

Category: security (important)
Bug References: 1164903,1169832,1170826,1172868,1174153,1174191,1174977
CVE References: CVE-2020-14039,CVE-2020-15586,CVE-2020-16845
JIRA References: 
Sources used:
SUSE Linux Enterprise Module for Development Tools 15-SP2 (src):    go1.14-1.14.7-1.15.1
SUSE Linux Enterprise Module for Development Tools 15-SP1 (src):    go1.14-1.14.7-1.15.1

NOTE: This line indicates an update has been released for the listed product(s). At times this might be only a partial fix. If you have questions please reach out to maintenance coordination.
Comment 26 Swamp Workflow Management 2020-09-10 22:15:39 UTC
openSUSE-SU-2020:1405-1: An update that solves three vulnerabilities and has four fixes is now available.

Category: security (important)
Bug References: 1164903,1169832,1170826,1172868,1174153,1174191,1174977
CVE References: CVE-2020-14039,CVE-2020-15586,CVE-2020-16845
JIRA References: 
Sources used:
openSUSE Leap 15.1 (src):    go1.14-1.14.7-lp151.13.1
Comment 27 Swamp Workflow Management 2020-09-11 10:15:41 UTC
openSUSE-SU-2020:1407-1: An update that solves three vulnerabilities and has four fixes is now available.

Category: security (important)
Bug References: 1164903,1169832,1170826,1172868,1174153,1174191,1174977
CVE References: CVE-2020-14039,CVE-2020-15586,CVE-2020-16845
JIRA References: 
Sources used:
openSUSE Leap 15.2 (src):    go1.14-1.14.7-lp152.2.3.1
Comment 28 OBSbugzilla Bot 2020-09-11 17:30:06 UTC
This is an autogenerated message for OBS integration:
This bug (1170826) was mentioned in
https://build.opensuse.org/request/show/833837 Factory / go1.14
https://build.opensuse.org/request/show/833838 Factory / go1.15
Comment 31 OBSbugzilla Bot 2020-09-14 07:40:07 UTC
This is an autogenerated message for OBS integration:
This bug (1170826) was mentioned in
https://build.opensuse.org/request/show/834214 Factory / go1.13
Comment 34 Swamp Workflow Management 2020-09-20 22:14:31 UTC
openSUSE-RU-2020:1489-1: An update that has one recommended fix can now be installed.

Category: recommended (moderate)
Bug References: 1170826
CVE References: 
JIRA References: 
Sources used:
openSUSE Leap 15.1 (src):    go1.13-1.13.15-lp151.11.1
Comment 35 Swamp Workflow Management 2020-09-21 10:19:33 UTC
openSUSE-RU-2020:1492-1: An update that has one recommended fix can now be installed.

Category: recommended (moderate)
Bug References: 1170826
CVE References: 
JIRA References: 
Sources used:
openSUSE Leap 15.2 (src):    go1.13-1.13.15-lp152.2.10.1
Comment 36 Swamp Workflow Management 2020-09-29 13:14:34 UTC
SUSE-SU-2020:2776-1: An update that solves one vulnerability, contains one feature and has two fixes is now available.

Category: security (moderate)
Bug References: 1170826,1175132,1176031
CVE References: CVE-2020-24553
JIRA References: ECO-1484
Sources used:
SUSE Linux Enterprise Module for Development Tools 15-SP2 (src):    go1.15-1.15.2-1.3.1
SUSE Linux Enterprise Module for Development Tools 15-SP1 (src):    go1.15-1.15.2-1.3.1

NOTE: This line indicates an update has been released for the listed product(s). At times this might be only a partial fix. If you have questions please reach out to maintenance coordination.