Bug 1112690 - gcc8: ASAN backtrace contain only hexadecimal offsets
gcc8: ASAN backtrace contain only hexadecimal offsets
Status: RESOLVED FIXED
Classification: openSUSE
Product: openSUSE Tumbleweed
Classification: openSUSE
Component: Development
Current
Other Other
: P5 - None : Normal (vote)
: ---
Assigned To: Tom de Vries
E-mail List
:
Depends on:
Blocks:
  Show dependency treegraph
 
Reported: 2018-10-22 08:43 UTC by Petr Gajdos
Modified: 2018-10-31 11:20 UTC (History)
2 users (show)

See Also:
Found By: ---
Services Priority:
Business Priority:
Blocker: ---
Marketing QA Status: ---
IT Deployment: ---


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Petr Gajdos 2018-10-22 08:43:32 UTC
$ ASAN_OPTIONS=abort_on_error=1 gdb --args tiff2pdf POC -o out
GNU gdb (GDB; openSUSE Tumbleweed) 8.2
[..]
(gdb) run
[..]
=================================================================
==26212==ERROR: AddressSanitizer: heap-use-after-free on address 0x603000000c70 at pc 0x7ffff70a9c40 bp 0x7fffffffe6f0 sp 0x7fffffffdea0
READ of size 32 at 0x603000000c70 thread T0
    #0 0x7ffff70a9c3f  (/usr/lib64/libasan.so.5+0x9ac3f)
    #1 0x55555555d8b4  (/usr/bin/tiff2pdf+0x98b4)
    #2 0x5555555781b0  (/usr/bin/tiff2pdf+0x241b0)
    #3 0x55555555c793  (/usr/bin/tiff2pdf+0x8793)
    #4 0x7ffff6c71fea in __libc_start_main (/lib64/libc.so.6+0x22fea)
    #5 0x55555555d459  (/usr/bin/tiff2pdf+0x9459)

0x603000000c70 is located 0 bytes inside of 32-byte region [0x603000000c70,0x603000000c90)
freed by thread T0 here:
    #0 0x7ffff70fab50 in __interceptor_free (/usr/lib64/libasan.so.5+0xebb50)
    #1 0x7ffff7eecb2e in TIFFFreeDirectory /usr/src/debug/tiff-4.0.9-0.x86_64/libtiff/tif_dir.c:1266

previously allocated by thread T0 here:
    #0 0x7ffff70faed0 in malloc (/usr/lib64/libasan.so.5+0xebed0)
    #1 0x7ffff7edfe9f in setByteArray /usr/src/debug/tiff-4.0.9-0.x86_64/libtiff/tif_dir.c:54

SUMMARY: AddressSanitizer: heap-use-after-free (/usr/lib64/libasan.so.5+0x9ac3f) 
Shadow bytes around the buggy address:
  0x0c067fff8130: fd fd fa fa fd fd fd fd fa fa fd fd fd fd fa fa
  0x0c067fff8140: fd fd fd fd fa fa fd fd fd fd fa fa fd fd fd fd
  0x0c067fff8150: fa fa fd fd fd fd fa fa fd fd fd fd fa fa fd fd
  0x0c067fff8160: fd fd fa fa fd fd fd fd fa fa fd fd fd fd fa fa
  0x0c067fff8170: fd fd fd fd fa fa fd fd fd fd fa fa fd fd fd fd
=>0x0c067fff8180: fa fa fd fd fd fa fa fa fd fd fd fd fa fa[fd]fd
  0x0c067fff8190: fd fd fa fa fd fd fd fd fa fa 00 00 00 00 fa fa
  0x0c067fff81a0: 00 00 00 00 fa fa 00 00 00 00 fa fa 00 00 00 00
  0x0c067fff81b0: fa fa 00 00 00 00 fa fa fd fd fd fa fa fa 00 00
  0x0c067fff81c0: 00 00 fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x0c067fff81d0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
Shadow byte legend (one shadow byte represents 8 application bytes):
  Addressable:           00
  Partially addressable: 01 02 03 04 05 06 07 
  Heap left redzone:       fa
  Freed heap region:       fd
  Stack left redzone:      f1
  Stack mid redzone:       f2
  Stack right redzone:     f3
  Stack after return:      f5
  Stack use after scope:   f8
  Global redzone:          f9
  Global init order:       f6
  Poisoned by user:        f7
  Container overflow:      fc
  Array cookie:            ac
  Intra object redzone:    bb
  ASan internal:           fe
  Left alloca redzone:     ca
  Right alloca redzone:    cb
==26212==ABORTING

Program received signal SIGABRT, Aborted.
0x00007ffff6c8708b in raise () from /lib64/libc.so.6
(gdb) bt
#0  0x00007ffff6c8708b in raise () from /lib64/libc.so.6
#1  0x00007ffff6c704e9 in abort () from /lib64/libc.so.6
#2  0x00007ffff711835b in ?? () from /usr/lib64/libasan.so.5
#3  0x00007ffff71203e8 in ?? () from /usr/lib64/libasan.so.5
#4  0x00007ffff710555d in ?? () from /usr/lib64/libasan.so.5
#5  0x00007ffff70a9c5f in ?? () from /usr/lib64/libasan.so.5
#6  0x000055555555d8b5 in t2p_writeproc (handle=0x61f000000080, data=<optimized out>, size=<optimized out>) at tiff2pdf.c:405
#7  0x00005555555781b1 in t2p_write_pdf_stream (output=0x619000000580, len=<optimized out>, buffer=<optimized out>) at tiff2pdf.c:3998
#8  t2p_write_pdf_transfer_stream (i=1, output=0x619000000580, t2p=0x61f000000080) at tiff2pdf.c:5026
#9  t2p_write_pdf (t2p=0x61f000000080, input=0x619000000080, output=0x619000000580) at tiff2pdf.c:5506
#10 0x000055555555c794 in main (argc=<optimized out>, argv=<optimized out>) at tiff2pdf.c:808
(gdb)

This is on TW changeroot (after tiff build). gcc version is 8-2.1.

You see I get expected debug info in gdb backtrace, but not in ASAN report. It is not a big issue for me at it is possible to get the backtrace with GDB when ASAN_OPTIONS=abort_on_error=1 as above. Just that the ASAN report outside gdb is not much useful.
Comment 1 Richard Biener 2018-10-22 09:09:15 UTC
Tom, can you see what it takes to teach libbacktrace our separate debuginfo scheme or what is missing here?
Comment 2 Tom de Vries 2018-10-23 09:29:40 UTC
(In reply to Richard Biener from comment #1)
> Tom, can you see what it takes to teach libbacktrace our separate debuginfo
> scheme or what is missing here?

I haven't reproduced this yet, but while investigating I came across PR86198 - "Libbacktrace does not properly work with .note.gnu.build-id section" ( https://gcc.gnu.org/bugzilla/show_bug.cgi?id=86198 ), fixed on master but not on any release branch, and not in any patch included in the gcc-8 package on tumbleweed.

This may be a duplicate.
Comment 3 Richard Biener 2018-10-23 09:41:48 UTC
(In reply to Tom de Vries from comment #2)
> (In reply to Richard Biener from comment #1)
> > Tom, can you see what it takes to teach libbacktrace our separate debuginfo
> > scheme or what is missing here?
> 
> I haven't reproduced this yet, but while investigating I came across PR86198
> - "Libbacktrace does not properly work with .note.gnu.build-id section" (
> https://gcc.gnu.org/bugzilla/show_bug.cgi?id=86198 ), fixed on master but
> not on any release branch, and not in any patch included in the gcc-8
> package on tumbleweed.
> 
> This may be a duplicate.

Ah, thanks.  I'll backport the patch and will update gcc8 for Tumbleweed.
I'm not 100% sure it will fix things though.  Let's see...
Comment 4 Petr Gajdos 2018-10-23 10:03:42 UTC
Feel free to ask me for testing.
Comment 5 Richard Biener 2018-10-23 10:24:39 UTC
(In reply to Petr Gajdos from comment #4)
> Feel free to ask me for testing.

Updated gcc8 is now building in devel:gcc
Comment 6 Tom de Vries 2018-10-23 12:20:19 UTC
(In reply to Tom de Vries from comment #2)
> I haven't reproduced this yet

I managed to reproduce this on tumbleweed changeroot using atexit. A complication here is that libc isn't stripped, so the tests script makes a local stripped version of libc to force libasan's libbacktrace to look for the buildid debuginfo for libc.

Test source:
...
$ cat atexit.c
#include <stdlib.h>

int a;

void
foo (void)
{
  int *ptr = new int[1];
  a = ptr[1];
}

int
main (void)
{
  atexit (foo);
  return 0;
}
...

Test script:
...
$ cat atexit.sh
#!/bin/sh

g++-8 \
    -fsanitize=address \
    atexit.c

# a.out picks up debug info from unstripped libc, so we find
# __run_exit_handlers in the log
./a.out \
    > LOG \
    2>&1

bash -xc "grep -c __run_exit_handlers LOG"

# Create local libc, and strip it
cp /lib64/libc.so.6 .
strip \
    --strip-debug \
    --strip-unneeded \
      libc.so.6

# Use local libc
export LD_LIBRARY_PATH=$(pwd -L)

# Show which libc is picked up, to verify it's the local one
ldd ./a.out \
    2>&1 \
    | grep libc.so.6

# Debug info is missing for __run_exit_handlers
./a.out \
    > LOG.2 \
    2>&1
bash -xc "grep -c __run_exit_handlers LOG.2"

# Reproduce using gdb: __run_exit_handlers is shown in gdb backtrace, which
# shows that the buildid debug info is available, but it's missing in the asan
# backtrace.
export ASAN_OPTIONS=abort_on_error=1
gdb \
    -batch \
    a.out \
    -ex r \
    -ex bt \
    > LOG.3 \
    2>&1
bash -xc "grep -c __run_exit_handlers LOG.3"
...

Test output:
...
$ ./atexit.sh
+ grep -c __run_exit_handlers LOG
2
        libc.so.6 => /leap-home/vries/atexit/libc.so.6 (0x00007ff3f12cd000)
+ grep -c __run_exit_handlers LOG.2
0
+ grep -c __run_exit_handlers LOG.3
1
...
Comment 7 Tom de Vries 2018-10-23 13:52:18 UTC
(In reply to Richard Biener from comment #5)
> Updated gcc8 is now building in devel:gcc

With https://build.opensuse.org/package/binary/download/devel:gcc/gcc8/openSUSE_Factory/x86_64/libasan5-8.2.1+r265419-47.1.x86_64.rpm installed, I get:
...
+ grep -c __run_exit_handlers LOG
2
        libc.so.6 => /leap-home/vries/tiff/atexit/libc.so.6 (0x00007f393673a000)
+ grep -c __run_exit_handlers LOG.2
2
+ grep -c __run_exit_handlers LOG.3
3
...

So, this is fixed.
Comment 8 Petr Gajdos 2018-10-23 15:39:37 UTC
(In reply to Tom de Vries from comment #7)
> So, this is fixed.

Yes. Even with my testcase:

$ tiff2pdf POC -o out
[..]
=================================================================
==13684==ERROR: AddressSanitizer: heap-use-after-free on address 0x603000000c70 at pc 0x7f8e8745cc40 bp 0x7ffe75e93230 sp 0x7ffe75e929e0
READ of size 32 at 0x603000000c70 thread T0
    #0 0x7f8e8745cc3f  (/usr/lib64/libasan.so.5+0x9ac3f)
    #1 0x5600523a28b4 in t2p_writeproc /usr/src/debug/tiff-4.0.9-0.x86_64/tools/tiff2pdf.c:405
    #2 0x5600523bd1b0 in t2p_write_pdf_stream /usr/src/debug/tiff-4.0.9-0.x86_64/tools/tiff2pdf.c:3998
    #3 0x5600523bd1b0 in t2p_write_pdf_transfer_stream /usr/src/debug/tiff-4.0.9-0.x86_64/tools/tiff2pdf.c:5026
    #4 0x5600523bd1b0 in t2p_write_pdf /usr/src/debug/tiff-4.0.9-0.x86_64/tools/tiff2pdf.c:5506
    #5 0x5600523a1793 in main /usr/src/debug/tiff-4.0.9-0.x86_64/tools/tiff2pdf.c:808
    #6 0x7f8e87024fea in __libc_start_main (/lib64/libc.so.6+0x22fea)
    #7 0x5600523a2459  (/usr/bin/tiff2pdf+0x9459)
[..]
$

Thanks!
Comment 9 Swamp Workflow Management 2018-10-24 13:30:10 UTC
This is an autogenerated message for OBS integration:
This bug (1112690) was mentioned in
https://build.opensuse.org/request/show/644286 Factory / gcc8
Comment 10 Swamp Workflow Management 2018-10-25 14:40:06 UTC
This is an autogenerated message for OBS integration:
This bug (1112690) was mentioned in
https://build.opensuse.org/request/show/644670 Factory / gcc8
Comment 11 Swamp Workflow Management 2018-10-25 15:21:23 UTC
This is an autogenerated message for OBS integration:
This bug (1112690) was mentioned in
https://build.opensuse.org/request/show/644675 Factory / gcc8
Comment 12 Swamp Workflow Management 2018-10-31 11:20:11 UTC
This is an autogenerated message for OBS integration:
This bug (1112690) was mentioned in
https://build.opensuse.org/request/show/645704 Factory / gcc8