Project

General

Profile

Actions

Bug #18061

open

Execshield test: libruby.so.N.N.N: FAIL: property-note test because no .note.gnu.property section found

Added by jaruga (Jun Aruga) 10 months ago. Updated about 7 hours ago.

Status:
Open
Priority:
Normal
Assignee:
-
Target version:
-
[ruby-core:104774]

Description

I found an issue in our company's internal test called "execshield" by a security tool annobin - annocheck command [1][2].

Hardened: libruby.so.2.7.4: FAIL: property-note test because no .note.gnu.property section found

Here is the reproducer on the upstream latest master, commit is 5f2987d6c2ae9ace3178ac3e1bbb4ac7079101eb,

$ autoconf
$ ./configure --enable-shared
$ make

$ ls libruby.so.3.1.0
libruby.so.3.1.0*

If you are using Red Hat based Linux distro, it's easy to install by the RPM package like this.

$ sudo dnf -y install annobin-annocheck
$ sudo yum -y install annobin-annocheck

Then

$ annocheck libruby.so.3.1.0

If you are using other Linux distros such as Ubuntu, you can use it by a container I prepared.

Prepare the following Dockerfile.

$ cat Dockerfile 
FROM docker.io/fedora:34

RUN cat /etc/fedora-release
RUN dnf -y install annobin-annocheck
WORKDIR /work

Then build the container image with the Dockerfile and run the annocheck command for the libruby.so.3.1.0 on your host environment. The -v is an option for bind mount between host and container environment.

$ docker build --rm -t fedora-annocheck .

$ docker run --rm -t -v $(pwd):/work fedora-annocheck annocheck /work/libruby.so.3.1.0
annocheck: Version 9.79.
Hardened: libruby.so.3.1.0: FAIL: bind-now test because not linked with -Wl,-z,now 
Hardened: libruby.so.3.1.0: FAIL: notes test because gaps were detected in the annobin coverage 
Hardened: libruby.so.3.1.0: FAIL: cf-protection test because no .note.gnu.property section = no control flow information 
Hardened: libruby.so.3.1.0: FAIL: property-note test because no .note.gnu.property section found 
Hardened: Rerun annocheck with --verbose to see more information on the tests.

The message Hardened: libruby.so.3.1.0: FAIL: property-note test because no .note.gnu.property section found is what I found in our internal test. For other FAIL messages, maybe it can be fixed by changing how to build.

Asking a colleague, I was told that the coroutine/*/Context.S files such as coroutine/x86/Context.S cause the failure. Do you have any idea how to fix this? Thanks.


Files

0001-Add-.note.gnu.property-sections.patch (2.64 KB) 0001-Add-.note.gnu.property-sections.patch nobu (Nobuyoshi Nakada), 08/04/2021 04:20 PM
0001-Add-.note.gnu.property-sections.patch (3.69 KB) 0001-Add-.note.gnu.property-sections.patch nobu (Nobuyoshi Nakada), 08/06/2021 12:26 PM
config-pie.log (11.4 KB) config-pie.log jaruga (Jun Aruga), 05/13/2022 08:11 PM

Updated by jaruga (Jun Aruga) 10 months ago

Thanks for the patch! Let me check it.

Updated by jaruga (Jun Aruga) 10 months ago

I applied the patch for the Ruby 3.0.2 in RHEL 8, built. But I still see the failed message.

annocheck: Version 9.65.
Hardened: libruby.so.3.0.2: FAIL: property-note test because no .note.gnu.property section found
Hardened: Rerun annocheck with --verbose to see more information on the tests.

I also applied the patch to the Ruby on master branch 1cb5a669d39fe741388be7ac45fc88c336b097cf . But the message is same.

$ autoconf
$ ./configure --enable-shared
$ make

$ annocheck libruby.so.3.1.0
annocheck: Version 9.79.
Hardened: libruby.so.3.1.0: FAIL: bind-now test because not linked with -Wl,-z,now 
Hardened: libruby.so.3.1.0: FAIL: notes test because gaps were detected in the annobin coverage 
Hardened: libruby.so.3.1.0: FAIL: cf-protection test because no .note.gnu.property section = no control flow information 
Hardened: libruby.so.3.1.0: FAIL: property-note test because no .note.gnu.property section found 
Hardened: Rerun annocheck with --verbose to see more information on the tests.

So, I don't think that adding the empty section works.

Updated by jaruga (Jun Aruga) 10 months ago

By the way, I found a case generating ".note.gnu.property section" in the OpenSSL project.
https://github.com/openssl/openssl/commit/51994e505dbb1cd0dd76869ec962e2948b77b585

The commit message explains how they verified the change. So in the case of Ruby, here is a way to verify?

I also applied the patch to the Ruby on master branch 1cb5a669d39fe741388be7ac45fc88c336b097cf .

On the above commit, I see an error because there is no .note.gnu.property non-empty section?

$ autoconf

$ CC="gcc -Wl,-z,cet-report=error -fcf-protection" ./configure --enable-shared

$ make
...
/bin/ld: coroutine/amd64/Context.o: error: missing IBT and SHSTK properties
/bin/ld: probes.o: error: missing IBT and SHSTK properties
collect2: error: ld returned 1 exit status
make: *** [Makefile:277: miniruby] Error 1

$ echo $?
2

Updated by jaruga (Jun Aruga) 10 months ago

It seems we can check if the .note.gnu.property exists in a binary file by the command readelf -S --wide <file>. The binary file ruby has it. But the file libruby.so.3.1.0 doesn't it.

$ readelf -S --wide ruby
There are 40 section headers, starting at offset 0x36038:

Section Headers:
  [Nr] Name              Type            Address          Off    Size   ES Flg Lk Inf Al
  [ 0]                   NULL            0000000000000000 000000 000000 00      0   0  0
  [ 1] .interp           PROGBITS        0000000000400318 000318 00001c 00   A  0   0  1
  [ 2] .note.gnu.property NOTE            0000000000400338 000338 000020 00   A  0   0  8
...
$ readelf -S --wide libruby.so.3.1.0
There are 42 section headers, starting at offset 0xc15a38:

Section Headers:
  [Nr] Name              Type            Address          Off    Size   ES Flg Lk Inf Al
  [ 0]                   NULL            0000000000000000 000000 000000 00      0   0  0
  [ 1] .note.gnu.build-id NOTE            0000000000000270 000270 000024 00   A  0   0  4
  [ 2] .gnu.hash         GNU_HASH        0000000000000298 000298 0033e4 00   A  3   0  8
...

Updated by jaruga (Jun Aruga) 10 months ago

Thanks for the patch! Let me check it.

Updated by ioquatix (Samuel Williams) 9 months ago

On another side, we have an Intel CET [2], [3]:
"Control-flow Enforcement Technology (CET) provides the following capabilities to defend
against ROP/JOP style control-flow subversion attacks:
- Shadow Stack - return address protection to defend against Return Oriented Programming,
- Indirect branch tracking - free branch protection to defend against Jump/Call Oriented Programming.‚Äč"

Intel CET uses following processor-specific program property types in .note.gnu.property [3, p85, p87]:

* GNU_PROPERTY_X86_FEATURE_1_IBT This indicates that all executable sections are compatible
with IBT (see Section 13.1.1) when endbr64 instruction starts each valid target where an indirect
branch instruction can land. 8
* GNU_PROPERTY_X86_FEATURE_1_SHSTK This indicates that all executable sections are compatible
with SHSTK (see Section 13.1.2) where return address popped from shadow stack always matches
return address popped from normal stack.

We should confirm that coroutine stack swapping is compatible with GNU_PROPERTY_X86_FEATURE_1_SHSTK.

Updated by jaruga (Jun Aruga) 9 months ago

Thanks for the patch! Let me check it.

0001-Add-.note.gnu.property-sections.patch (3.69 KB)

I tried the 2nd patch on an older master commit 1cb5a669d39fe741388be7ac45fc88c336b097cf , as I couldn't build on the latest master commit due to #18072 . But I still see similar or same result.

$ gcc --version
gcc (GCC) 11.2.1 20210728 (Red Hat 11.2.1-1)
Copyright (C) 2021 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

With -Wl,-z,cet-report=error

$ autoconf
$ CC="gcc -Wl,-z,cet-report=error -fcf-protection" ./configure --enable-shared
$ make
...
vm_call_iseq_optimized.inc updated
generating vmtc.inc
compiling vm.c
compiling vm_backtrace.c
compiling vm_dump.c
compiling vm_sync.c
making trace_point.rbinc
compiling vm_trace.c
assembling coroutine/amd64/Context.S
processing probes in object files
compiling enc/ascii.c
compiling enc/us_ascii.c
compiling enc/unicode.c
compiling enc/utf_8.c
generating enc/trans/newline.c ...
compiling enc/trans/newline.c
compiling ./missing/setproctitle.c
compiling ./missing/strlcat.c
compiling ./missing/strlcpy.c
compiling addr2line.c
compiling dmyenc.c
linking miniruby
/bin/ld: probes.o: error: missing IBT and SHSTK properties
collect2: error: ld returned 1 exit status
make: *** [Makefile:277: miniruby] Error 1

Check by annocheck and readelf.

$ autoconf
$ ./configure --enable-shared
$ make
$ annocheck libruby.so.3.1.0
annocheck: Version 9.79.
Hardened: libruby.so.3.1.0: FAIL: bind-now test because not linked with -Wl,-z,now 
Hardened: libruby.so.3.1.0: FAIL: notes test because gaps were detected in the annobin coverage 
Hardened: libruby.so.3.1.0: FAIL: cf-protection test because no .note.gnu.property section = no control flow information 
Hardened: libruby.so.3.1.0: FAIL: property-note test because no .note.gnu.property section found 
Hardened: Rerun annocheck with --verbose to see more information on the tests.

$ readelf -S --wide libruby.so.3.1.0
  => No .note.gnu.property section.
Actions #10

Updated by jaruga (Jun Aruga) 9 months ago

  • Backport changed from 2.6: UNKNOWN, 2.7: UNKNOWN, 3.0: UNKNOWN to 2.6: UNKNOWN, 2.7: REQUIRED, 3.0: REQUIRED

Updated by ioquatix (Samuel Williams) 10 days ago

I investigated this issue today.

It doesn't appear to just be a matter of adding a few properties, this actually involves correctly implementing an intel-specific shadow stack.

Assembly coroutine backend and x86 CET support (in QEMU): https://lists.sr.ht/~philmd/qemu/patches/4691

My understanding is we'd need to implement the shadow stack handling in the coroutine code. Or maybe we can just specify that it's not supported and that's also okay?

I think in the future, it's more likely that C compilers will provide native coroutine functions - in this situation maybe we can just use those instead of our native implementations and this problem will go away.

(1) Can we fix this issue without introducing shadow stacks?
(2) Is that sufficient to pass the tests above?

By the way, the error shows an issue with probes.o and while I see the issue also exists for the coroutine implementation, the fix provided by @nobu (Nobuyoshi Nakada) at least hides the error - whether the actual shadow stack is working is anyone's guess - actual programs may just crash on coroutine swap.

See this branch with Nobu's patch: https://github.com/ruby/ruby/pull/5895

From https://www.phoronix.com/scan.php?page=news_item&px=Intel-CET-IBT-SHSTK-glibc:

Landing yesterday in Glibc for Intel's Control-flow Enforcement Technology (CET) were the instructions for Indirect Branch Tracking (IBT) and Shadow Stack (SHSTK).

These Intel CET bits for the GNU C Library amount to a fair amount of code being added. The commit message explains some of the CET steps taken. The Control-flow Enforcement Technology behavior can be changed for SHSTK/IBT at run-time through the "GLIBC_TUNABLES" environment variable.

CET aims to fend off return-oriented programming (ROP) and call-jump-oriented programming (COP/JOP) attacks via a shadow stack to keep track of expected return addresses and to raise faults when the return addresses don't match up with what's found in the shadow stack. The Indirect branch tracking is for the stopping of jump/call oriented attacks. CET has yet to appear in released Intel processors while this security feature has already been squared away in the GCC compiler and other areas of the software stack and is now getting setup within Glibc.

which links to https://sourceware.org/git/?p=glibc.git;a=commit;h=f753fa7dea3367bd3eb7b543103ff8cda182a3fa which has a lot more details.

Updated by jaruga (Jun Aruga) 8 days ago

@ioquatix (Samuel Williams) Thank you for your investigation! I prepared make test-annocheck command for your convenient, and for us to test this case on CI. You can pick up my commit from https://github.com/ruby/ruby/pull/5900 .

Updated by jaruga (Jun Aruga) 5 days ago

I am testing Samuel's PR: https://github.com/ruby/ruby/pull/5895.

To pass the security tool, annocheck with the PR, I am trying to build with minimal compiler flags.

I am using the following gcc on my local.

$ gcc --version
gcc (GCC) 11.2.1 20220127 (Red Hat 11.2.1-9)
Copyright (C) 2021 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

The following command works. The number of failures by annobin is reduced from 5 to 2 by the flags.

$ ./configure --enable-shared --with-gcc="gcc -fcf-protection -Wl,-z,now"

But the following command doesn't work with -Wl,-pie.

$ ./configure --enable-shared --with-gcc="gcc -fcf-protection -Wl,-z,now -Wl,-pie"
checking for ruby... /bin/ruby
tool/config.guess already exists
tool/config.sub already exists
checking build system type... x86_64-pc-linux-gnu
checking host system type... x86_64-pc-linux-gnu
checking target system type... x86_64-pc-linux-gnu
checking for ld... ld
checking for gcc-ar... gcc-ar
checking for g++... g++
checking for gcc-nm... gcc-nm
checking for gcc-ranlib... gcc-ranlib
checking for gcc... gcc -fcf-protection -Wl,-z,now -Wl,-pie
checking whether the C compiler works... no
configure: error: in `/home/jaruga/git/ruby/ruby':
configure: error: C compiler cannot create executables
See `config.log' for more details

I would upload the config.log (config-pie.log). Do you know what's wrong?

Updated by jaruga (Jun Aruga) 3 days ago

@ioquatix (Samuel Williams) My PR above was merged. Now after you rebase your PR on the latest master branch, you can test your PR with make test-annocheck on your local or test with CI with the following modification. Thanks.

$ git diff
diff --git a/.github/workflows/compilers.yml b/.github/workflows/compilers.yml
index ddb53cadb8..8ae6e40b9e 100644
--- a/.github/workflows/compilers.yml
+++ b/.github/workflows/compilers.yml
@@ -87,7 +87,7 @@ jobs:
               # https://bugs.ruby-lang.org/issues/18061
               # https://sourceware.org/annobin/annobin.html/Test-pie.html
               # https://sourceware.org/annobin/annobin.html/Test-notes.html
-              TEST_ANNOCHECK_OPTS: "--skip-pie --skip-notes"
+              TEST_ANNOCHECK_OPTS: "--skip-pie"
             check: true
           - { key: default_cc, name: clang-15,  value: clang-15,  container: clang-15 }
           - { key: default_cc, name: clang-14,  value: clang-14,  container: clang-14 }

Updated by jaruga (Jun Aruga) 2 days ago

@nobu (Nobuyoshi Nakada) what is your intent for this commit, https://github.com/ruby/ruby/commit/a7577dbfd3ea53cccf7aaf94208069784ad17791 ? What issue did you see?

Before the commit, you see the annocheck was executed in make test-annocheck using Fedora container in Ubuntu focal container on gcc-11 annocheck case on Ubuntu host in .github/workflows/compilers.yml. The log is here. But after the commit, the annocheck is not executed any more in the make test-annocheck in the gcc-11 annocheck case on the CI any more. The log is here.

Updated by jaruga (Jun Aruga) 2 days ago

On the current latest master f2dc97294086c31604a45ee350c4fa5d51762f45, here is the result on my Fedora 35. The Fedora container and annocheck are not executed.

$ git clean -fdx
$ ./autogen.sh
$ ./configure --enable-shared --with-gcc="gcc -fcf-protection -Wl,-z,now"
$ make
$ TEST_ANNOCHECK_OPTS="--skip-pie --skip-notes" make test-annocheck
make: Nothing to be done for 'test-annocheck'.

Here is the test on the previous commit of the commit a7577dbfd3ea53cccf7aaf94208069784ad17791 above. The Fedora container and annocheck can be executed multiple times after make.

$ git clean -fdx
$ git checkout 641c3830df8177b38fcfca33d3a0ece8a2c1e7d2
$ ./autogen.sh
$ ./configure --enable-shared --with-gcc="gcc -fcf-protection -Wl,-z,now"
$ make

$ TEST_ANNOCHECK_OPTS="--skip-pie --skip-notes" make test-annocheck
...
+ /bin/docker run --rm -t -v /home/jaruga/git/ruby/ruby:/work ruby-fedora-annocheck annocheck --verbose --skip-pie --skip-notes ruby
...
Hardened: ruby: Overall: PASS.

$ TEST_ANNOCHECK_OPTS="--skip-pie --skip-notes" make test-annocheck
...
+ /bin/docker run --rm -t -v /home/jaruga/git/ruby/ruby:/work ruby-fedora-annocheck annocheck --verbose --skip-pie --skip-notes ruby
...
Hardened: ruby: Overall: PASS.

Updated by nobu (Nobuyoshi Nakada) 1 day ago

jaruga (Jun Aruga) wrote in #note-15:

@nobu (Nobuyoshi Nakada) what is your intent for this commit, https://github.com/ruby/ruby/commit/a7577dbfd3ea53cccf7aaf94208069784ad17791 ? What issue did you see?

TEST_RUNNABLE is a flag for native/cross compiling.
Since annocheck works on Fedora Docker image, it is unrelated to the host OS, and it may work even when cross compiling if Docker works.

Updated by jaruga (Jun Aruga) 1 day ago

nobu (Nobuyoshi Nakada) wrote in #note-17:

jaruga (Jun Aruga) wrote in #note-15:

@nobu (Nobuyoshi Nakada) what is your intent for this commit, https://github.com/ruby/ruby/commit/a7577dbfd3ea53cccf7aaf94208069784ad17791 ? What issue did you see?

TEST_RUNNABLE is a flag for native/cross compiling.
Since annocheck works on Fedora Docker image, it is unrelated to the host OS, and it may work even when cross compiling if Docker works.

I see. Thanks for explaining it, and thanks for fixing the issue on the commit https://github.com/ruby/ruby/commit/2411f0ad8cf608d6aae221ce4d803b5c5271795a . I confirmed it works after the commit on the CI (the log is here) and my local Fedora. But as you said, it may work on Windows, Mac, *BSD too where Docker is installed. If it works there, maybe the current target linux-test-annocheck is not enough.

Updated by nobu (Nobuyoshi Nakada) 1 day ago

jaruga (Jun Aruga) wrote in #note-18:

But as you said, it may work on Windows, Mac, *BSD too where Docker is installed. If it works there, maybe the current target linux-test-annocheck is not enough.

Isn't Docker always Linux, regardless hosts?

Updated by jaruga (Jun Aruga) 1 day ago

Isn't Docker always Linux, regardless hosts?

Yes, Docker where annocheck is running in is always Linux (= docker.io/fedora:latest container). But the target_os (= "linux" on host os: Fedora and Ubuntu) in common.mk is maybe not "linux" on host: Windows, Mac and *BSD, right? I will try to test make annocheck on Mac.

https://github.com/ruby/ruby/blob/11af23ee923858fbcec18f2d1365296fdfe24dc2/common.mk#L1450-L1453

test-annocheck: $(target_os)-test-annocheck
linux-test-annocheck: $(PROGRAM)
	$(tooldir)/test-annocheck.sh $(PROGRAM)
$(target_os)-test-annocheck: PHONY

Updated by jaruga (Jun Aruga) about 7 hours ago

I will try to test make annocheck on Mac.

I sent the PR here. It's working in progress.
https://github.com/ruby/ruby/pull/5916

Actions

Also available in: Atom PDF