Project

General

Profile

Actions
Copy link

Bug #12454

closed

Regexp: Segfault due to Invalid Read in regcomp.c : add_char_opt_map_info

Added by grajagandev (David Moore) over 8 years ago. Updated over 8 years ago.

Status:
Closed
Assignee:
-
Target version:
-
ruby -v:
ruby 2.4.0dev (2016-06-02) [i686-linux]
Backport:
2.1: WONTFIX, 2.2: DONE, 2.3: DONE
[ruby-core:75828]

Description

A crafted regular expression will cause a segmentation fault due to an invalid one byte read on 32-bit Ubuntu 14.04. The interesting part of the regular expression is: '\g<0>' which is a recursive reference to the 0th sub-pattern.

grajagandev# cat load-re.rb 
File.open(ARGV[0]) do |f|  
  @re = Regexp.new("/" + File.read(f) + "/")
end  
grajagandev# xxd op_node_left 
0000000: 4928 3f3c 5418 5256 6400 3e3c 2a29 303c  I(?<T.RVd.><*)0<
0000010: 285c 673c 303e 2967 5c58 5c04 002b 2b24  (\g<0>)g\X\..++$
0000020: 2424 2424 2424 2424 2424 2324 2424 2424  $$$$$$$$$$#$$$$$
0000030: 2424 2424 5d7b 5c00 000a                 $$$$]{\...
grajagandev# uname -a
Linux x-Acer 3.19.0-25-generic #26~14.04.1-Ubuntu SMP Fri Jul 24 21:18:00 UTC 2015 i686 i686 i686 GNU/Linux
grajagandev# valgrind --max-stackframe=90000000 ruby load-re.rb op_node_left
==2293== Memcheck, a memory error detector
==2293== Copyright (C) 2002-2013, and GNU GPL'd, by Julian Seward et al.
==2293== Using Valgrind-3.10.1 and LibVEX; rerun with -h for copyright info
==2293== Command: ruby load-re.rb op_node_left
==2293== 
load-re.rb:2: warning: regular expression has ']' without escape
==2293== Invalid read of size 1
==2293==    at 0x498B32: add_char_opt_map_info (regcomp.c:4742)
==2293==    by 0x498B32: optimize_node_left (regcomp.c:4982)
==2293==    by 0x4966D7: optimize_node_left (regcomp.c:5228)
==2293==    by 0x4910A2: optimize_node_left (regcomp.c:4947)
==2293==    by 0x4B1FD4: set_optimize_info_from_tree (regcomp.c:5375)
==2293==    by 0x4B1FD4: onig_compile (regcomp.c:5825)
==2293==    by 0x44E6C9: onig_new_with_source (re.c:850)
==2293==    by 0x44E6C9: make_regexp (re.c:874)
==2293==    by 0x44E6C9: rb_reg_initialize (re.c:2732)
==2293==    by 0x44EC1D: rb_reg_initialize_str (re.c:2766)
==2293==    by 0x453A3D: rb_reg_init_str (re.c:2802)
==2293==    by 0x453A3D: rb_reg_initialize_m (re.c:3344)
==2293==    by 0x73E269: call_cfunc_m1 (vm_insnhelper.c:1460)
==2293==    by 0x7A7E1C: vm_call0_cfunc_with_frame (vm_eval.c:131)
==2293==    by 0x7A7E1C: vm_call0_cfunc (vm_eval.c:148)
==2293==    by 0x7A7E1C: vm_call0_body.constprop.118 (vm_eval.c:186)
==2293==    by 0x7AA85B: vm_call0 (vm_eval.c:61)
==2293==    by 0x7AA85B: rb_call0 (vm_eval.c:344)
==2293==    by 0x141AB9: rb_obj_call_init (eval.c:1319)
==2293==    by 0x2EDAC3: rb_class_new_instance (object.c:1853)
==2293==  Address 0x100 is not stack'd, malloc'd or (recently) free'd
==2293== 
load-re.rb:2: [BUG] Segmentation fault at 0x000100
ruby 2.4.0dev (2016-06-02) [i686-linux]

-- Control frame information -----------------------------------------------
c:0006 p:---- s:0017 e:000016 CFUNC  :initialize
c:0005 p:---- s:0015 e:000014 CFUNC  :new
c:0004 p:0036 s:0011 e:000010 BLOCK  load-re.rb:2 [FINISH]
c:0003 p:---- s:0008 e:000007 CFUNC  :open
c:0002 p:0024 s:0004 E:0020c0 EVAL   load-re.rb:1 [FINISH]
c:0001 p:0000 s:0002 E:001c18 (none) [FINISH]

-- Ruby level backtrace information ----------------------------------------
load-re.rb:1:in `<main>'
load-re.rb:1:in `open'
load-re.rb:2:in `block in <main>'
load-re.rb:2:in `new'
load-re.rb:2:in `initialize'

-- Machine register context ------------------------------------------------
  GS: 0x0000000b  FS: 0x00000000  ES: 0x0000007b  DS: 0x0000007b EDI: 0x00000100
 ESI: 0x00000000 EBP: 0xbeb5bc14 ESP: 0xbeb5b6e0 EBX: 0x00b37000 EDX: 0x05421750
 ECX: 0x04b27e38 EAX: 0xffff827f TRA: 0x0000000e ERR: 0x00000004 EIP: 0x00498b32
  CS: 0x00000073 EFL: 0x00000080 UES: 0x00000000  SS: 0x0000007b

-- C level backtrace information -------------------------------------------
/usr/local/bin/ruby(rb_vm_bugreport+0xec7) [0xa530a7] vm_dump.c:688
/usr/local/bin/ruby(rb_bug_context+0x12e) [0xa2627e] error.c:408
/usr/local/bin/ruby(sigsegv+0x88) [0x553548] signal.c:896
/lib/i386-linux-gnu/libpthread.so.0 [0x485f1e0]
/usr/local/bin/ruby(optimize_node_left+0x9212) [0x498b32] regcomp.c:4742
/usr/local/bin/ruby(optimize_node_left+0x6db8) [0x4966d8] regcomp.c:5228
/usr/local/bin/ruby(optimize_node_left+0x1783) [0x4910a3] regcomp.c:4947
/usr/local/bin/ruby(onig_compile+0x9d5) [0x4b1fd5] regcomp.c:5375
/usr/local/bin/ruby(rb_reg_initialize+0x74a) [0x44e6ca] re.c:850
/usr/local/bin/ruby(rb_reg_initialize_str+0x18e) [0x44ec1e] re.c:2766
/usr/local/bin/ruby(rb_reg_initialize_m+0x34e) [0x453a3e] re.c:2802
/usr/local/bin/ruby(call_cfunc_m1+0x5a) [0x73e26a] vm_insnhelper.c:1460
/usr/local/bin/ruby(vm_call0_body.constprop.118+0xa6d) [0x7a7e1d] vm_eval.c:131
/usr/local/bin/ruby(rb_call0+0x6cc) [0x7aa85c] vm_eval.c:61
/usr/local/bin/ruby(rb_obj_call_init+0x9a) [0x141aba] eval.c:1319
/usr/local/bin/ruby(rb_class_new_instance+0x74) [0x2edac4] object.c:1853
/usr/local/bin/ruby(call_cfunc_m1+0x5a) [0x73e26a] vm_insnhelper.c:1460
/usr/local/bin/ruby(vm_call_cfunc+0x37b) [0x757a6b] vm_insnhelper.c:1639
/usr/local/bin/ruby(vm_call_method_each_type+0x1cd) [0x79d0ad] vm_insnhelper.c:2026
/usr/local/bin/ruby(vm_call_method+0x25c) [0x79eedc] vm_insnhelper.c:2161
/usr/local/bin/ruby(vm_exec_core+0x7f48) [0x781ff8] insns.def:1064
/usr/local/bin/ruby(vm_exec+0xe3) [0x791bb3] vm.c:1653
/usr/local/bin/ruby(rb_yield+0x89b) [0x7a16fb] vm.c:973
/usr/local/bin/ruby(rb_ensure+0x135) [0x140245] eval.c:900
/usr/local/bin/ruby(rb_io_s_open+0xc8) [0x1e5768] io.c:6394
/usr/local/bin/ruby(call_cfunc_m1+0x5a) [0x73e26a] vm_insnhelper.c:1460
/usr/local/bin/ruby(vm_call_cfunc+0x37b) [0x757a6b] vm_insnhelper.c:1639
/usr/local/bin/ruby(vm_call_method_each_type+0x1cd) [0x79d0ad] vm_insnhelper.c:2026
/usr/local/bin/ruby(vm_call_method+0x25c) [0x79eedc] vm_insnhelper.c:2161
/usr/local/bin/ruby(vm_exec_core+0x6399) [0x780449] insns.def:965
/usr/local/bin/ruby(vm_exec+0xe3) [0x791bb3] vm.c:1653
/usr/local/bin/ruby(ruby_exec_internal+0x173) [0x12df93] eval.c:244
/usr/local/bin/ruby(ruby_run_node+0x1d5) [0x13cd35] eval.c:308
/usr/local/bin/ruby(main+0xb6) [0x12bc06] main.c:36

-- Other runtime information -----------------------------------------------

* Loaded script: load-re.rb

* Loaded features:

    0 enumerator.so
    1 thread.rb
    2 rational.so
    3 complex.so
    4 /usr/local/lib/ruby/2.4.0/i686-linux/enc/encdb.so
    5 /usr/local/lib/ruby/2.4.0/i686-linux/enc/trans/transdb.so
    6 /usr/local/lib/ruby/2.4.0/unicode_normalize.rb
    7 /usr/local/lib/ruby/2.4.0/i686-linux/rbconfig.rb
    8 /usr/local/lib/ruby/2.4.0/rubygems/compatibility.rb
    9 /usr/local/lib/ruby/2.4.0/rubygems/defaults.rb
   10 /usr/local/lib/ruby/2.4.0/rubygems/deprecate.rb
   11 /usr/local/lib/ruby/2.4.0/rubygems/errors.rb
   12 /usr/local/lib/ruby/2.4.0/rubygems/version.rb
   13 /usr/local/lib/ruby/2.4.0/rubygems/requirement.rb
   14 /usr/local/lib/ruby/2.4.0/rubygems/platform.rb
   15 /usr/local/lib/ruby/2.4.0/rubygems/basic_specification.rb
   16 /usr/local/lib/ruby/2.4.0/rubygems/stub_specification.rb
   17 /usr/local/lib/ruby/2.4.0/rubygems/util/list.rb
   18 /usr/local/lib/ruby/2.4.0/i686-linux/stringio.so
   19 /usr/local/lib/ruby/2.4.0/rubygems/specification.rb
   20 /usr/local/lib/ruby/2.4.0/rubygems/exceptions.rb
   21 /usr/local/lib/ruby/2.4.0/rubygems/core_ext/kernel_gem.rb
   22 /usr/local/lib/ruby/2.4.0/monitor.rb
   23 /usr/local/lib/ruby/2.4.0/rubygems/core_ext/kernel_require.rb
   24 /usr/local/lib/ruby/2.4.0/rubygems.rb
   25 /usr/local/lib/ruby/2.4.0/rubygems/dependency.rb
   26 /usr/local/lib/ruby/2.4.0/rubygems/path_support.rb

* Process memory map:

00108000-00b34000 r-xp 00000000 08:07 2498475    /usr/local/bin/ruby
00b34000-00b37000 r--p 00a2c000 08:07 2498475    /usr/local/bin/ruby
00b37000-00b38000 rw-p 00a2f000 08:07 2498475    /usr/local/bin/ruby
00b38000-00b40000 rw-p 00000000 00:00 0 
04000000-04020000 r-xp 00000000 08:07 917607     /lib/i386-linux-gnu/ld-2.19.so
04020000-04021000 r--p 0001f000 08:07 917607     /lib/i386-linux-gnu/ld-2.19.so
04021000-04022000 rw-p 00020000 08:07 917607     /lib/i386-linux-gnu/ld-2.19.so
04022000-04023000 rwxp 00000000 00:00 0 
04822000-04824000 rw-p 00000000 00:00 0 
04824000-04825000 r-xp 00000000 08:07 2110738    /usr/lib/valgrind/vgpreload_core-x86-linux.so
04825000-04826000 r--p 00000000 08:07 2110738    /usr/lib/valgrind/vgpreload_core-x86-linux.so
04826000-04827000 rw-p 00001000 08:07 2110738    /usr/lib/valgrind/vgpreload_core-x86-linux.so
04827000-04835000 r-xp 00000000 08:07 2110703    /usr/lib/valgrind/vgpreload_memcheck-x86-linux.so
04835000-04836000 r--p 0000d000 08:07 2110703    /usr/lib/valgrind/vgpreload_memcheck-x86-linux.so
04836000-04837000 rw-p 0000e000 08:07 2110703    /usr/lib/valgrind/vgpreload_memcheck-x86-linux.so
04837000-04838000 r--p 00855000 08:07 2105916    /usr/lib/locale/locale-archive
04838000-04839000 ---p 00000000 00:00 0 
04839000-0483c000 rw-p 00000000 00:00 0 
0483c000-0483e000 r-xp 00000000 08:07 1057096    /usr/local/lib/ruby/2.4.0/i686-linux/enc/encdb.so
0483e000-0483f000 r--p 00002000 08:07 1057096    /usr/local/lib/ruby/2.4.0/i686-linux/enc/encdb.so
0483f000-04840000 rw-p 00003000 08:07 1057096    /usr/local/lib/ruby/2.4.0/i686-linux/enc/encdb.so
04840000-04842000 r-xp 00000000 08:07 1057113    /usr/local/lib/ruby/2.4.0/i686-linux/enc/trans/transdb.so
04842000-04843000 r--p 00002000 08:07 1057113    /usr/local/lib/ruby/2.4.0/i686-linux/enc/trans/transdb.so
04843000-04844000 rw-p 00003000 08:07 1057113    /usr/local/lib/ruby/2.4.0/i686-linux/enc/trans/transdb.so
04844000-04845000 rw-p 00000000 00:00 0 
0484f000-04850000 rw-p 00000000 00:00 0 
04850000-04868000 r-xp 00000000 08:07 917596     /lib/i386-linux-gnu/libpthread-2.19.so
04868000-04869000 r--p 00018000 08:07 917596     /lib/i386-linux-gnu/libpthread-2.19.so
04869000-0486a000 rw-p 00019000 08:07 917596     /lib/i386-linux-gnu/libpthread-2.19.so
0486a000-0486c000 rw-p 00000000 00:00 0 
0486c000-0486f000 r-xp 00000000 08:07 917601     /lib/i386-linux-gnu/libdl-2.19.so
0486f000-04870000 r--p 00002000 08:07 917601     /lib/i386-linux-gnu/libdl-2.19.so
04870000-04871000 rw-p 00003000 08:07 917601     /lib/i386-linux-gnu/libdl-2.19.so
04871000-04879000 r-xp 00000000 08:07 917608     /lib/i386-linux-gnu/libcrypt-2.19.so
04879000-0487a000 r--p 00008000 08:07 917608     /lib/i386-linux-gnu/libcrypt-2.19.so
0487a000-0487b000 rw-p 00009000 08:07 917608     /lib/i386-linux-gnu/libcrypt-2.19.so
0487b000-048a2000 rw-p 00000000 00:00 0 
048a2000-048e6000 r-xp 00000000 08:07 917509     /lib/i386-linux-gnu/libm-2.19.so
048e6000-048e7000 r--p 00043000 08:07 917509     /lib/i386-linux-gnu/libm-2.19.so
048e7000-048e8000 rw-p 00044000 08:07 917509     /lib/i386-linux-gnu/libm-2.19.so
048e8000-04a90000 r-xp 00000000 08:07 917604     /lib/i386-linux-gnu/libc-2.19.so
04a90000-04a92000 r--p 001a8000 08:07 917604     /lib/i386-linux-gnu/libc-2.19.so
04a92000-04a93000 rw-p 001aa000 08:07 917604     /lib/i386-linux-gnu/libc-2.19.so
04a93000-04a98000 rw-p 00000000 00:00 0 
04a98000-04e98000 rwxp 00000000 00:00 0 
04e98000-05098000 r--p 00000000 08:07 2105916    /usr/lib/locale/locale-archive
05098000-05498000 rwxp 00000000 00:00 0 
05498000-054ad000 r-xp 00000000 08:07 3299233    /usr/local/lib/ruby/2.4.0/i686-linux/stringio.so
054ad000-054ae000 r--p 00014000 08:07 3299233    /usr/local/lib/ruby/2.4.0/i686-linux/stringio.so
054ae000-054af000 rw-p 00015000 08:07 3299233    /usr/local/lib/ruby/2.4.0/i686-linux/stringio.so
054c7000-054e3000 r-xp 00000000 08:07 917533     /lib/i386-linux-gnu/libgcc_s.so.1
054e3000-054e4000 rw-p 0001b000 08:07 917533     /lib/i386-linux-gnu/libgcc_s.so.1
054e4000-0680e000 r--s 00000000 08:07 2498475    /usr/local/bin/ruby
0680e000-0682f000 r--s 00000000 08:07 917596     /lib/i386-linux-gnu/libpthread-2.19.so
0682f000-068c4000 r--s 00000000 08:07 2098869    /usr/lib/debug/lib/i386-linux-gnu/libpthread-2.19.so
068c4000-06a71000 r--s 00000000 08:07 917604     /lib/i386-linux-gnu/libc-2.19.so
38000000-3837a000 r-xp 00000000 08:07 2110679    /usr/lib/valgrind/memcheck-x86-linux
3837b000-3837d000 rw-p 0037a000 08:07 2110679    /usr/lib/valgrind/memcheck-x86-linux
3837d000-3946d000 rw-p 00000000 00:00 0 
61db0000-62bf1000 rwxp 00000000 00:00 0 
62bf1000-62bf2000 rw-s 00000000 08:07 1712866    /tmp/vgdb-pipe-shared-mem-vgdb-2293-by-root-on-???
62bf2000-62c1e000 rwxp 00000000 00:00 0 
62c20000-62d21000 rwxp 00000000 00:00 0 
62d21000-62d23000 ---p 00000000 00:00 0 
62d23000-62e23000 rwxp 00000000 00:00 0          [stack:2293]
62e23000-62e25000 ---p 00000000 00:00 0 
62e25000-656a7000 rwxp 00000000 00:00 0 
656a9000-656f5000 rwxp 00000000 00:00 0 
656f7000-657f3000 rwxp 00000000 00:00 0 
657f3000-657f5000 ---p 00000000 00:00 0 
657f5000-658f5000 rwxp 00000000 00:00 0          [stack:2294]
658f5000-658f7000 ---p 00000000 00:00 0 
658f7000-659bf000 rwxp 00000000 00:00 0 
659c1000-68044000 rwxp 00000000 00:00 0 
b77e9000-b77eb000 r--p 00000000 00:00 0          [vvar]
be35f000-beb5e000 rw-p 00000000 00:00 0 
bfb3f000-bfb60000 rw-p 00000000 00:00 0 


[NOTE]
You may have encountered a bug in the Ruby interpreter or extension libraries.
Bug reports are welcome.
For details: http://www.ruby-lang.org/bugreport.html

==2293== 
==2293== HEAP SUMMARY:
==2293==     in use at exit: 2,406,986 bytes in 27,164 blocks
==2293==   total heap usage: 46,362 allocs, 19,198 frees, 5,393,134 bytes allocated
==2293== 
==2293== LEAK SUMMARY:
==2293==    definitely lost: 296 bytes in 2 blocks
==2293==    indirectly lost: 3,579 bytes in 70 blocks
==2293==      possibly lost: 1,493,795 bytes in 17,541 blocks
==2293==    still reachable: 909,316 bytes in 9,551 blocks
==2293==         suppressed: 0 bytes in 0 blocks
==2293== Rerun with --leak-check=full to see details of leaked memory
==2293== 
==2293== For counts of detected and suppressed errors, rerun with: -v
==2293== ERROR SUMMARY: 1 errors from 1 contexts (suppressed: 0 from 0)
Killed

Files

Download all files
load-re.rb (79 Bytes) load-re.rb grajagandev (David Moore), 06/02/2016 05:42 AM
op_node_left (58 Bytes) op_node_left grajagandev (David Moore), 06/02/2016 05:42 AM
bug-12454-ascii (49 Bytes) bug-12454-ascii grajagandev (David Moore), 06/02/2016 03:42 PM
Actions
Copy link
 #1 [ruby-core:75829]

Updated by grajagandev (David Moore) over 8 years ago

Here is a simpler case - it is an ascii only, valid regular expression according to grep:

# cat bug-12454-ascii
((?<V>0(00*)(\0>))00*00*00*$00*)(\g<0>)00*00*00*

# xxd bug-12454-ascii
0000000: 2828 3f3c 563e 3028 3030 2a29 285c 303e  ((?<V>0(00*)(\0>
0000010: 2929 3030 2a30 302a 3030 2a24 3030 2a29  ))00*00*00*$00*)
0000020: 285c 673c 303e 2930 302a 3030 2a30 302a  (\g<0>)00*00*00*
0000030: 24                                       $

# grep -f bug-12454-ascii -E -r
#
Actions
Copy link
 #2 [ruby-core:76213]

Updated by nobu (Nobuyoshi Nakada) over 8 years ago

More shorter code, probably same issue:

$ ruby -e 'Regexp.new("((?<v>))\\g<0>")'
-e:1:in `initialize': undefined type (bug): /((?<v>))\g<0>/ (RegexpError)
	from -e:1:in `new'
	from -e:1:in `<main>'
Actions
Copy link
 #3

Updated by naruse (Yui NARUSE) over 8 years ago

  • Status changed from Open to Closed

Applied in changeset r55562.

  • regcomp.c (noname_disable_map): don't optimize out group 0
    Ruby's Regexp doesn't allow normal numbered groups if the regexp
    has named groups. On such case it optimizes out related NT_ENCLOSE.
    But even on the case it can use \g<0>.
    This fix not to remove NT_ENCLOSE whose regnum is 0.
    [ruby-core:75828] [Bug #12454]
Actions
Copy link
 #4 [ruby-core:76710]

Updated by usa (Usaku NAKAMURA) over 8 years ago

  • Backport changed from 2.1: UNKNOWN, 2.2: UNKNOWN, 2.3: UNKNOWN to 2.1: WONTFIX, 2.2: REQUIRED, 2.3: REQUIRED
Actions
Copy link
 #5 [ruby-core:76888]

Updated by nagachika (Tomoyuki Chikanaga) over 8 years ago

  • Backport changed from 2.1: WONTFIX, 2.2: REQUIRED, 2.3: REQUIRED to 2.1: WONTFIX, 2.2: REQUIRED, 2.3: DONE

ruby_2_3 r55908 merged revision(s) 55562.

Actions
Copy link
 #6 [ruby-core:76929]

Updated by usa (Usaku NAKAMURA) over 8 years ago

  • Backport changed from 2.1: WONTFIX, 2.2: REQUIRED, 2.3: DONE to 2.1: WONTFIX, 2.2: DONE, 2.3: DONE

ruby_2_2 r55939 merged revision(s) 55562.

Actions
Copy link

Also available in: Atom PDF

Like0
Like0Like0Like0Like0Like0Like0