Ruby » Ruby master

Bug #12191

Updated by shyouhei (Shyouhei Urabe) about 8 years ago

Hi, I work with IBM's XL compiler and we're noticing that there we're getting compile time failures due to ANSI aliasing rule violations.  

 For example, in https://github.com/ruby/ruby/blob/trunk/sprintf.c    This function:  

 ~~~C 
 rb_str_vcatf(VALUE str, const char *fmt, va_list ap) 
 { 
     rb_printf_buffer_extra buffer; 
 #define f buffer.base 
     VALUE klass; 

     StringValue(str); 
     rb_str_modify(str); 
     f._flags = __SWR | __SSTR; 
     f._bf._size = 0; 
     f._w = rb_str_capacity(str); 
     f._bf._base = (unsigned char *)str; 
 **      f._p = (unsigned char *)RSTRING_END(str); 
     klass = RBASIC(str)->klass; 
     RBASIC_CLEAR_CLASS(str); 
 **      f.vwrite = ruby__sfvwrite; 
     f.vextra = ruby__sfvextra; 
     buffer.value = 0; 
     BSD_vfprintf(&f, fmt, ap); 
     RBASIC_SET_CLASS_RAW(str, klass); 
     rb_str_resize(str, (char *)f._p - RSTRING_PTR(str)); 
 #undef f 

     return str; 
 } 
 ~~~ 

 When the bolded macros are expanded, they look like this:  

 ~~~ 
 include/ruby/ruby.h:869:#define RSTRING_END(str) \ 
 include/ruby/ruby.h-870-      (!(RBASIC(str)->flags & RSTRING_NOEMBED) ? \ 
 include/ruby/ruby.h-871-       (RSTRING(str)->as.ary + RSTRING_EMBED_LEN(str)) : \ 
 include/ruby/ruby.h-872-       (RSTRING(str)->as.heap.ptr + RSTRING(str)->as.heap.len)) 

 include/ruby/ruby.h:1086:#define RSTRING(obj) (R_CAST(RString)(obj)) 

 include/ruby/ruby.h:1082:#define RBASIC(obj)    (R_CAST(RBasic)(obj)) 

 include/ruby/ruby.h:1081:#define R_CAST(st)     (struct st*) 

 internal.h:852:#define RBASIC_CLEAR_CLASS(obj)          (((struct RBasicRaw *)((VALUE)(obj)))->klass = 0) 
 ~~~ 

 


 The function violates the ANSI aliasing rule since it takes an unsigned long, casts it to a pointer to either RBasic or RBasicRaw and then dereferences it. (RBasic).klass and (RBasicRaw).klass both alias unsigned long, but not each other, as RBasic and RBasicRaw are different types.  


 Additionally, other functions in sprintf.c also seem to have aliasing violations.   

 A fix such as changing line https://github.com/ruby/ruby/blob/trunk/internal.h#L1063 from  

 ~~~ 
  
 #define RBASIC_CLEAR_CLASS(obj)          (((struct RBasicRaw *)((VALUE)(obj)))->klass = 0) 
 ~~~ 
 To: 

 ~~~ 
 #ifdef __ibmxl__ 
 #define RBASIC_CLEAR_CLASS(obj)          memset(&(((struct RBasicRaw *)((VALUE)(obj)))->klass), 0, sizeof(((struct RBasicRaw *)((VALUE)(obj)))->klass)) 
 #else 
 #define RBASIC_CLEAR_CLASS(obj)          (((struct RBasicRaw *)((VALUE)(obj)))->klass = 0) 
 #endif 
 ~~~ 

 but there should be no need to make a special case for the XL compiler as it's following the ANSI aliasing rules.   

Back