Project

General

Profile

Backport #3297

[patch] Array#pack("i_") is broken in big endian 64bit systems

Added by kkaempf (Klaus Kämpf) over 9 years ago. Updated about 2 months ago.

Status:
Closed
Priority:
Normal
Assignee:
-
[ruby-core:30246]

Description

=begin
See also http://redmine.ruby-lang.org/issues/show/1131 for a similar issue (String#unpack("V"))

Array#pack("i_") is broken in big endian 64 bit systems

irb> [1].pack("i_")
=> "\000\000\000\000"

The root cause is similar to the one of issue 1131, use of a wrong offset in a 64bit long.
But the fix is a bit simpler and is probably correct for all other architectures too.

According to the documentation, the "i" format of pack/unpack uses a fixed-size (32bit) integer, while adding "_" uses the system-size.
However, the implementation of Array#pack uses a 'long' type (64bit !) to store an intermediate value, thus breaking OFF() in accessing the wrong 4 bytes of the value.

Fix: Use 'int' to store the intermediate value (use of int is correct for the fixed-size as well as the system-size case) and don't use OFF()

With this patch, the pack_spec and unpack_spec tests of rubyspec.org run without issues.
=end


Files

array_pack.patch (584 Bytes) array_pack.patch kkaempf (Klaus Kämpf), 05/15/2010 04:10 AM

History

#1

Updated by akr (Akira Tanaka) over 9 years ago

=begin
OFF* macros are already removed from Ruby 1.8.8.

=end

Updated by shyouhei (Shyouhei Urabe) over 8 years ago

  • Status changed from Open to Feedback

Sorry for this late reply, does this still happen? I guess it has already been fixed for a while.

#3

Updated by jeremyevans0 (Jeremy Evans) about 2 months ago

  • Status changed from Feedback to Closed

Also available in: Atom PDF