Project

General

Profile

Actions

Feature #22549

open

String#bit_put, the writer for the region form of String#bit_get

Feature #22549: String#bit_put, the writer for the region form of String#bit_get

Added by hasumikin (hitoshi hasumi) 3 days ago.

Status:
Open
Assignee:
-
Target version:
-
[ruby-dev:<unknown>]

Description

PR: (WIP)

This is a reply to the request of Matz for a writer that corresponds to bit_get (#22483#note-6). The main question is a value that is wider than the region, and the section after the contracts compares the two answers.

What this adds

String#bit_put(offset, value, lsb_first: true)          -> self
String#bit_put(offset, length, value, lsb_first: true)  -> self
String#bit_put(range, value, lsb_first: true)           -> self

bit_put writes the Integer value into a bit region and returns self. The arguments before value are the arguments of bit_get, with the same meaning of lsb_first:, so s.bit_get(*region) returns value after s.bit_put(*region, value). The value is the last argument, as in String#setbyte(index, integer) and IO::Buffer#set_value(type, offset, value). The one-bit form is the case length = 1.

data = "\x00\x00".b
data.bit_put(1, 1)                  # => "\x02\x00"
data.bit_put(4, 8, 0xAF)            # => "\xF2\x0A"
data.bit_get(4, 8)                  # => 0xAF

hdr = "\x00\x00\x00\x00".b                    # a DNS header
hdr.bit_put(0, 16, 0x1234, lsb_first: false)  # => "\x12\x34\x00\x00"

Use cases

  • Protocol headers (MSB-first). bit_put(offset, width, value, lsb_first: false) writes a field by the coordinates of the RFC diagram, without shifts and masks over setbyte.

    hdr = "\x00\x00\x00\x00".b                    # DNS header: ID, then the 16 flag bits
    hdr.bit_put(0, 16, 0x1234, lsb_first: false)  # ID
    hdr.bit_put(16, 1, lsb_first: false)          # QR
    hdr.bit_put(17, 4, 0, lsb_first: false)       # OPCODE
    hdr.bit_put(23, 1, lsb_first: false)          # RD
    hdr.bit_put(24, 1, lsb_first: false)          # RA
    hdr # => "\x12\x34\x81\x80"
    
  • Non-aligned sensor frames on a microcontroller. A telemetry format packs 12 + 10 + 14 bits into a 36-bit frame, so every second frame starts in the middle of a byte. With setbyte the encoder needs a read-modify-write of the shared bytes and two code paths, one per alignment. With bit_put it needs one.

    frames.each_with_index do |(temp, hum, co2), i|
      off = i * 36
      data.bit_put(off, 12, temp)
      data.bit_put(off + 12, 10, hum)
      data.bit_put(off + 22, 14, co2)
    end
    
  • Hardware registers. bit_put(5, 3, mode) writes a 3-bit mode field as the datasheet writes it, and leaves the other fields of the register image as they are.

API Contracts

  1. Arguments follow #22118 and #22279. A negative offset raises IndexError. A negative length raises ArgumentError. A position or length beyond 2**64 - 1 raises ArgumentError. The lsb_first value must be true or false. A Range with a length raises ArgumentError, as in bit_count.
  2. The region must lie within the receiver, with the rules of bit_set: a region that extends past the end raises IndexError and is not clamped. bit_put never extends the receiver. An empty region accepts only 0 and writes nothing.
  3. The value must fit in the region. A value outside 0...(1 << length) raises RangeError, and the receiver is not changed. This includes a negative value. The one-bit form accepts 0 and 1. The next two sections give the reasons.
  4. lsb_first changes the bit numbering and the significance, as in bit_get. The byte order never changes. With lsb_first: true, the least significant bit of value goes to the first bit of the region. With lsb_first: false, bit length - 1 of value goes there, which is the layout that an RFC diagram shows for the field.
  5. Frozen receivers raise FrozenError, even for an empty region, as in bit_set. Encoding has no effect. The method treats the receiver as a byte sequence, as in #22118.

The main question: a value wider than the region

s.bit_put(0, 4, 0x1F) gives a 5-bit value to a 4-bit region. Two answers are reasonable.

  • A: RangeError. Nothing is written.
  • B: truncate. The low length bits are written, and the other bits are dropped.

Current Ruby has precedents for both. String#setbyte(0, 0x141), [0x141].pack("C") and IO::Buffer#set_value(:U8, 0, 0x141) write 0x41 (B: truncate). 256.chr raises an error (A: RangeError).

The choice depends on what the field is for, so I sorted the use cases by that.

The field is Examples Overflow is Better
a container for a value protocol header, sensor frame, device register a bug A
a register of modular arithmetic 4-bit continuity counter of MPEG-TS, checksum, low 12 bits of a hash, 8-bit register of a CPU emulator intended B

The first row holds the use cases of this ticket and of #22483. With B, an overflow there is silent and the program sends a corrupt packet.

In the second row the wrap is intended. The continuity counter of an MPEG-TS packet is a 4-bit field that goes from 15 back to 0. With A, the caller writes the wrap as a mask:

packet = "\x47\x00\x00\x10".b   # the counter is the last 4 bits of the header
counter = 15
packet.bit_put(28, 4, counter, lsb_first: false)        # => "\x47\x00\x00\x1F"

counter += 1                                            # 16 does not fit in 4 bits
packet.bit_put(28, 4, counter, lsb_first: false)        # RangeError: 16 out of 4-bit range
packet.bit_put(28, 4, counter & 0xF, lsb_first: false)  # => "\x47\x00\x00\x10", the field is 0

The cost of A here is the & 0xF (& 0b1111). It is one expression at a line where the caller knows the width, and it shows the reader that the wrap is intended. The same mask writes a negative value as two's complement, because -1 & 0xF is 15.

I propose A.

  • The round trip with bit_get holds without a condition.
  • It is the policy of the other bit methods: bit_get does not clamp a region because "a truncated value would be wrong", and bit_set raises IndexError instead of a clamp.
  • The caller who wants B writes one mask. The caller who wants A under B writes a guard at every call, and a missing guard is silent.
  • A can be relaxed later. B cannot be made strict later.

Why RangeError, not ArgumentError

The other bit methods raise ArgumentError for a negative length, for a position beyond 2**64 - 1, for a wrong lsb_first value and for a wrong number of arguments. All of them are mistakes in the code of the caller. A value that does not fit is different: it depends on the data at run time, and it is the one error of bit_put that a caller can want to handle.

With a class of its own, rescue RangeError catches the overflow of the value and nothing else.

def put_saturated(s, offset, width, value)
  s.bit_put(offset, width, value)
rescue RangeError
  s.bit_put(offset, width, value.negative? ? 0 : (1 << width) - 1)
end

s = "\x00\x00".b
put_saturated(s, 4, 12, 5000)   # => "\xF0\xFF", the field holds 4095
put_saturated(s, 4, -1, 5000)   # ArgumentError: negative bit length
put_saturated(s, 8, 12, 5000)   # IndexError: bit range out of range

With ArgumentError, the same rescue would also catch the negative length of the second call, and a mistake in the code would pass as a saturated write. The three classes give one meaning each: ArgumentError for a wrong call, IndexError for a wrong position, and RangeError for a wrong value.

A negative value raises RangeError too. The condition is one, "outside 0...(1 << length)", so one rescue covers both sides of it. Integer#chr sets the precedent: 256.chr and -1.chr raise the same RangeError.

Scope: no overflow: keyword in this ticket

A keyword such as overflow: :wrap or overflow: :saturate, like the OVERFLOW of Redis BITFIELD, can be added later on top of A without an incompatibility. The use cases above do not need it.

No data to display

Actions

Also available in: PDF Atom