Feature #22549
openString#bit_put, the writer for the region form of String#bit_get
Description
PR: (WIP)
This is a reply to the request of Matz for a writer that corresponds to bit_get (#22483#note-6). The main question is a value that is wider than the region, and the section after the contracts compares the two answers.
What this adds¶
String#bit_put(offset, value, lsb_first: true) -> self
String#bit_put(offset, length, value, lsb_first: true) -> self
String#bit_put(range, value, lsb_first: true) -> self
bit_put writes the Integer value into a bit region and returns self. The arguments before value are the arguments of bit_get, with the same meaning of lsb_first:, so s.bit_get(*region) returns value after s.bit_put(*region, value). The value is the last argument, as in String#setbyte(index, integer) and IO::Buffer#set_value(type, offset, value). The one-bit form is the case length = 1.
data = "\x00\x00".b
data.bit_put(1, 1) # => "\x02\x00"
data.bit_put(4, 8, 0xAF) # => "\xF2\x0A"
data.bit_get(4, 8) # => 0xAF
hdr = "\x00\x00\x00\x00".b # a DNS header
hdr.bit_put(0, 16, 0x1234, lsb_first: false) # => "\x12\x34\x00\x00"
Use cases¶
-
Protocol headers (MSB-first).
bit_put(offset, width, value, lsb_first: false)writes a field by the coordinates of the RFC diagram, without shifts and masks oversetbyte.hdr = "\x00\x00\x00\x00".b # DNS header: ID, then the 16 flag bits hdr.bit_put(0, 16, 0x1234, lsb_first: false) # ID hdr.bit_put(16, 1, lsb_first: false) # QR hdr.bit_put(17, 4, 0, lsb_first: false) # OPCODE hdr.bit_put(23, 1, lsb_first: false) # RD hdr.bit_put(24, 1, lsb_first: false) # RA hdr # => "\x12\x34\x81\x80" -
Non-aligned sensor frames on a microcontroller. A telemetry format packs 12 + 10 + 14 bits into a 36-bit frame, so every second frame starts in the middle of a byte. With
setbytethe encoder needs a read-modify-write of the shared bytes and two code paths, one per alignment. Withbit_putit needs one. -
Hardware registers.
bit_put(5, 3, mode)writes a 3-bit mode field as the datasheet writes it, and leaves the other fields of the register image as they are.
API Contracts¶
- Arguments follow #22118 and #22279. A negative
offsetraisesIndexError. A negativelengthraisesArgumentError. A position or length beyond2**64 - 1raisesArgumentError. Thelsb_firstvalue must betrueorfalse. A Range with alengthraisesArgumentError, as inbit_count. - The region must lie within the receiver, with the rules of
bit_set: a region that extends past the end raisesIndexErrorand is not clamped.bit_putnever extends the receiver. An empty region accepts only0and writes nothing. - The value must fit in the region. A
valueoutside0...(1 << length)raisesRangeError, and the receiver is not changed. This includes a negativevalue. The one-bit form accepts0and1. The next two sections give the reasons. lsb_firstchanges the bit numbering and the significance, as inbit_get. The byte order never changes. Withlsb_first: true, the least significant bit ofvaluegoes to the first bit of the region. Withlsb_first: false, bitlength - 1ofvaluegoes there, which is the layout that an RFC diagram shows for the field.- Frozen receivers raise
FrozenError, even for an empty region, as inbit_set. Encoding has no effect. The method treats the receiver as a byte sequence, as in #22118.
The main question: a value wider than the region¶
s.bit_put(0, 4, 0x1F) gives a 5-bit value to a 4-bit region. Two answers are reasonable.
- A: RangeError. Nothing is written.
- B: truncate. The low
lengthbits are written, and the other bits are dropped.
Current Ruby has precedents for both. String#setbyte(0, 0x141), [0x141].pack("C") and IO::Buffer#set_value(:U8, 0, 0x141) write 0x41 (B: truncate). 256.chr raises an error (A: RangeError).
The choice depends on what the field is for, so I sorted the use cases by that.
| The field is | Examples | Overflow is | Better |
|---|---|---|---|
| a container for a value | protocol header, sensor frame, device register | a bug | A |
| a register of modular arithmetic | 4-bit continuity counter of MPEG-TS, checksum, low 12 bits of a hash, 8-bit register of a CPU emulator | intended | B |
The first row holds the use cases of this ticket and of #22483. With B, an overflow there is silent and the program sends a corrupt packet.
In the second row the wrap is intended. The continuity counter of an MPEG-TS packet is a 4-bit field that goes from 15 back to 0. With A, the caller writes the wrap as a mask:
packet = "\x47\x00\x00\x10".b # the counter is the last 4 bits of the header
counter = 15
packet.bit_put(28, 4, counter, lsb_first: false) # => "\x47\x00\x00\x1F"
counter += 1 # 16 does not fit in 4 bits
packet.bit_put(28, 4, counter, lsb_first: false) # RangeError: 16 out of 4-bit range
packet.bit_put(28, 4, counter & 0xF, lsb_first: false) # => "\x47\x00\x00\x10", the field is 0
The cost of A here is the & 0xF (& 0b1111). It is one expression at a line where the caller knows the width, and it shows the reader that the wrap is intended. The same mask writes a negative value as two's complement, because -1 & 0xF is 15.
I propose A.
- The round trip with
bit_getholds without a condition. - It is the policy of the other bit methods:
bit_getdoes not clamp a region because "a truncated value would be wrong", andbit_setraisesIndexErrorinstead of a clamp. - The caller who wants B writes one mask. The caller who wants A under B writes a guard at every call, and a missing guard is silent.
- A can be relaxed later. B cannot be made strict later.
Why RangeError, not ArgumentError¶
The other bit methods raise ArgumentError for a negative length, for a position beyond 2**64 - 1, for a wrong lsb_first value and for a wrong number of arguments. All of them are mistakes in the code of the caller. A value that does not fit is different: it depends on the data at run time, and it is the one error of bit_put that a caller can want to handle.
With a class of its own, rescue RangeError catches the overflow of the value and nothing else.
def put_saturated(s, offset, width, value)
s.bit_put(offset, width, value)
rescue RangeError
s.bit_put(offset, width, value.negative? ? 0 : (1 << width) - 1)
end
s = "\x00\x00".b
put_saturated(s, 4, 12, 5000) # => "\xF0\xFF", the field holds 4095
put_saturated(s, 4, -1, 5000) # ArgumentError: negative bit length
put_saturated(s, 8, 12, 5000) # IndexError: bit range out of range
With ArgumentError, the same rescue would also catch the negative length of the second call, and a mistake in the code would pass as a saturated write. The three classes give one meaning each: ArgumentError for a wrong call, IndexError for a wrong position, and RangeError for a wrong value.
A negative value raises RangeError too. The condition is one, "outside 0...(1 << length)", so one rescue covers both sides of it. Integer#chr sets the precedent: 256.chr and -1.chr raise the same RangeError.
Scope: no overflow: keyword in this ticket¶
A keyword such as overflow: :wrap or overflow: :saturate, like the OVERFLOW of Redis BITFIELD, can be added later on top of A without an incompatibility. The use cases above do not need it.
No data to display