Actions
Feature #16655
closed
Each test on test-all should run `srand(seed)` at setup
Description
Points¶
- call
srand($seed)before each test.$seedis given seed number by--seed N
Background¶
Some tests use rand() to produce random number to generate random test examples.
Example:
def test_local_variable_set_wb
assert_ruby_status([], <<-'end;', '[Bug #13605]', timeout: 30)
b = binding
n = 20_000
n.times do |i|
v = rand(2_000)
name = "n#{v}"
value = Object.new
b.local_variable_set name, value
end
end;
end
This test generates 2,000 randomly named ivar which point random numbers.
However, several tests calls srand(0) and this random sequence is affected by test methods order.
Two problems:
- Test authors of this kind of tests should assume that every test processes uses very different random sequence to increase the test coverage.
- We can not get reproducible results on such tests which use random sequence with
--seedbecause ofsrand(0)using tests.
Proposal¶
To solve the above two problems, I'll insert srand($seed) at the beginning of each test ($seed is given by --seed option).
This proposal respects --seed option more, so we can expect more deterministic results.
I measured performance overhead and I can't get meaningful slow down.
Additional note¶
Surprisingly, --seed option is ignored just now on test-all, so I'll fix this problem too.
Updated by 
Updated by
Actions
Like0
Like0Like0Like0