Bug #14078: Test Failure - OpenSSL::TestSSLSession#test_ctx_server_session_cb - with OpenSSL 1.1.0g

• Status: Rejected
• Assignee: -
• Target version: -
• ruby -v: ruby 2.5.0dev (2017-11-04 trunk 60635) [x64-mingw32]
• Backport: 2.3: UNKNOWN, 2.4: UNKNOWN

[MSP-Greg (Greg L)]

The most recent ruby-loco MinGW build (using OpenSSL 1.1.0g) failed with the following failure. It was the only failure/error in the build.

Retrying...
[1/9] OpenSSL::TestSSLSession#test_ctx_client_session_cb = 0.10 s
[2/9] OpenSSL::TestSSLSession#test_ctx_server_session_cb = 0.28 s
[3/9] OpenSSL::TestSSLSession#test_dup = 0.00 s
[4/9] OpenSSL::TestSSLSession#test_resumption = 0.36 s
[5/9] OpenSSL::TestSSLSession#test_server_session_cache = 0.75 s
[6/9] OpenSSL::TestSSLSession#test_session = 0.09 s
[7/9] OpenSSL::TestSSLSession#test_session_exts_read = 0.00 s
[8/9] OpenSSL::TestSSLSession#test_session_time = 0.00 s
[9/9] OpenSSL::TestSSLSession#test_session_timeout = 0.00 s

  1) Failure:
OpenSSL::TestSSLSession#test_ctx_server_session_cb [C:/projects/ruby-loco/src/ruby/test/openssl/utils.rb:268]:
exceptions on 1 threads:
#<Thread:0x000000000299c008@C:/projects/ruby-loco/src/ruby/test/openssl/utils.rb:244 dead>:
C:/projects/ruby-loco/src/ruby/test/lib/minitest/unit.rb:201:in `assert': <true> expected but was
<false>. (MiniTest::Assertion)
	from C:/projects/ruby-loco/src/ruby/test/lib/test/unit/assertions.rb:37:in `assert'
	from C:/projects/ruby-loco/src/ruby/test/lib/test/unit/assertions.rb:300:in `assert_equal'
	from C:/projects/ruby-loco/src/ruby/test/openssl/test_ssl_session.rb:338:in `block (2 levels) in test_ctx_server_session_cb'
	from C:/projects/ruby-loco/src/ruby/test/openssl/test_ssl_session.rb:388:in `server_connect_with_session'
	from C:/projects/ruby-loco/src/ruby/test/openssl/test_ssl_session.rb:331:in `block in test_ctx_server_session_cb'
	from C:/projects/ruby-loco/src/ruby/test/openssl/utils.rb:246:in `block (2 levels) in start_server'

I did look at the openssl/openssl pr mentioned in the test comments, but the conversation was beyond my c & OpenSSL knowledge... Thanks.

[rhenium (Kazuki Yamaguchi)]

• Status changed from Open to Feedback

I don't reproduce. Are you sure the DLLs used at runtime are from OpenSSL 1.1.0g?

[MSP-Greg (Greg L)]

Kazuki,

Are you sure the DLLs used at runtime are from OpenSSL 1.1.0g?

Yes. MSYS2/MinGW only allows one package to be installed at once, so Appveyor can't have two available for the build. Also, downloading the build from Appveyor and installing locally, both OPENSSL_VERSION and OPENSSL_LIBRARY_VERSION are OpenSSL 1.1.0g 2 Nov 2017.

Now that I'm running locally, all tests pass (I ran it several times), as shown below:

ruby -v --disable-gems runner.rb -j3 -v -I./lib --show-skip openssl
391 tests, 3905 assertions, 0 failures, 0 errors, 6 skips

Another Appveyor rolling build will run at 6:00 P JST, I guess we'll see if it's intermittent. Thanks for 60636, I did notice (later) that you had a different test upstream...
Greg

[MSP-Greg (Greg L)]

I've tried to repro this locally and on Appveyor.

Running the test twenty times, using runner.rb, both parallel and not, no failures, see https://ci.appveyor.com/project/MSP-Greg/testing-appveyor/build/4

Locally, I can't repro with a full build.

But, it was failing with a full Appveyor build. For now, I've patched around it. I'll try to check it from time to time in the future.

@rhenium (Kazuki Yamaguchi) - I'm okay to close, I'll revisit if I determine what's going on...

[rhenium (Kazuki Yamaguchi)]

• Status changed from Feedback to Rejected

OK, closing this for now.

[MSP-Greg (Greg L)]

@rhenium (Kazuki Yamaguchi)

Re the beta, thought I'd update with recent OpenSSL::TestSSLSession#test_ctx_server_session_cb results, all using 1.1.0g

1. Passed - MinGW trunk local when building (make test-all -j3)

2. Passed - vc140 trunk local when building (make test-all -j3), custom vc140 builds of openssl & zlib

3. Passed - MinGW & vc10 local builds when tested with runner and test/openssl folder

4. Passed - Appveyor test of ruby/openssl fork, using ruby-loco trunk, see https://ci.appveyor.com/project/MSP-Greg/openssl-ukpb8/build/1

5. Failed - Appveyor ruby-loco build without openssl patch, result below:

  1) Failure:
OpenSSL::TestSSLSession#test_ctx_server_session_cb [C:/projects/ruby-loco/src/ruby/test/openssl/utils.rb:268]:
exceptions on 1 threads:
#<Thread:0x000000000287bc78@C:/projects/ruby-loco/src/ruby/test/openssl/utils.rb:244 dead>:
C:/projects/ruby-loco/src/ruby/test/lib/minitest/unit.rb:201:in `assert': <true> expected but was
<false>. (MiniTest::Assertion)
	from C:/projects/ruby-loco/src/ruby/test/lib/test/unit/assertions.rb:37:in `assert'
	from C:/projects/ruby-loco/src/ruby/test/lib/test/unit/assertions.rb:300:in `assert_equal'
	from C:/projects/ruby-loco/src/ruby/test/openssl/test_ssl_session.rb:338:in `block (2 levels) in test_ctx_server_session_cb'
	from C:/projects/ruby-loco/src/ruby/test/openssl/test_ssl_session.rb:388:in `server_connect_with_session'
	from C:/projects/ruby-loco/src/ruby/test/openssl/test_ssl_session.rb:331:in `block in test_ctx_server_session_cb'
	from C:/projects/ruby-loco/src/ruby/test/openssl/utils.rb:246:in `block (2 levels) in start_server'

Summing up, all local tests pass, and the ruby/openssl repo passes. With the rolling ruby-loco build on Appveyor and make test-all -j3, it still fails.