Project

General

Profile

Misc #15220

Adding OpenSSL 1.1.1 on Travis CI gcc-8 case

Added by jaruga (Jun Aruga) 7 days ago. Updated about 16 hours ago.

Status:
Open
Priority:
Normal
Assignee:
-
[ruby-core:89342]

Description

Currently Travis CI test cases are running on OpenSSL 1.0.1f as the default setting.

I want to add the latest version OpenSSL 1.1.1 to the gcc-8 test case on Travis CI.

I sent pull-request for that.
https://github.com/ruby/ruby/pull/1980

Motivation

The motivation is that ruby/openssl has very good CI environment covering the supported SSL collections.

However the benefits to enable OpenSSL 1.1.1 for the main repository (ruby/ruby) are

  1. To make us check the entire logic on the latest OpenSSL as early as possible. For example, I can find this kind of issue [1] as early as possible.

  2. We can check it on ruby-2.5 branch too. That is related to #15219 [2]

As we are working for OpenSSL 1.1.1 on Windows CI [3] and python project is testing on the custom OpenSSL built from the source code [4], I think that we can adapt it to Travis CI.

Detail of implementation

In the new process, the OpenSSL is built from source code.
That takes 134.63 sec = 2 min 14 sec
I could not find the better idea than that.

I found the binary openssl package that someone is managing [5].
But it might not be valid for Trusty. And it seems that the repository is quite personal one.

I am using cache/directories element [6].

I created tool/install_openssl.sh script. But some logic might be moved to configure.ac like [7].

How do you think?

History

#1 Updated by jaruga (Jun Aruga) 7 days ago

  • Description updated (diff)

#2 [ruby-core:89366] Updated by shyouhei (Shyouhei Urabe) 6 days ago

First, I understand the motivation behind this request. We should enrich our build matrix.

That said, 2+ min overhead for each and every time we commit something is too much. I can hardly +1 this particular patch.

Maybe we can use docker like ruby/openssl does today, or maybe we can have our own .deb prepared somewhere.

#3 [ruby-core:89389] Updated by jaruga (Jun Aruga) 4 days ago

Maybe we can use docker like ruby/openssl does today, or maybe we can have our own .deb prepared somewhere.

I sent pull-request to use docker on the only gcc-8 test case as an experiment.
https://github.com/ruby/ruby/pull/1983

The result on my repository is here.
https://travis-ci.org/junaruga/ruby/builds/440699457

I installed needed OpenSSL 1.1.1 in the docker container image in advance.
But unfortunately the total running time is longer than above OpenSSL source compiled case.
But I think this experiment includes some tips to run ruby tests in the docker container.

I did put the built container image here.
https://hub.docker.com/r/junaruga/ruby-docker/tags/

Top directory's Dockerfile would refer this container image.

Use case

Build the base container, and push the built image to the repository.

$ cd tool/ci
$ docker build --rm --no-cache -t ruby-docker .
$ docker login docker.io
$ docker tag ruby-docker docker.io/junaruga/ruby-docker:test
$ docker push docker.io/junaruga/ruby-docker:test

Run the test on local.
If you are using docker >= 17.06, maybe you need --network=host option to docker run.
See .travis.yml for detail.

$ cd ../.. (<= Back to top directory)

$ docker build --rm -t ruby .
$ docker run --rm -t ruby tool/ci/test.sh

You can set arguments optionally.

$ docker build --rm \
  --build-arg TEST_USER=travis \
  --build-arg WORK_DIR=$(pwd) \
  -t ruby .

$ docker run \
  -e CC="gcc-8" \
  -e OPENSSL_VERSION="1.1.1" \
  ruby tool/ci/test.sh
  • tool/ci/test.sh is same with commands on previous Travis's before_install, before_script and script section.
  • We might be able to run below test commands in parallel with multi Travis instances to save the running time.
make -s test => Travis instance 1
make -s test-all => Travis instance 2
make -s test-spec => Travis instance 1

#4 [ruby-core:89390] Updated by jaruga (Jun Aruga) 4 days ago

  • As an another idea to run the heavy task such as installing openssl from source or running the docker. If we are running on Travis's cron mode too, we can run the heavy tasks for only cron running.

.travis.yml

script: |
  if [ "${TRAVIS_EVENT_TYPE}" = "cron" ]; then
    a heavy task
  else
    a light task
  fi

#5 [ruby-core:89417] Updated by jaruga (Jun Aruga) about 16 hours ago

So far I proposed 2 possible solutions to test with OpenSSL 1.1.1 with the pull-requests.

  • 1. Add OpenSSL 1.1.1 test case to Travis CI: https://github.com/ruby/ruby/pull/1980
    • Demerit:
      • 2+ minutes to install OpenSSL 1.1.1 every time is not comfortable.
      • Added new commit for conditional logic with TRAVIS_EVENT_TYPE is not clear from Travis jobs page.
  • 2. Use docker image to test OpenSSL 1.1.1: https://github.com/ruby/ruby/pull/1983
    • Merit: It's a scalable way to test on various environment with source built dependencies.
    • Demerit:
      • Total running time 17 minutes is longer than the solution 1.
      • It's harder to maintain the docker image.

And here is 3rd possible solution now.
This is the most recommended and realistic way that I think. :)

Add a conditional job to test new version OpenSSL 1.1.1 to Travis CI.
https://github.com/ruby/ruby/pull/1984

  • Merit:
    • The total running time is same with current one.
    • We can check supported test cases regularly when Travis is run on cron mode or we set special environment variable from Travis setting page.
    • The run job is clear from Travis jobs page.
  • Demerit:
    • OpenSSL 1.1.1 test case is not run on pull-request.

Also available in: Atom PDF