Bug #12433: uncommon.mk:614: recipe for target 'yes-test-almost' failed - missing UnicodeData.txt

• Status: Closed
• Assignee: nobu (Nobuyoshi Nakada)
• Target version: -
• ruby -v: ruby -v: ruby 2.4.0dev (2016-05-27 trunk 55184) [x86_64-linux]
• Backport: 2.1: DONTNEED, 2.2: DONTNEED, 2.3: DONTNEED

[vo.x (Vit Ondruch)]

Trying to run make check from latest snapshot, the test suite fails:

/builddir/build/BUILD/ruby-2.4.0-r55184/test/ruby/enc/test_case_comprehensive.rb:25:in `readlines': No such file or directory @ rb_sysopen - /builddir/build/BUILD/ruby-2.4.0-r55184/enc/unicode/data/8.0.0/UnicodeData.txt (Errno::ENOENT)
	from /builddir/build/BUILD/ruby-2.4.0-r55184/test/ruby/enc/test_case_comprehensive.rb:25:in `read_data_file'
	from /builddir/build/BUILD/ruby-2.4.0-r55184/test/ruby/enc/test_case_comprehensive.rb:51:in `read_data'
	from /builddir/build/BUILD/ruby-2.4.0-r55184/test/ruby/enc/test_case_comprehensive.rb:92:in `all_tests'
	from /builddir/build/BUILD/ruby-2.4.0-r55184/test/ruby/enc/test_case_comprehensive.rb:96:in `generate_casefold_tests'
	from /builddir/build/BUILD/ruby-2.4.0-r55184/test/ruby/enc/test_case_comprehensive.rb:114:in `<class:TestComprehensiveCaseFold>'
	from /builddir/build/BUILD/ruby-2.4.0-r55184/test/ruby/enc/test_case_comprehensive.rb:16:in `<top (required)>'
	from /builddir/build/BUILD/ruby-2.4.0-r55184/lib/rubygems/core_ext/kernel_require.rb:55:in `require'
	from /builddir/build/BUILD/ruby-2.4.0-r55184/lib/rubygems/core_ext/kernel_require.rb:55:in `require'
	from /builddir/build/BUILD/ruby-2.4.0-r55184/test/lib/test/unit.rb:842:in `block in non_options'
	from /builddir/build/BUILD/ruby-2.4.0-r55184/test/lib/test/unit.rb:836:in `each'
	from /builddir/build/BUILD/ruby-2.4.0-r55184/test/lib/test/unit.rb:836:in `non_options'
	from /builddir/build/BUILD/ruby-2.4.0-r55184/test/lib/test/unit.rb:64:in `process_args'
	from /builddir/build/BUILD/ruby-2.4.0-r55184/test/lib/test/unit.rb:130:in `process_args'
	from /builddir/build/BUILD/ruby-2.4.0-r55184/test/lib/test/unit.rb:981:in `process_args'
	from /builddir/build/BUILD/ruby-2.4.0-r55184/test/lib/test/unit.rb:986:in `run'
	from /builddir/build/BUILD/ruby-2.4.0-r55184/test/lib/test/unit.rb:993:in `run'
	from ./test/runner.rb:40:in `<main>'
uncommon.mk:614: recipe for target 'yes-test-almost' failed
make: *** [yes-test-almost] Error 1

This is very likely due to r55168

[duerst (Martin Dürst)]

• Assignee changed from duerst (Martin Dürst) to nobu (Nobuyoshi Nakada)

Vit Ondruch wrote:

/builddir/build/BUILD/ruby-2.4.0-r55184/test/ruby/enc/test_case_comprehensive.rb:25:in `readlines': No such file or directory @ rb_sysopen - /builddir/build/BUILD/ruby-2.4.0-r55184/enc/unicode/data/8.0.0/UnicodeData.txt (Errno::ENOENT)

This is surfacing a problem that we knew about already for some time.

The various files in enc/unicode/data/ are needed to build some files that get committed and compiled, and for various tests. Earlier, we included the downloading of the files in the build process. This led to problems in CI because gperf is not available on all CI machines, but needed to build some of the files.

So I propose the following:
- Make sure that the necessary Unicode data files always get downloaded (unless maybe there is a network problem).
- Make execution of the build steps that need (a specific version of) gperf dependent on the availability of gperf.
- Always execute the tests.

I'll try to implement some of the above. But I don't have any confidence I can check for the ability of gperf, so I'm assigning this bug to Nobu.

[duerst (Martin Dürst)]

• Related to Bug #12007: Newly added Unicode data file doesn't get downloaded added

[duerst (Martin Dürst)]

In r55188, I have changed the testing so that it doesn't produce an exception anymore, but just fails a test. However, this doesn't close this bug.

[vo.x (Vit Ondruch)]

I experiencing two issues with preview1:

  1) Failure:
TestComprehensiveCaseFold#test_AAAAA_data_files_available [/builddir/build/BUILD/ruby-2.4.0-preview1/test/ruby/enc/test_case_comprehensive.rb:223]:
File /builddir/build/BUILD/ruby-2.4.0-preview1/enc/unicode/data/8.0.0/UnicodeData.txt missing.

  2) Failure:
TestUnicodeNormalize#test_AAAAA_data_file_available [/builddir/build/BUILD/ruby-2.4.0-preview1/test/test_unicode_normalize.rb:142]:
File /builddir/build/BUILD/ruby-2.4.0-preview1/enc/unicode/data/8.0.0/NormalizationTest.txt missing.

I assume they are of the same nature ...

[duerst (Martin Dürst)]

Yes, they show that the Unicode data files didn't get downloaded, and therefore, it's not possible to run the tests in test_case_comprehensive.rb and test_unicode_normalize.rb.

[nobu (Nobuyoshi Nakada)]

• Status changed from Open to Closed

Applied in changeset r55521.

---

Skip unavailable tests

• test/ruby/enc/test_case_comprehensive.rb: noting to test if
  Unicode data files are available.
  [ruby-core:76160] [Bug #12433]
• test/test_unicode_normalize.rb: ditto.

[usa (Usaku NAKAMURA)]

• Backport changed from 2.1: UNKNOWN, 2.2: UNKNOWN, 2.3: UNKNOWN to 2.1: DONTNEED, 2.2: DONTNEED, 2.3: DONTNEED