Bug #15390: Error installing nokogiri on ruby-2.6.0-rc1

• Status: Third Party's Issue
• Assignee: -
• Target version: -
• ruby -v: ruby 2.6.0rc2 (2018-12-15 trunk 66408) [x86_64-darwin18]
• Backport: 2.4: UNKNOWN, 2.5: UNKNOWN

[ned (Ned Hadzo)]

I am having issues with installing nokogiri on my Mac, Mojave 10.14.1 (18B75).

nedims-MacBook-Pro:~ nedim$ rvm use ruby-2.6.0-rc1
Using /Users/nedim/.rvm/gems/ruby-2.6.0-rc1
nedims-MacBook-Pro:~ nedim$ gem install nokogiri
Building native extensions. This could take a while...
ERROR:  Error installing nokogiri:
	ERROR: Failed to build gem native extension.

    current directory: /Users/nedim/.rvm/gems/ruby-2.6.0-rc1/gems/nokogiri-1.8.5/ext/nokogiri
/Users/nedim/.rvm/rubies/ruby-2.6.0-rc1/bin/ruby -I /Users/nedim/.rvm/rubies/ruby-2.6.0-rc1/lib/ruby/2.6.0 -r ./siteconf20181207-15716-1b00x63.rb extconf.rb
checking if the C compiler accepts ... yes
checking if the C compiler accepts -Wno-error=unused-command-line-argument-hard-error-in-future... no
Building nokogiri using packaged libraries.
Using mini_portile version 2.3.0
checking for iconv.h... yes
checking for gzdopen() in -lz... yes
sh: CC_WRAPPER: command not found
checking for iconv using --with-opt-* flags... yes
************************************************************************
IMPORTANT NOTICE:

Building Nokogiri with a packaged version of libxml2-2.9.8
with the following patches applied:
	- 0001-Revert-Do-not-URI-escape-in-server-side-includes.patch
	- 0002-Fix-nullptr-deref-with-XPath-logic-ops.patch
	- 0003-Fix-infinite-loop-in-LZMA-decompression.patch

Team Nokogiri will keep on doing their best to provide security
updates in a timely manner, but if this is a concern for you and want
to use the system library instead; abort this installation process and
reinstall nokogiri as follows:

    gem install nokogiri -- --use-system-libraries
        [--with-xml2-config=/path/to/xml2-config]
        [--with-xslt-config=/path/to/xslt-config]

If you are using Bundler, tell it to use the option:

    bundle config build.nokogiri --use-system-libraries
    bundle install

Note, however, that nokogiri is not fully compatible with arbitrary
versions of libxml2 provided by OS/package vendors.
************************************************************************
Extracting libxml2-2.9.8.tar.gz into tmp/x86_64-apple-darwin18.2.0/ports/libxml2/2.9.8... OK
Running git apply with /Users/nedim/.rvm/gems/ruby-2.6.0-rc1/gems/nokogiri-1.8.5/patches/libxml2/0001-Revert-Do-not-URI-escape-in-server-side-includes.patch... OK
Running git apply with /Users/nedim/.rvm/gems/ruby-2.6.0-rc1/gems/nokogiri-1.8.5/patches/libxml2/0002-Fix-nullptr-deref-with-XPath-logic-ops.patch... OK
Running git apply with /Users/nedim/.rvm/gems/ruby-2.6.0-rc1/gems/nokogiri-1.8.5/patches/libxml2/0003-Fix-infinite-loop-in-LZMA-decompression.patch... OK
Running 'configure' for libxml2 2.9.8... ERROR, review '/Users/nedim/.rvm/gems/ruby-2.6.0-rc1/gems/nokogiri-1.8.5/ext/nokogiri/tmp/x86_64-apple-darwin18.2.0/ports/libxml2/2.9.8/configure.log' to see what happened. Last lines are:
========================================================================
checking whether to enable maintainer-specific portions of Makefiles... yes
checking build system type... x86_64-apple-darwin18.2.0
checking host system type... x86_64-apple-darwin18.2.0
checking for a BSD-compatible install... /usr/local/bin/ginstall -c
checking whether build environment is sane... yes
checking for x86_64-apple-darwin18.2.0-strip... no
checking for strip... strip
checking for a thread-safe mkdir -p... /usr/local/bin/gmkdir -p
checking for gawk... no
checking for mawk... no
checking for nawk... no
checking for awk... awk
checking whether make sets $(MAKE)... yes
checking whether make supports nested variables... yes
checking whether make supports nested variables... (cached) yes
checking for x86_64-apple-darwin18.2.0-gcc... $(CC_WRAPPER) gcc
checking whether the C compiler works... no
configure: error: in `/Users/nedim/.rvm/gems/ruby-2.6.0-rc1/gems/nokogiri-1.8.5/ext/nokogiri/tmp/x86_64-apple-darwin18.2.0/ports/libxml2/2.9.8/libxml2-2.9.8':
configure: error: C compiler cannot create executables
See `config.log' for more details
========================================================================
*** extconf.rb failed ***
Could not create Makefile due to some reason, probably lack of necessary
libraries and/or headers.  Check the mkmf.log file for more details.  You may
need configuration options.

Provided configuration options:
	--with-opt-dir
	--with-opt-include
	--without-opt-include=${opt-dir}/include
	--with-opt-lib
	--without-opt-lib=${opt-dir}/lib
	--with-make-prog
	--without-make-prog
	--srcdir=.
	--curdir
	--ruby=/Users/nedim/.rvm/rubies/ruby-2.6.0-rc1/bin/$(RUBY_BASE_NAME)
	--help
	--clean
	--use-system-libraries
	--enable-static
	--disable-static
	--with-zlib-dir
	--without-zlib-dir
	--with-zlib-include
	--without-zlib-include=${zlib-dir}/include
	--with-zlib-lib
	--without-zlib-lib=${zlib-dir}/lib
	--enable-cross-build
	--disable-cross-build
/Users/nedim/.rvm/gems/ruby-2.6.0-rc1/gems/mini_portile2-2.3.0/lib/mini_portile2/mini_portile.rb:402:in `block in execute': Failed to complete configure task (RuntimeError)
	from /Users/nedim/.rvm/gems/ruby-2.6.0-rc1/gems/mini_portile2-2.3.0/lib/mini_portile2/mini_portile.rb:373:in `chdir'
	from /Users/nedim/.rvm/gems/ruby-2.6.0-rc1/gems/mini_portile2-2.3.0/lib/mini_portile2/mini_portile.rb:373:in `execute'
	from /Users/nedim/.rvm/gems/ruby-2.6.0-rc1/gems/mini_portile2-2.3.0/lib/mini_portile2/mini_portile.rb:110:in `configure'
	from /Users/nedim/.rvm/gems/ruby-2.6.0-rc1/gems/mini_portile2-2.3.0/lib/mini_portile2/mini_portile.rb:153:in `cook'
	from extconf.rb:365:in `block (2 levels) in process_recipe'
	from extconf.rb:257:in `block in chdir_for_build'
	from extconf.rb:256:in `chdir'
	from extconf.rb:256:in `chdir_for_build'
	from extconf.rb:364:in `block in process_recipe'
	from extconf.rb:262:in `tap'
	from extconf.rb:262:in `process_recipe'
	from extconf.rb:551:in `<main>'

To see why this extension failed to compile, please check the mkmf.log which can be found here:

  /Users/nedim/.rvm/gems/ruby-2.6.0-rc1/extensions/x86_64-darwin-18/2.6.0/nokogiri-1.8.5/mkmf.log

extconf failed, exit code 1

Gem files will remain installed in /Users/nedim/.rvm/gems/ruby-2.6.0-rc1/gems/nokogiri-1.8.5 for inspection.
Results logged to /Users/nedim/.rvm/gems/ruby-2.6.0-rc1/extensions/x86_64-darwin-18/2.6.0/nokogiri-1.8.5/gem_make.out
nedims-MacBook-Pro:~ nedim$ 

[hsbt (Hiroshi SHIBATA)]

• Status changed from Open to Feedback
• Assignee set to hsbt (Hiroshi SHIBATA)

I can reproduce this. But I can install nokogiri with brew install libxml2 libxslt and gem install nokogiri -- --use-system-libraries=true.

Can you try it?

[nobu (Nobuyoshi Nakada)]

• Status changed from Feedback to Third Party's Issue
• Assignee deleted (hsbt (Hiroshi SHIBATA))

See https://github.com/sparklemotion/nokogiri/pull/1820

[ned (Ned Hadzo)]

hsbt (Hiroshi SHIBATA) wrote:

I can reproduce this. But I can install nokogiri with brew install libxml2 libxslt and gem install nokogiri -- --use-system-libraries=true.

Can you try it?

It works. Thank you very much. I can run Rails on 2.6 now :)

[ctm (Cliff Matthews)]

I absolutely hate to waste anyone's time with speculation, but this may not be a problem with nokogiri, per-se. I mention this based on having spent a little time tracking this one down on my own and coming to partial conclusions when it showed up in 2.6.0-preview3.

The issue appears to be with CC_WRAPPER, which wasn't present in preview1 or preview2, both of which compile nokogiri fine out of the box. At one point when I was tracking the problem down in preview3, I decided to look at recent commits to see if anything since preview3 had updated the CC_WRAPPER code, and I found:

commit 2b894084eff60a010b0f606c5c9731094358b041
Author: nobu <nobu@b2dd03c8-39d4-4d8f-98ff-823fe69b080e>
Date:   Sat Nov 17 01:42:30 2018 +0000

    test/mkmf/base.rb: use CC_WRAPPER same as the toplevel

so I thought to myself "Oh, nobu has run into the same thing and fixed it." I didn't bother actually downloading the the source with that fix in it and testing, although in retrospect, I should have.

Today I decided to check out rc1 and I found that nokogiri w/o using standard libs still doesn't compile and there's still a "sh: CC_WRAPPER: command not found" complaint, so I did some web searching and found this ticket.

I'm adding this comment now, because I have a bit of a chest cold and am already behind on several other things so I don't know that I'll make any more progress, especially since I'm not particularly familiar with the intricacies of autotools and mkmf. As such, there's a chance that this comment is premature and will waste other people's time, but perhaps it will catch the attention of someone who knows this stuff inside and out who can quickly see if there's really an issue.

So, for me, using macOS Mojave (10.14.1) and XCode 10.1, rbenv 1.1.1 and ruby-build stable 20181207 I can install nokogiri with the following command on 2.6.0-preview1, 2.6.0-preview2 and have it fail on 2.6.0-preview3 and 2.6.0-rc1:

gem install nokogiri -v '1.8.5' --source 'https://rubygems.org/'

If I search for the string CC_WRAPPER I only find it present in preview3 and rc1:

bash-3.2$ find ~/.rbenv/versions/2.6.0-* -type f -print0 | xargs -0 grep -l CC_WRAPPER | grep -v /gems/
/Users/newctm/.rbenv/versions/2.6.0-preview3/lib/ruby/2.6.0/x86_64-darwin18/rbconfig.rb
/Users/newctm/.rbenv/versions/2.6.0-rc1/lib/ruby/2.6.0/mkmf.rb
/Users/newctm/.rbenv/versions/2.6.0-rc1/lib/ruby/2.6.0/x86_64-darwin18/rbconfig.rb

I hope this info is useful and not a red herring.

[ctm (Cliff Matthews)]

For what it's worth, if I comment out a line in configure.ac, I can still build and install ruby locally and the version I install can then build and install nokogiri. However, my guess is that the altered code wouldn't build properly on some other version of macOS and/or XCode. IOW, I'm in way over my head, but this does strongly suggest there's room for improvement:

bash-3.2[trunk]$ git diff
diff --git a/configure.ac b/configure.ac
index 4e3f1ef0e5..fb4917bca8 100644
--- a/configure.ac
+++ b/configure.ac
@@ -3915,7 +3915,7 @@ AC_SUBST(PACKAGE)
 AS_MESSAGE([$PACKAGE library version = $ruby_version])
 
 AS_IF([test x"$CC_WRAPPER" != x], [
-    CC='$(CC_WRAPPER) '"${CC@%:@$CC_WRAPPER }"
+    # CC='$(CC_WRAPPER) '"${CC@%:@$CC_WRAPPER }"
     CPP='$(CC_WRAPPER) '"${CPP@%:@$CC_WRAPPER }"
     CC_WRAPPER='$(rubyarchdir)/darwin-cc'
     XCC_WRAPPER='$(top_srcdir)/tool/darwin-cc'

[ctm (Cliff Matthews)]

• ruby -v changed from 2.6.0rc1 (2018-12-06 trunk 66253) [x86_64-darwin18] to ruby 2.6.0rc2 (2018-12-15 trunk 66408) [x86_64-darwin18]

I have confirmed that this bug is still present in rc2.

I have not changed the Status from "Third Party's Issue" to "Open", but unless I'm overlooking something, I do think that would be appropriate.

I'm hoping that someone who is familiar with the CC_WRAPPER code can double-check and see if a fix would be appropriate for 2.6.0.

[nobu (Nobuyoshi Nakada)]

Try nokogiri v1.9.0.rc1.
https://github.com/sparklemotion/nokogiri/issues/1833#issuecomment-445699368

[ctm (Cliff Matthews)]

Thank you Nobuyoshi!

As you know, I indeed was overlooking something, specifically your comment and link on note #2 when you correctly changed the Status to "Third Party's Issue." I saw the workaround in note #1 and the confirmation that the workaround worked in note #3 and conflated the workaround with the actual work you had already done (that I overlooked) that not only diagnosed the actual problem, but supplied a PR.

I apologize for wasting your (and anyone else who read my comments) time.

[nobu (Nobuyoshi Nakada)]

Don't mind, one-line comment may be overlooked easily.
I'll paste the content not only a link, next time.

[franklinyu (Franklin Yu)]

Possibly related: https://github.com/Homebrew/brew/issues/5068

Also this section from Nokogiri documentation: https://nokogiri.org/tutorials/installing_nokogiri.html#unable-to-find-libraries-on-macos-mojave