Backport #7959

configure contains non-portable shell code

Added by Jeremy Evans about 2 years ago. Updated over 1 year ago.

[ruby-core:52893]
Status:Assigned
Priority:Normal
Assignee:Tomoyuki Chikanaga

Description

On 2.0.0 and trunk, configure contains non-portable shell code. When run on OpenBSD, configure emits the following errors and results in a ruby that builds but cannot run without manually specifying -I for the ruby lib directories:

...
checking for prefix of external symbols... NONE
checking for pthread.h... (cached) yes
checking if make is GNU make... no
checking for safe null command for make... :
checking for memmem... no
sed: 1: ""s/${[${}"]*}/"&"/g"": invalid command code "
sed: 1: ""s/${[${}"]*}/"&"/g"": invalid command code "
sed: 1: ""s/${[${}"]*}/"&"/g"": invalid command code "
sed: 1: ""s/${[${}"]*}/"&"/g;s/ ...: invalid command code "
././configure: s/\//g: not found
././configure: s/\//;s/\$//": not found
sed: 1: ""s/${[${}"]*}/"&"/g"": invalid command code "
sed: 1: ""s/${[${}"]*}/"&"/g"": invalid command code "
sed: 1: ""s/${[${}"]*}/"&"/g"": invalid command code "
sed: 1: ""s/${[${}"]*}/"&"/g"": invalid command code "
sed: 1: ""s/${[${}"]*}/"&"/g"": invalid command code "
sed: 1: ""s/${[${}"]*}/"&"/g"": invalid command code "
sed: 1: ""s/${[${}"]*}/"&"/g;s/ ...: invalid command code "
././configure: s/\//g: not found
././configure: s/\//;s/\$//": not found
sed: 1: ""s/${[${}"]*}/"&"/g"": invalid command code "
sed: 1: ""s/${[${}"]*}/"&"/g"": invalid command code "
sed: 1: ""s/${[${}"]*}/"&"/g"": invalid command code "
sed: 1: ""s/${[${}"]*}/"&"/g"": invalid command code "
sed: 1: ""s/${[${}"]*}/"&"/g"": invalid command code "
sed: 1: ""s/${[${}"]*}/"&"/g"": invalid command code "
sed: 1: ""s/${[${}"]*}/"&"/g"": invalid command code "
sed: 1: ""s/${[${}"]*}/"&"/g"": invalid command code "
sed: 1: ""s/${[${}"]*}/"&"/g;s/ ...: invalid command code "
././configure: s/\//g: not found
././configure: s/\//;s/\$//": not found
sed: 1: ""s/${[${}"]*}/"&"/g"": invalid command code "
sed: 1: ""s/${[${}"]*}/"&"/g;s/ ...: invalid command code "
././configure: s/\//g: not found
././configure: s/\//;s/\$//": not found
checking for nroff... no
checking for awf... no
.ext/include/x86_64-openbsd/ruby/config.h updated
verconf.h updated
ruby library version = 2.0
configure: creating ./config.status
config.status: creating Makefile
config.status: creating ruby-2.0.pc

The last revision that worked correctly was 39288.

It's trivial to work around the issue by using "bash configure" instead of "./configure", but that shouldn't be necessary. I'm sorry for not reporting this before the official release of 2.0.0, but this issue was introduced after rc2.


Related issues

Related to Backport200 - Backport #8071: Ruby 2.0 ./configure fails with syntax error on Illumos d... Closed 03/11/2013

Associated revisions

Revision 39506
Added by Nobuyoshi Nakada about 2 years ago

configure.in: non-portable shell behavior

  • configure.in (unexpand_shvar): get rid of non-portable shell behavior on OpenBSD, so no extra quotes. [Bug #7959]

Revision 39514
Added by Nobuyoshi Nakada about 2 years ago

configure.in: remove no longer used variable

  • configure.in (unexpand_shvar): remove variable which is no longer used since r39506. [Bug #7959]

Revision 39525
Added by Nobuyoshi Nakada about 2 years ago

configure.in: escape double-quotes

  • configure.in (unexpand_shvar): escape double-quotes in backquotes in double-quotes for some shells. [Bug #7959]

Revision 39709
Added by Tomoyuki Chikanaga almost 2 years ago

merge revision(s) 39506,39514: [Backport #7959]

* configure.in (unexpand_shvar): get rid of non-portable shell
  behavior on OpenBSD, so no extra quotes.  [Bug #7959]

Revision 39741
Added by Nobuyoshi Nakada almost 2 years ago

configure.in: fix for old shells

  • configure.in (rm, shvar_to_cpp, unexpand_shvar): local is not available on old shells.
  • configure.in (shvar_to_cpp): escape quotes for old shells. [Bug #7959] [Bug #8071]

Revision 39825
Added by Tomoyuki Chikanaga almost 2 years ago

merge revision(s) 39525: [Backport #8102]

configure.in: escape double-quotes

* configure.in (unexpand_shvar): escape double-quotes in backquotes in

double-quotes for some shells. [Bug #7959]

Revision 40647
Added by Tomoyuki Chikanaga almost 2 years ago

merge revision(s) 39636,39741,40454: [Backport #8071] [Backport #8387]

configure.in: suppress error messages

* configure.in: suppress error messages, rm: cannot remove

'conftest.dSYM', from AC_EGREP_CPP with CFLAGS=-g on Darwin.
* configure.in (rm, shvar_to_cpp, unexpand_shvar): local is not
available on old shells.

* configure.in (shvar_to_cpp): escape quotes for old shells.
  [Bug #7959] [Bug #8071]

* configure.in (RUBY_RM_RECURSIVE): this hack is needed by only
  autoconf 2.69 or earlier on darwin.

Revision 40758
Added by Nobuyoshi Nakada almost 2 years ago

configure.in: shvar_to_cpp in ruby

  • configure.in: save configured load path values into verconf.in.
  • common.mk (verconf.h): create from verconf.in with shvar_to_cpp.rb.
  • tool/shvar_to_cpp.rb: turn shell variables into C macros. [Bug #7959]

History

#1 Updated by Nobuyoshi Nakada about 2 years ago

  • Status changed from Open to Closed
  • % Done changed from 0 to 100

This issue was solved with changeset r39506.
Jeremy, thank you for reporting this issue.
Your contribution to Ruby is greatly appreciated.
May Ruby be with you.


configure.in: non-portable shell behavior

  • configure.in (unexpand_shvar): get rid of non-portable shell behavior on OpenBSD, so no extra quotes. [Bug #7959]

#2 Updated by Nobuyoshi Nakada about 2 years ago

  • Tracker changed from Bug to Backport
  • Project changed from Ruby trunk to Backport200
  • Category deleted (build)
  • Status changed from Closed to Assigned
  • Assignee changed from Nobuyoshi Nakada to Tomoyuki Chikanaga

#3 Updated by Tomoyuki Chikanaga about 2 years ago

nobu san

Should the following line in configure.in be eliminated?

l.3287 quote=['']

#4 Updated by Nobuyoshi Nakada about 2 years ago

Yes, it should not be there anymore, I missed to remove it.

#5 Updated by Tomoyuki Chikanaga almost 2 years ago

  • Status changed from Assigned to Closed

This issue was solved with changeset r39709.
Jeremy, thank you for reporting this issue.
Your contribution to Ruby is greatly appreciated.
May Ruby be with you.


merge revision(s) 39506,39514: [Backport #7959]

* configure.in (unexpand_shvar): get rid of non-portable shell
  behavior on OpenBSD, so no extra quotes.  [Bug #7959]

#6 Updated by George Koehler almost 2 years ago

=begin
Hello, Rubyists. This fix is not working for me. I am still getting this bug, or a very similar bug.

I am running OpenBSD 5.2 and autoconf 2.69. I follow trunk and ruby_2_0_0 with git. Here are my ruby -v
* ruby 2.1.0dev (2013-03-13 trunk 39737) [x86_64-openbsd5.2]
* ruby 2.0.0p57 (2013-03-12 revision 39728) [x86_64-openbsd5.2]

When I configure trunk, I see errors about "no closing quote".

$ ../configure --prefix=$HOME/prefix \
--with-baseruby=/usr/local/bin/ruby19 \
CPPFLAGS=-I/usr/local/include LDFLAGS=-L/usr/local/lib
...
checking if make is GNU make... no
checking for safe null command for make... :
checking for memmem... no
../configure: no closing quote
../configure: no closing quote
../configure: no closing quote
../configure: no closing quote
checking for nroff... no
checking for awf... no
.ext/include/x86_64-openbsd5.2/ruby/config.h unchanged
verconf.h unchanged
ruby library version = 2.1.0
configure: creating ./config.status
config.status: creating Makefile
config.status: creating ruby-2.1.pc

Then verconf.h looks like a mess:

#define RUBY_BASE_NAME "ruby"
#define RUBY_VERSION_NAME RUBY_BASE_NAME"-"RUBY_LIB_VERSION
#define RUBY_LIB_VERSION_STYLE 3 /* full */
#define RUBY_EXEC_PREFIX "/home/kernigh/prefix"
#define RUBY_LIB_PREFIX "arch"RUBY_EXEC_PREFIX"/lib/"RUBY_BASE_NAME"arch"
#define RUBY_ARCH_PREFIX_FOR(arch) "arch"RUBY_LIB_PREFIX"/"arch"arch"
#define RUBY_SITEARCH_PREFIX_FOR(arch) "arch"RUBY_LIB_PREFIX"/"arch"arch"
#define RUBY_LIB "arch"RUBY_LIB_PREFIX"/"RUBY_LIB_VERSION"arch"
#define RUBY_ARCH_LIB_FOR(arch) "arch"RUBY_LIB"/"arch"arch"
#define RUBY_SITE_LIB "arch"RUBY_LIB_PREFIX"/site_ruby"
#define RUBY_SITE_ARCH_LIB_FOR(arch) "arch"RUBY_SITE_LIB2"/"arch"arch"
#define RUBY_VENDOR_LIB "arch"RUBY_LIB_PREFIX"/vendor_ruby"
#define RUBY_VENDOR_ARCH_LIB_FOR(arch) "arch"RUBY_VENDOR_LIB2"/"arch"arch"
#define RUBY_PLATFORM "x86_64-openbsd5.2"

This ruby can't find the installed libraries:

$ ./ruby -v
ruby 2.1.0dev (2013-03-12 trunk 39730) [x86_64-openbsd5.2]
$ ./ruby -e 'p $:'
internal:gem_prelude:1:in require': cannot load such file -- rubygems.rb (LoadError)
from <internal:gem_prelude>:1:in
'
$ ./ruby --disable=gems -e 'p $:'
["archarch/home/kernigh/prefix/lib/rubyarch/site_ruby/2.1.0", "archarcharch/home/kernigh/prefix/lib/rubyarch/site_ruby/2.1.0/x86_64-openbsd5.2arch", "archarch/home/kernigh/prefix/lib/rubyarch/site_ruby", "archarch/home/kernigh/prefix/lib/rubyarch/vendor_ruby/2.1.0", "archarcharch/home/kernigh/prefix/lib/rubyarch/vendor_ruby/2.1.0/x86_64-openbsd5.2arch", "archarch/home/kernigh/prefix/lib/rubyarch/vendor_ruby", "arch/home/kernigh/prefix/lib/rubyarch/2.1.0", "archarch/home/kernigh/prefix/lib/rubyarch/2.1.0/x86_64-openbsd5.2arch"]

When I configure ruby_2_0_0, the problem is slightly different:

$ ../configure --prefix=$HOME/prefix \
--with-baseruby=/usr/local/bin/ruby19 \
CPPFLAGS=-I/usr/local/include LDFLAGS=-L/usr/local/lib
...
checking if make is GNU make... no
checking for safe null command for make... :
checking for memmem... no
sed: 1: ""s/${[${}"]*}/"&"/g;;s ...: invalid command code "
../configure: s/\//g: not found
../configure: s/\//;s/\$//": not found
sed: 1: ""s/${[${}"]*}/"&"/g;;s ...: invalid command code "
../configure: s/\//g: not found
../configure: s/\//;s/\$//": not found
sed: 1: ""s/${[${}"]*}/"&"/g;;s ...: invalid command code "
../configure: s/\//g: not found
../configure: s/\//;s/\$//": not found
sed: 1: ""s/${[${}"]*}/"&"/g;;s ...: invalid command code "
../configure: s/\//g: not found
../configure: s/\//;s/\$//": not found
checking for nroff... no
checking for awf... no
.ext/include/x86_64-openbsd5.2/ruby/config.h unchanged
verconf.h unchanged
ruby library version = 2.0.0
configure: creating ./config.status
config.status: creating Makefile
config.status: creating ruby-2.0.pc

#define RUBY_BASE_NAME "ruby"
#define RUBY_VERSION_NAME RUBY_BASE_NAME"-"RUBY_LIB_VERSION
#define RUBY_LIB_VERSION_STYLE 3 /* full */
#define RUBY_LIB_PREFIX "arch"
#define RUBY_ARCH_PREFIX_FOR(arch) "arch"
#define RUBY_SITEARCH_PREFIX_FOR(arch) "arch"
#define RUBY_LIB "arch"RUBY_LIB_PREFIX"/"RUBY_LIB_VERSION"arch"
#define RUBY_ARCH_LIB_FOR(arch) "arch"RUBY_LIB"/"arch"arch"
#define RUBY_SITE_LIB "arch"RUBY_LIB_PREFIX"/site_ruby"
#define RUBY_SITE_ARCH_LIB_FOR(arch) "arch"RUBY_SITE_LIB2"/"arch"arch"
#define RUBY_VENDOR_LIB "arch"RUBY_LIB_PREFIX"/vendor_ruby"
#define RUBY_VENDOR_ARCH_LIB_FOR(arch) "arch"RUBY_VENDOR_LIB2"/"arch"arch"
#define RUBY_PLATFORM "x86_64-openbsd5.2"

$ ./ruby -v
ruby 2.0.0p57 (2013-03-12 revision 39728) [x86_64-openbsd5.2]
$ ./ruby -e 'p $:'
internal:gem_prelude:1:in require': cannot load such file -- rubygems.rb (LoadError)
from <internal:gem_prelude>:1:in
'
$ ./ruby --disable=gems -e 'p $:'
["archarch/site_ruby/2.0.0", "archarcharch/site_ruby/2.0.0/x86_64-openbsd5.2arch", "archarch/site_ruby", "archarch/vendor_ruby/2.0.0", "archarcharch/vendor_ruby/2.0.0/x86_64-openbsd5.2arch", "archarch/vendor_ruby", "arch/2.0.0", "archarch/2.0.0/x86_64-openbsd5.2arch"]

Like jeremyevans0, I can work around this issue by using bash. I run "bash ../configure ..." and rebuild trunk Ruby, then trunk Ruby no longer has this problem.

I don't know where in configure causes this problem. I am guessing that the problem might be in shvar_to_cpp, but I am not sure. OpenBSD /bin/sh is a modified pdksh. To try to reproduce this problem with another system, one might try using pdksh. Also, bug #8071 reported a similar problem with ksh93, so one might try using ksh93.

=end

#7 Updated by Jeremy Evans almost 2 years ago

It appears this bug is still present in ruby 2.0.0-p195 and trunk. Bisecting v2_0_0_rc2 and trunk, the last revision that worked correctly is still 39288. Can this bug please be reopened? I'll be happy to test patches.

ruby 2.0.0-p195 on OpenBSD doesn't even build if you configure with /bin/sh and not with bash, due to issues with RUBY_EXEC_PREFIX:

cc -O2 -pipe -fPIC -include ruby/config.h -include ruby/missing.h -D_FORTIFY_SOURCE=2 -fstack-protector -fno-strict-overflow -fvisibility=hidden -DRUBY_EXPORT -DOPENSSL_NO_STATIC_ENGINE -I/usr/local/include -I. -I.ext/include/i386-openbsd -I./include -I. -o ruby.o -c ruby.c
ruby.c: In function 'ruby_init_loadpath_safe':
ruby.c:495: error: expected expression before ';' token
*** Error 1 in /usr/obj/ports/ruby-2.0.0-p195/ruby-2.0.0-p195 (Makefile:326 'ruby.o')

#8 Updated by Luis Lavena almost 2 years ago

  • Status changed from Closed to Assigned
  • Assignee changed from Tomoyuki Chikanaga to Nobuyoshi Nakada

#9 Updated by Luis Lavena almost 2 years ago

  • % Done changed from 100 to 50

#10 Updated by Yui NARUSE almost 2 years ago

  • Tracker changed from Backport to Bug
  • Project changed from Backport200 to Ruby trunk

#11 Updated by Nobuyoshi Nakada almost 2 years ago

  • Status changed from Assigned to Closed
  • % Done changed from 50 to 100

This issue was solved with changeset r40758.
Jeremy, thank you for reporting this issue.
Your contribution to Ruby is greatly appreciated.
May Ruby be with you.


configure.in: shvar_to_cpp in ruby

  • configure.in: save configured load path values into verconf.in.
  • common.mk (verconf.h): create from verconf.in with shvar_to_cpp.rb.
  • tool/shvar_to_cpp.rb: turn shell variables into C macros. [Bug #7959]

#12 Updated by Jeremy Evans almost 2 years ago

I tested trunk and ./configure appears to work correctly on OpenBSD (no more error messages). Can these changes please be backported to 2.0.0?

#13 Updated by Tomoyuki Chikanaga almost 2 years ago

  • Tracker changed from Bug to Backport
  • Project changed from Ruby trunk to Backport200
  • Status changed from Closed to Assigned
  • Assignee changed from Nobuyoshi Nakada to Tomoyuki Chikanaga

#14 Updated by Michal Papis over 1 year ago

still getting errors on ruby_2_0_0, any progress on fixing it?

#15 Updated by Tomoyuki Chikanaga over 1 year ago

Backporting r40758 to ruby_2_0_0 cause severe conflict. It seems depend on several changesets in trunk include splitting loadpath.c and strategy of creating verconf.h.

nobu, could you make a backport patch? Or is there any other approach to fix this issue on 2.0.0?

#16 Updated by Nobuyoshi Nakada over 1 year ago

r39171,r39296,r39333,r39334,r39345,r39347,r39348,r40261,r40756,r40757,r40758,r40762,r40763,r40770,r40771,r40777,r40779,r40784,r41352,r41357

#17 Updated by Michal Papis over 1 year ago

=begin
any chance to get this merged somehow? I get merge conflicts trying to apply some of the revisions on 2.0.0-p247 source
=end

Also available in: Atom PDF