Backport #7959

configure contains non-portable shell code

Added by Jeremy Evans about 1 year ago. Updated 7 months ago.

[ruby-core:52893]
Status:Assigned
Priority:Normal
Assignee:Tomoyuki Chikanaga

Description

On 2.0.0 and trunk, configure contains non-portable shell code. When run on OpenBSD, configure emits the following errors and results in a ruby that builds but cannot run without manually specifying -I for the ruby lib directories:

...
checking for prefix of external symbols... NONE
checking for pthread.h... (cached) yes
checking if make is GNU make... no
checking for safe null command for make... :
checking for memmem... no
sed: 1: ""s/${[${}"]*}/"&"/g"": invalid command code "
sed: 1: ""s/${[${}"]*}/"&"/g"": invalid command code "
sed: 1: ""s/${[${}"]*}/"&"/g"": invalid command code "
sed: 1: ""s/${[${}"]*}/"&"/g;s/ ...: invalid command code "
././configure: s/\//g: not found
././configure: s/\//;s/\$//": not found
sed: 1: ""s/${[${}"]*}/"&"/g"": invalid command code "
sed: 1: ""s/${[${}"]*}/"&"/g"": invalid command code "
sed: 1: ""s/${[${}"]*}/"&"/g"": invalid command code "
sed: 1: ""s/${[${}"]*}/"&"/g"": invalid command code "
sed: 1: ""s/${[${}"]*}/"&"/g"": invalid command code "
sed: 1: ""s/${[${}"]*}/"&"/g"": invalid command code "
sed: 1: ""s/${[${}"]*}/"&"/g;s/ ...: invalid command code "
././configure: s/\//g: not found
././configure: s/\//;s/\$//": not found
sed: 1: ""s/${[${}"]*}/"&"/g"": invalid command code "
sed: 1: ""s/${[${}"]*}/"&"/g"": invalid command code "
sed: 1: ""s/${[${}"]*}/"&"/g"": invalid command code "
sed: 1: ""s/${[${}"]*}/"&"/g"": invalid command code "
sed: 1: ""s/${[${}"]*}/"&"/g"": invalid command code "
sed: 1: ""s/${[${}"]*}/"&"/g"": invalid command code "
sed: 1: ""s/${[${}"]*}/"&"/g"": invalid command code "
sed: 1: ""s/${[${}"]*}/"&"/g"": invalid command code "
sed: 1: ""s/${[${}"]*}/"&"/g;s/ ...: invalid command code "
././configure: s/\//g: not found
././configure: s/\//;s/\$//": not found
sed: 1: ""s/${[${}"]*}/"&"/g"": invalid command code "
sed: 1: ""s/${[${}"]*}/"&"/g;s/ ...: invalid command code "
././configure: s/\//g: not found
././configure: s/\//;s/\$//": not found
checking for nroff... no
checking for awf... no
.ext/include/x86_64-openbsd/ruby/config.h updated
verconf.h updated
ruby library version = 2.0
configure: creating ./config.status
config.status: creating Makefile
config.status: creating ruby-2.0.pc

The last revision that worked correctly was 39288.

It's trivial to work around the issue by using "bash configure" instead of "./configure", but that shouldn't be necessary. I'm sorry for not reporting this before the official release of 2.0.0, but this issue was introduced after rc2.


Related issues

Related to Backport200 - Backport #8071: Ruby 2.0 ./configure fails with syntax error on Illumos d... Closed 03/11/2013

Associated revisions

Revision 39709
Added by Tomoyuki Chikanaga about 1 year ago

merge revision(s) 39506,39514: [Backport #7959]

* configure.in (unexpand_shvar): get rid of non-portable shell
  behavior on OpenBSD, so no extra quotes.  [Bug #7959]

Revision 39825
Added by Tomoyuki Chikanaga about 1 year ago

merge revision(s) 39525: [Backport #8102]

configure.in: escape double-quotes

* configure.in (unexpand_shvar): escape double-quotes in backquotes in

double-quotes for some shells. [Bug #7959]

Revision 40647
Added by Tomoyuki Chikanaga 12 months ago

merge revision(s) 39636,39741,40454: [Backport #8071] [Backport #8387]

configure.in: suppress error messages

* configure.in: suppress error messages, rm: cannot remove

'conftest.dSYM', from ACEGREPCPP with CFLAGS=-g on Darwin.
* configure.in (rm, shvartocpp, unexpand_shvar): local is not
available on old shells.

* configure.in (shvar_to_cpp): escape quotes for old shells.
  [Bug #7959] [Bug #8071]

* configure.in (RUBY_RM_RECURSIVE): this hack is needed by only
  autoconf 2.69 or earlier on darwin.

History

#1 Updated by Nobuyoshi Nakada about 1 year ago

  • Status changed from Open to Closed
  • % Done changed from 0 to 100

This issue was solved with changeset r39506.
Jeremy, thank you for reporting this issue.
Your contribution to Ruby is greatly appreciated.
May Ruby be with you.


configure.in: non-portable shell behavior

  • configure.in (unexpand_shvar): get rid of non-portable shell behavior on OpenBSD, so no extra quotes. [Bug #7959]

#2 Updated by Nobuyoshi Nakada about 1 year ago

  • Tracker changed from Bug to Backport
  • Project changed from ruby-trunk to Backport200
  • Category deleted (build)
  • Status changed from Closed to Assigned
  • Assignee changed from Nobuyoshi Nakada to Tomoyuki Chikanaga

#3 Updated by Tomoyuki Chikanaga about 1 year ago

nobu san

Should the following line in configure.in be eliminated?

l.3287 quote=['']

#4 Updated by Nobuyoshi Nakada about 1 year ago

Yes, it should not be there anymore, I missed to remove it.

#5 Updated by Tomoyuki Chikanaga about 1 year ago

  • Status changed from Assigned to Closed

This issue was solved with changeset r39709.
Jeremy, thank you for reporting this issue.
Your contribution to Ruby is greatly appreciated.
May Ruby be with you.


merge revision(s) 39506,39514: [Backport #7959]

* configure.in (unexpand_shvar): get rid of non-portable shell
  behavior on OpenBSD, so no extra quotes.  [Bug #7959]

#6 Updated by George Koehler about 1 year ago

=begin
Hello, Rubyists. This fix is not working for me. I am still getting this bug, or a very similar bug.

I am running OpenBSD 5.2 and autoconf 2.69. I follow trunk and ruby200 with git. Here are my ruby -v
* ruby 2.1.0dev (2013-03-13 trunk 39737) [x86
64-openbsd5.2]
* ruby 2.0.0p57 (2013-03-12 revision 39728) [x86_64-openbsd5.2]

When I configure trunk, I see errors about "no closing quote".

$ ../configure --prefix=$HOME/prefix \
--with-baseruby=/usr/local/bin/ruby19 \
CPPFLAGS=-I/usr/local/include LDFLAGS=-L/usr/local/lib
...
checking if make is GNU make... no
checking for safe null command for make... :
checking for memmem... no
../configure: no closing quote
../configure: no closing quote
../configure: no closing quote
../configure: no closing quote
checking for nroff... no
checking for awf... no
.ext/include/x86_64-openbsd5.2/ruby/config.h unchanged
verconf.h unchanged
ruby library version = 2.1.0
configure: creating ./config.status
config.status: creating Makefile
config.status: creating ruby-2.1.pc

Then verconf.h looks like a mess:

#define RUBYBASENAME "ruby"
#define RUBYVERSIONNAME RUBYBASENAME"-"RUBYLIBVERSION
#define RUBYLIBVERSIONSTYLE 3 /* full */
#define RUBY
EXECPREFIX "/home/kernigh/prefix"
#define RUBY
LIBPREFIX "arch"RUBYEXECPREFIX"/lib/"RUBYBASENAME"arch"
#define RUBY
ARCHPREFIXFOR(arch) "arch"RUBYLIBPREFIX"/"arch"arch"
#define RUBYSITEARCHPREFIXFOR(arch) "arch"RUBYLIBPREFIX"/"arch"arch"
#define RUBY
LIB "arch"RUBYLIBPREFIX"/"RUBYLIBVERSION"arch"
#define RUBYARCHLIBFOR(arch) "arch"RUBYLIB"/"arch"arch"
#define RUBYSITELIB "arch"RUBYLIBPREFIX"/siteruby"
#define RUBY
SITEARCHLIBFOR(arch) "arch"RUBYSITELIB2"/"arch"arch"
#define RUBY
VENDORLIB "arch"RUBYLIBPREFIX"/vendorruby"
#define RUBYVENDORARCHLIBFOR(arch) "arch"RUBYVENDORLIB2"/"arch"arch"
#define RUBYPLATFORM "x8664-openbsd5.2"

This ruby can't find the installed libraries:

$ ./ruby -v
ruby 2.1.0dev (2013-03-12 trunk 39730) [x8664-openbsd5.2]
$ ./ruby -e 'p $:'
<internal:gem
prelude>:1:in require': cannot load such file -- rubygems.rb (LoadError)
from <internal:gem_prelude>:1:in
'
$ ./ruby --disable=gems -e 'p $:'
["archarch/home/kernigh/prefix/lib/rubyarch/siteruby/2.1.0", "archarcharch/home/kernigh/prefix/lib/rubyarch/siteruby/2.1.0/x8664-openbsd5.2arch", "archarch/home/kernigh/prefix/lib/rubyarch/siteruby", "archarch/home/kernigh/prefix/lib/rubyarch/vendorruby/2.1.0", "archarcharch/home/kernigh/prefix/lib/rubyarch/vendorruby/2.1.0/x8664-openbsd5.2arch", "archarch/home/kernigh/prefix/lib/rubyarch/vendorruby", "arch/home/kernigh/prefix/lib/rubyarch/2.1.0", "archarch/home/kernigh/prefix/lib/rubyarch/2.1.0/x86_64-openbsd5.2arch"]

When I configure ruby20_0, the problem is slightly different:

$ ../configure --prefix=$HOME/prefix \
--with-baseruby=/usr/local/bin/ruby19 \
CPPFLAGS=-I/usr/local/include LDFLAGS=-L/usr/local/lib
...
checking if make is GNU make... no
checking for safe null command for make... :
checking for memmem... no
sed: 1: ""s/${[${}"]*}/"&"/g;;s ...: invalid command code "
../configure: s/\//g: not found
../configure: s/\//;s/\$//": not found
sed: 1: ""s/${[${}"]*}/"&"/g;;s ...: invalid command code "
../configure: s/\//g: not found
../configure: s/\//;s/\$//": not found
sed: 1: ""s/${[${}"]*}/"&"/g;;s ...: invalid command code "
../configure: s/\//g: not found
../configure: s/\//;s/\$//": not found
sed: 1: ""s/${[${}"]*}/"&"/g;;s ...: invalid command code "
../configure: s/\//g: not found
../configure: s/\//;s/\$//": not found
checking for nroff... no
checking for awf... no
.ext/include/x86_64-openbsd5.2/ruby/config.h unchanged
verconf.h unchanged
ruby library version = 2.0.0
configure: creating ./config.status
config.status: creating Makefile
config.status: creating ruby-2.0.pc

#define RUBYBASENAME "ruby"
#define RUBYVERSIONNAME RUBYBASENAME"-"RUBYLIBVERSION
#define RUBYLIBVERSIONSTYLE 3 /* full */
#define RUBY
LIBPREFIX "arch"
#define RUBY
ARCHPREFIXFOR(arch) "arch"
#define RUBYSITEARCHPREFIXFOR(arch) "arch"
#define RUBY
LIB "arch"RUBYLIBPREFIX"/"RUBYLIBVERSION"arch"
#define RUBYARCHLIBFOR(arch) "arch"RUBYLIB"/"arch"arch"
#define RUBYSITELIB "arch"RUBYLIBPREFIX"/siteruby"
#define RUBY
SITEARCHLIBFOR(arch) "arch"RUBYSITELIB2"/"arch"arch"
#define RUBY
VENDORLIB "arch"RUBYLIBPREFIX"/vendorruby"
#define RUBYVENDORARCHLIBFOR(arch) "arch"RUBYVENDORLIB2"/"arch"arch"
#define RUBYPLATFORM "x8664-openbsd5.2"

$ ./ruby -v
ruby 2.0.0p57 (2013-03-12 revision 39728) [x8664-openbsd5.2]
$ ./ruby -e 'p $:'
<internal:gem
prelude>:1:in require': cannot load such file -- rubygems.rb (LoadError)
from <internal:gem_prelude>:1:in
'
$ ./ruby --disable=gems -e 'p $:'
["archarch/siteruby/2.0.0", "archarcharch/siteruby/2.0.0/x8664-openbsd5.2arch", "archarch/siteruby", "archarch/vendorruby/2.0.0", "archarcharch/vendorruby/2.0.0/x8664-openbsd5.2arch", "archarch/vendorruby", "arch/2.0.0", "archarch/2.0.0/x86_64-openbsd5.2arch"]

Like jeremyevans0, I can work around this issue by using bash. I run "bash ../configure ..." and rebuild trunk Ruby, then trunk Ruby no longer has this problem.

I don't know where in configure causes this problem. I am guessing that the problem might be in shvartocpp, but I am not sure. OpenBSD /bin/sh is a modified pdksh. To try to reproduce this problem with another system, one might try using pdksh. Also, bug #8071 reported a similar problem with ksh93, so one might try using ksh93.

=end

#7 Updated by Jeremy Evans 12 months ago

It appears this bug is still present in ruby 2.0.0-p195 and trunk. Bisecting v200_rc2 and trunk, the last revision that worked correctly is still 39288. Can this bug please be reopened? I'll be happy to test patches.

ruby 2.0.0-p195 on OpenBSD doesn't even build if you configure with /bin/sh and not with bash, due to issues with RUBYEXECPREFIX:

cc -O2 -pipe -fPIC -include ruby/config.h -include ruby/missing.h -DFORTIFYSOURCE=2 -fstack-protector -fno-strict-overflow -fvisibility=hidden -DRUBYEXPORT -DOPENSSLNOSTATICENGINE -I/usr/local/include -I. -I.ext/include/i386-openbsd -I./include -I. -o ruby.o -c ruby.c
ruby.c: In function 'rubyinitloadpath_safe':
ruby.c:495: error: expected expression before ';' token
*** Error 1 in /usr/obj/ports/ruby-2.0.0-p195/ruby-2.0.0-p195 (Makefile:326 'ruby.o')

#8 Updated by Luis Lavena 12 months ago

  • Status changed from Closed to Assigned
  • Assignee changed from Tomoyuki Chikanaga to Nobuyoshi Nakada

#9 Updated by Luis Lavena 12 months ago

  • % Done changed from 100 to 50

#10 Updated by Yui NARUSE 11 months ago

  • Tracker changed from Backport to Bug
  • Project changed from Backport200 to ruby-trunk

#11 Updated by Nobuyoshi Nakada 11 months ago

  • Status changed from Assigned to Closed
  • % Done changed from 50 to 100

This issue was solved with changeset r40758.
Jeremy, thank you for reporting this issue.
Your contribution to Ruby is greatly appreciated.
May Ruby be with you.


configure.in: shvartocpp in ruby

  • configure.in: save configured load path values into verconf.in.
  • common.mk (verconf.h): create from verconf.in with shvartocpp.rb.
  • tool/shvartocpp.rb: turn shell variables into C macros. [Bug #7959]

#12 Updated by Jeremy Evans 11 months ago

I tested trunk and ./configure appears to work correctly on OpenBSD (no more error messages). Can these changes please be backported to 2.0.0?

#13 Updated by Tomoyuki Chikanaga 11 months ago

  • Tracker changed from Bug to Backport
  • Project changed from ruby-trunk to Backport200
  • Status changed from Closed to Assigned
  • Assignee changed from Nobuyoshi Nakada to Tomoyuki Chikanaga

#14 Updated by Michal Papis 11 months ago

still getting errors on ruby20_0, any progress on fixing it?

#15 Updated by Tomoyuki Chikanaga 11 months ago

Backporting r40758 to ruby20_0 cause severe conflict. It seems depend on several changesets in trunk include splitting loadpath.c and strategy of creating verconf.h.

nobu, could you make a backport patch? Or is there any other approach to fix this issue on 2.0.0?

#16 Updated by Nobuyoshi Nakada 7 months ago

r39171,r39296,r39333,r39334,r39345,r39347,r39348,r40261,r40756,r40757,r40758,r40762,r40763,r40770,r40771,r40777,r40779,r40784,r41352,r41357

#17 Updated by Michal Papis 7 months ago

=begin
any chance to get this merged somehow? I get merge conflicts trying to apply some of the revisions on 2.0.0-p247 source
=end

Also available in: Atom PDF