Feature #9018

Make statically linked extensions easier to use

Added by Lukas S almost 2 years ago. Updated almost 2 years ago.



Sorry for the lengthy post, my english is not that good and the idea is not that simple to explain either.

==Current situation

For platforms where dynamic linking is not available, the ruby build system provides a way to disable dynamic loading of '.bundle's.
When this is disabled, a program embedding libruby-static needs to link against the static library build for an extension and manually call its (({Init_someextension()})) function to use the extension.

This has some issues:

(1) A program embedding libruby-static needs to link a whole bunch of static libraries.
(2) The program then needs to call the corresponding (({Init_something()})) function for each extension
(3) require wouldn't work as usual for the extension parts. E.g.: (({require 'etc'})) won't work, because there is no etc file in the include path.
Instead the things provided by the extension will be available as soon as the embedding program calls the init function of the extension.


I think this could be made a lot easier:

(1) While building libruby-static, also link all extensions into libruby-static.a, so programs linking this library also get all extensions.
(2) Add dummy files into the ruby library that tell the require function to call the Init function for the ext being required, so the extension won't
be available before it is required, and can be required just like when using dynamic linking.

===Details on 2.:
The require mechanism implemented in (({rb_require_safe})) (load.c) currently differentiates between two file types: a ruby file and a shared object.

I would suggest adding a third type which is just a dummy file (recognized via file extension, e.g. 'staticext').

When a file of this type is required, the require mechanism determines the corresponding Init_someextension function (either via a table that is filled at compile time,
or (and preferably) via building a function name from the filename (or contents of the file???) and getting the functions memory address
via something like (({dlsym}))).

It then calls the function if it has not already been called (it'd need to keep a list of files that were already included for this). If the function is not defined, a (({LoadError})) is raised.

The dummy files would need to be put into the ruby lib, just like bundles are put there when linking dynamically. It may be a good idea to put them in a special folder to make separation easier though.
With the example of the 'etc'-extension, such a dummy file could have a path something like this: (({/usr/lib/ruby/2.0.0/linked-ext/etc.staticext})).

If this was implemented, using statically linked extensions would be just like using dynamically loaded bundles from the ruby side, which is (imho) the biggest priority.


To make my idea a little bit more clear, here is a example of how this might work for the extension 'etc':

Build process:
(1) While compiling, the user passes the options '(({--disable-dln}))', '(({--with-static-linked-ext}))' and '(({--enable-shared=no}))' to configure, causing dynamic linking to be disallowed.
(2) The user uncomments '(({option nodynamic}))' and '(({etc}))' in (({ext/Setup})).
(3) While building, the build system links the static library generated for etc into libruby-static.
(4) The build system also places a file called '(({etc.staticext}))' in the ruby library folder.

At runtime:
(1) A script calls (({require 'etc'})).
(2) (({rb_require_safe})) finds the (({etc.staticext})) file.
(3) (({rb_require_safe})) determines the init function name as Init_etc and calls this function.
(4) The etc extension is usable in the script.


#1 Updated by Nobuyoshi Nakada almost 2 years ago

  • Status changed from Open to Feedback
  1. A program embedding libruby-static needs to link a whole bunch of static libraries.

All static libraries are linked into libruby-static.a.

  1. The program then needs to call the corresponding Init_something() function for each extension

Those functions are called at bootstrap.

I don't think a dummy file is needed.

#2 Updated by Lukas S almost 2 years ago

All static libraries are linked into libruby-static.a.

Okay then I need to fix my build process. Can You tell me what flag exactly is making this happen?

Those functions are called at bootstrap.

bootstrap = ruby_init?

I don't think a dummy file is needed.

I think it would be nice for the following reason:

Imagine a program embeds ruby to provide some scripting functionality to the user.

With how it is now, the user of that program needs to know wether an extension is a native extension (so he won't need to and in fact can't require it) or it is just a ruby file in the lib (in which case he needs to require it normally). I think this is very inconvenient for the user and makes using the standard library more complicated and involved.

With the dummy files the behaviour of statically linked and dynamically loaded extensions would be absolutely the same to the user, so the behaviour of ruby would be consistent everywhere, which is, imho, a very important thing.

Also available in: Atom PDF