Project

General

Profile

Actions

Feature #17730

closed

Ruby on macOS transitively links to ~150 dylibs

Added by rickmark (Rick Mark) 6 months ago. Updated 6 months ago.

Status:
Closed
Priority:
Normal
Assignee:
-
Target version:
-
[ruby-core:102919]

Description

By using -framework Security and -framework Foundation Ruby 3 pulls in about 150 dylibs when compiled for macOS

By using CoreCrypto / CoreFoundation I was able to reduce this to ~50. This greatly reduces Ruby's surface area and dependencies on macOS. Further CoreFoundation is only used for one call in the entire codebase of CFStringNormalize(m, kCFStringNormalizationFormC); - if we can replace this, Ruby could work with only libSystem and libgmp.


Files

ruby_deps_after.txt (2.24 KB) ruby_deps_after.txt rickmark (Rick Mark), 03/18/2021 07:16 PM
ruby_deps_before.txt (9.81 KB) ruby_deps_before.txt rickmark (Rick Mark), 03/18/2021 07:16 PM
0001-Remove-unneeded-dependencies-on-macOS.patch (2.36 KB) 0001-Remove-unneeded-dependencies-on-macOS.patch rickmark (Rick Mark), 03/18/2021 07:16 PM
ruby_final.txt (2.24 KB) ruby_final.txt rickmark (Rick Mark), 03/22/2021 07:50 PM

Related issues

Related to Ruby master - Feature #17741: Ruby links to `objc` for convenience - this should be moved into a native extOpenActions

Updated by nobu (Nobuyoshi Nakada) 6 months ago

-lobjc has been intentionally linked to link startup code with Objective-C support, by the request for extension libraries written in it.

Actions #2

Updated by nobu (Nobuyoshi Nakada) 6 months ago

  • Backport deleted (2.5: UNKNOWN, 2.6: UNKNOWN, 2.7: UNKNOWN, 3.0: UNKNOWN)
  • ruby -v deleted (3.0.0)
  • Tracker changed from Bug to Feature
Actions #3

Updated by Anonymous 6 months ago

  • Status changed from Open to Closed

Applied in changeset git|dc7044eb48f475e5ac34e994e89115052574c451.


Remove unneeded dependencies on macOS [Feature #17730]

Updated by rickmark (Rick Mark) 6 months ago

Looks like we applied the Foundation -> CoreFoundation patch.

What about the Security framework? It's the source of the worst dependencies like Metal, CoreML, etc...

(I know my patch proposal moves HEAD to macOS 10.10 or later, but that seems reasonable these days)

We could alternatively use basically any other source for random (the only reason Security is linked).

OpenSSL, /dev/urandom, or arc4random would all be acceptable I suppose

Updated by nobu (Nobuyoshi Nakada) 6 months ago

The Security framework is linked only when it is available but CommonRandom is not.
It is the case that the minimum required macOS version is 10.7(Lion)..10.9(Mavericks).
I'm not sure whether other sources are available on these old versions.

Updated by duerst (Martin Dürst) 6 months ago

rickmark (Rick Mark) wrote:

Further CoreFoundation is only used for one call in the entire codebase of CFStringNormalize(m, kCFStringNormalizationFormC); - if we can replace this, Ruby could work with only libSystem and libgmp.

I'm not at all familiar with the details on a Mac, nor with this library or function in particular. But Unicode String Normalization is provided by String#unicode_normalize, which is implemented in pure Ruby. There may be a bit of glue code necessary, but essentially, the above call should be convertible to something close to m.unicode_normalize(:nfc).

Updated by naruse (Yui NARUSE) 6 months ago

duerst (Martin Dürst) wrote in #note-6:

rickmark (Rick Mark) wrote:

Further CoreFoundation is only used for one call in the entire codebase of CFStringNormalize(m, kCFStringNormalizationFormC); - if we can replace this, Ruby could work with only libSystem and libgmp.

I'm not at all familiar with the details on a Mac, nor with this library or function in particular. But Unicode String Normalization is provided by String#unicode_normalize, which is implemented in pure Ruby. There may be a bit of glue code necessary, but essentially, the above call should be convertible to something close to m.unicode_normalize(:nfc).

Why it uses CFStringNormalize(m, kCFStringNormalizationFormC); is because it needs to just the same conversion as HFS+/APFS, which is different from standard NFC.

Updated by rickmark (Rick Mark) 6 months ago

nobu (Nobuyoshi Nakada) wrote in #note-5:

The Security framework is linked only when it is available but CommonRandom is not.
It is the case that the minimum required macOS version is 10.7(Lion)..10.9(Mavericks).
I'm not sure whether other sources are available on these old versions.

From compiling recently it looks like Security.framework is always linked (see the static linkage results from the text files)

I can rework this patch to do what we would expect (use Security.framework on 10.7-10.9 and then use CoreCrypto for > 10.10)

Will open a new issue for that work

Updated by rickmark (Rick Mark) 6 months ago

naruse (Yui NARUSE) wrote in #note-7:

duerst (Martin Dürst) wrote in #note-6:

rickmark (Rick Mark) wrote:

Further CoreFoundation is only used for one call in the entire codebase of CFStringNormalize(m, kCFStringNormalizationFormC); - if we can replace this, Ruby could work with only libSystem and libgmp.

I'm not at all familiar with the details on a Mac, nor with this library or function in particular. But Unicode String Normalization is provided by String#unicode_normalize, which is implemented in pure Ruby. There may be a bit of glue code necessary, but essentially, the above call should be convertible to something close to m.unicode_normalize(:nfc).

Why it uses CFStringNormalize(m, kCFStringNormalizationFormC); is because it needs to just the same conversion as HFS+/APFS, which is different from standard NFC.

Yep - the normalization is called UTF-8-MAC on libicu. This implies we can use other code to do this. Also since CF is opensource we could just bring in the routine directly and avoid any linkage to core macOS frameworks other then libSystem.B. This would also allow other platforms to do proper normalization for macOS as well? I will cut a new ticket for this.

Updated by rickmark (Rick Mark) 6 months ago

rickmark (Rick Mark) wrote in #note-8:

nobu (Nobuyoshi Nakada) wrote in #note-5:

The Security framework is linked only when it is available but CommonRandom is not.
It is the case that the minimum required macOS version is 10.7(Lion)..10.9(Mavericks).
I'm not sure whether other sources are available on these old versions.

From compiling recently it looks like Security.framework is always linked (see the static linkage results from the text files)

I can rework this patch to do what we would expect (use Security.framework on 10.7-10.9 and then use CoreCrypto for > 10.10)

Will open a new issue for that work

Nevermind - I just recompiled ruby from head. Looks like the linking is correct now. Must have been Foundation brining in Security causing my confusion. Final linking output attached

Updated by naruse (Yui NARUSE) 6 months ago

rickmark (Rick Mark) wrote in #note-9:

naruse (Yui NARUSE) wrote in #note-7:

duerst (Martin Dürst) wrote in #note-6:

rickmark (Rick Mark) wrote:

Further CoreFoundation is only used for one call in the entire codebase of CFStringNormalize(m, kCFStringNormalizationFormC); - if we can replace this, Ruby could work with only libSystem and libgmp.

I'm not at all familiar with the details on a Mac, nor with this library or function in particular. But Unicode String Normalization is provided by String#unicode_normalize, which is implemented in pure Ruby. There may be a bit of glue code necessary, but essentially, the above call should be convertible to something close to m.unicode_normalize(:nfc).

Why it uses CFStringNormalize(m, kCFStringNormalizationFormC); is because it needs to just the same conversion as HFS+/APFS, which is different from standard NFC.

Yep - the normalization is called UTF-8-MAC on libicu. This implies we can use other code to do this. Also since CF is opensource we could just bring in the routine directly and avoid any linkage to core macOS frameworks other then libSystem.B. This would also allow other platforms to do proper normalization for macOS as well? I will cut a new ticket for this.

Ruby also has UTF-8-MAC, but HFS+/APFS's one is sometimes updated.
We first used our own encoding table for the usage, but finally we switched to use OS's to ensure the behavior is exactly same as the filesystem.

Actions #12

Updated by mame (Yusuke Endoh) 6 months ago

  • Related to Feature #17741: Ruby links to `objc` for convenience - this should be moved into a native ext added
Actions

Also available in: Atom PDF