Bug #21645
openCan't `require "resolve"` on Windows under Bundler without warnings
Description
prism had disabled some workflows for ruby-head on windows because of the extraction from fiddle to being a bundled gem.
I attempted to renable them now, but it is still an issue: https://github.com/ruby/prism/pull/3687
During bundle install, the mini_portile2 gem (to build nokogiri) ends up requiring net/http, which in turn requires resolv.
On windows, resolv requires win32/resolv here: https://github.com/ruby/ruby/blob/6eb75f6c36288cdd7522f27ad18608d13031f2b8/lib/resolv.rb#L1026-L1027. win32-registry depends on fiddle, and so under bundler it fails with the following:
D:/rubyinstaller-head-x64/lib/ruby/3.5.0+4/rubygems/specification.rb:1419:in
'block in Gem::Specification#activate_dependencies': Could not find 'fiddle' (~>
1.0) among 61 total gem(s) (Gem::MissingSpecError)
Checked in 'GEM_PATH=D:/a/prism/prism/vendor/bundle/ruby/3.5.0+4' at:
D:/rubyinstaller-head-x64/lib/ruby/gems/3.5.0+4/specifications/win32-registry-0.1.0.gemspec,
execute `gem env` for more information
from
D:/rubyinstaller-head-x64/lib/ruby/3.5.0+4/rubygems/specification.rb:1405:in
'Array#each'
from
D:/rubyinstaller-head-x64/lib/ruby/3.5.0+4/rubygems/specification.rb:1405:in
'Gem::Specification#activate_dependencies'
from
D:/rubyinstaller-head-x64/lib/ruby/3.5.0+4/rubygems/specification.rb:1387:in
'Gem::Specification#activate'
from
D:/rubyinstaller-head-x64/lib/ruby/3.5.0+4/rubygems/core_ext/kernel_gem.rb:62:in
'block in Kernel#gem'
from
D:/rubyinstaller-head-x64/lib/ruby/3.5.0+4/rubygems/core_ext/kernel_gem.rb:62:in
'Thread::Mutex#synchronize'
from
D:/rubyinstaller-head-x64/lib/ruby/3.5.0+4/rubygems/core_ext/kernel_gem.rb:62:in
'Kernel#gem'
from D:/rubyinstaller-head-x64/lib/ruby/3.5.0+4/resolv.rb:1026:in
'Kernel#require'
from D:/rubyinstaller-head-x64/lib/ruby/3.5.0+4/resolv.rb:1026:in
'Resolv::DNS::Config.default_config_hash'
from D:/rubyinstaller-head-x64/lib/ruby/3.5.0+4/resolv.rb:95:in
'Resolv#initialize'
from D:/rubyinstaller-head-x64/lib/ruby/3.5.0+4/resolv.rb:3469:in
'<class:Resolv>'
from D:/rubyinstaller-head-x64/lib/ruby/3.5.0+4/resolv.rb:35:in '<top
(required)>'
from D:/rubyinstaller-head-x64/lib/ruby/3.5.0+4/net/http.rb:25:in
'Kernel#require'
from D:/rubyinstaller-head-x64/lib/ruby/3.5.0+4/net/http.rb:25:in '<top
(required)>'
from
D:/a/prism/prism/vendor/bundle/ruby/3.5.0+4/gems/mini_portile2-2.8.9/lib/mini_portile2/mini_portile.rb:2:in
'Kernel#require'
I have tried to make a small reproduction by simply running bundle exec ruby -rresolv but there I just get a warning:
D:/rubyinstaller-head-x64/lib/ruby/3.5.0+4/win32/registry.rb:2: warning: fiddle/import is found in fiddle, which is not part of the default gems since Ruby 3.5.0.
You can add fiddle to your Gemfile or gemspec to fix this error.
I am not sure why that is. I feel like it should at the very least raise. But I don't have a windows machine to investigate this. Here is a github action run that shows this: https://github.com/Earlopain/test/actions/runs/18653590178/job/53177107372
There is also a github issue: https://github.com/ruby/win32-registry/issues/11
Updated by Earlopain (Earlopain _) 21 days ago
Here is a standalone workflow that demonstrates the issue closer to how I actually encountered it:
name: Test
on: push
jobs:
make:
runs-on: windows-latest
steps:
- run: echo "source 'https://rubygems.org'" >> Gemfile
- run: echo "gem 'nokogiri'" >> Gemfile
- name: Set up Ruby
uses: ruby/setup-ruby@v1
with:
ruby-version: head
bundler-cache: true
https://github.com/Earlopain/test/actions/runs/18682884097/job/53267891760
I tried adding fiddle to the gemfile to work around this, but I believe since this is happening during installation, doing that is not effective.
Updated by mdalessio (Mike Dalessio) 21 days ago
Would this problem be avoided if mini_portile2 declared an explicit dependency on net-http? If it would be helpful I'm happy to cut a new release of mini_portile with that change.
Updated by Earlopain (Earlopain _) 21 days ago
Thanks for popping in!
Would this problem be avoided if mini_portile2 declared an explicit dependency on net-http
No, that would not do much I think. The problem is that win32-registry is a default gem but it depends on fiddle which is not one anymore.
Actually I think this is supposed to be handled by https://github.com/ruby/ruby/commit/02a9c05e3ecc5b4a382ca9c7ca3e4adb79844bc2 and https://github.com/ruby/ruby/commit/a487698cc789624924ef5e309ffbffcc36f754ca but for some reason doing that doesn't really work under bundler.
Updated by Eregon (Benoit Daloze) 20 days ago
My impression here is we should always respect dependencies before changing from stdlib/default gem to bundled gem.
Concretely, I think fiddle shouldn't be a bundled gem before its stdlib & default gems usages (resolv, win32/resolv, win32-registry, etc) have become bundled gems before.
Updated by hsbt (Hiroshi SHIBATA) 13 days ago
- Status changed from Open to Assigned
- Assignee set to hsbt (Hiroshi SHIBATA)
It should fallback to pwsh implementation under bundle environment.
diff --git ext/win32/lib/win32/resolv.rb ext/win32/lib/win32/resolv.rb
index 226de7cba36..dae53baf118 100644
--- ext/win32/lib/win32/resolv.rb
+++ ext/win32/lib/win32/resolv.rb
@@ -58,7 +58,7 @@ def read_s(key)
end
end
using SZ
- rescue LoadError
+ rescue LoadError, Gem::MissingSpecError
require "open3"
end
Can you try the above patch? I don't have head version of mingw platform now.
Updated by Earlopain (Earlopain _) 13 days ago
I approximatly applied your patch with the following in github ci:
- name: Patch
run: (Get-Content D:/rubyinstaller-head-x64/lib/ruby/3.5.0+4/win32/resolv.rb).Replace('LoadError', 'LoadError, Gem::MissingSpecError') | Set-Content D:/rubyinstaller-head-x64/lib/ruby/3.5.0+4/win32/resolv.rb
It seems to work. I have a successful run with this here https://github.com/ruby/prism/actions/runs/18905343559/job/53962176814?pr=3687
Updated by hsbt (Hiroshi SHIBATA) 12 days ago
- Status changed from Assigned to Closed
Applied in changeset git|15f2dcceb4787c5738dde48f580019c3765ce1b8.
[ruby/resolv] Fallback to powershell implementation under the bundle environment
[Bug #21645]
win32-registory can't load fiddle if Gemfile didn't have that dependency.
Updated by hsbt (Hiroshi SHIBATA) 12 days ago
@Earlopain Thanks for your confirmation. I fixed that at ruby/resolv and ruby/ruby.
Updated by Earlopain (Earlopain _) 10 days ago
There's a new windows-head build that I tried now. Unfortunately I still get this error. The difference seems to be that bundler-cache: true specifies a vendor directory for bundle install, which I skipped when applying the change. So I fear the patch didn't actually change anything.
If you follow the stacktrace, it never reaches win32/resolv.rb. It fails earlier in something that bundler does. Even if you would catch the error at require of win32/resolv, the required functionality (either powershell or native) would not be there.
Why is it even part of win32-registry in the first place? I don't see that file upstream in https://github.com/ruby/win32-registry, it just lives there in the ruby/ruby copy. If it would be possible to move this file elsewhere so it is not bound by the gemspec, then I think that would solve the problem. Could it be moved to the resolv gem instead? At least the ruby part.
On the other hand, why would specifying a vendor path have any impact here? Perhaps a bundler bug (I would expect it to raise the error with or without the vendor path specified). I cannot tell. To summarize:
name: Test
on: push
jobs:
make:
runs-on: windows-latest
steps:
- run: echo "source 'https://rubygems.org'" >> Gemfile
- run: echo "gem 'nokogiri'" >> Gemfile
- name: Set up Ruby
uses: ruby/setup-ruby@v1
with:
ruby-version: head
# Commenting out this line makes the error go away
- run: bundle config --local path D:\a\test\test\vendor\bundle
- run: bundle install
Updated by Earlopain (Earlopain _) 8 days ago
@hsbt (Hiroshi SHIBATA) not sure if you got notification for the last message, could you reopen this?
Updated by hsbt (Hiroshi SHIBATA) 8 days ago
- Status changed from Closed to Assigned
Updated by Earlopain (Earlopain _) about 5 hours ago
https://github.com/ruby/resolv/pull/107/commits/b2c775cd80f0ed934336dd1952152982cb3cef14 "improved" this by loading win32/resolv just once and catching the LoadError. CI now passes but as a consequence I guess it will no longer respect nameservers on windows (if there are any)