Project

General

Profile

Actions

Bug #18058

closed

3.1.0-dev with MJIT enabled Zlib::BufError during `gem install`

Added by byroot (Jean Boussier) over 2 years ago. Updated about 1 year ago.

Status:
Closed
Target version:
-
ruby -v:
ruby 3.1.0dev (2021-08-01T00:54:36Z master 605421f4eb) +JIT [x86_64-linux]
[ruby-core:<unknown>]

Description

It doesn't always happen (maybe 5-10% of the cases), but since I added MJIT to our CI builds I've been seeing this error a lot.

I was able to reproduce it in docker run -it rubylang/ruby:master-nightly-bionic after 15 or so tries:

# ruby -v
ruby 3.1.0dev (2021-08-01T00:54:36Z master 605421f4eb) [x86_64-linux]
# gem uninstall bundler -v 2.2.21; env RUBYOPT="--enable-jit --jit-max-cache=10000 --jit-min-calls=10 $RUBYOPT" gem install --no-document --user-install bundler -v 2.2.21
Successfully uninstalled bundler-2.2.21
WARNING:  You don't have /root/.local/share/gem/ruby/3.1.0/bin in your PATH,
	  gem executables will not run.
Exception while verifying /usr/local/lib/ruby/gems/3.1.0/cache/bundler-2.2.21.gem
ERROR:  While executing gem ... (Zlib::BufError)
    buffer error

e.g.

$ docker run -it rubylang/ruby:master-nightly-bionic bash -c 'while env RUBYOPT="--enable-jit --jit-max-cache=10000 --jit-min-calls=10 $RUBYOPT" gem install --no-document --user-install bundler -v 2.2.21; do gem uninstall bundler -v 2.2.21; done'
Fetching bundler-2.2.21.gem
WARNING:  You don't have /root/.local/share/gem/ruby/3.1.0/bin in your PATH,
	  gem executables will not run.
Successfully installed bundler-2.2.21
1 gem installed
Successfully uninstalled bundler-2.2.21
WARNING:  You don't have /root/.local/share/gem/ruby/3.1.0/bin in your PATH,
	  gem executables will not run.
Successfully installed bundler-2.2.21
1 gem installed
Successfully uninstalled bundler-2.2.21
WARNING:  You don't have /root/.local/share/gem/ruby/3.1.0/bin in your PATH,
	  gem executables will not run.
Successfully installed bundler-2.2.21
1 gem installed
Successfully uninstalled bundler-2.2.21
WARNING:  You don't have /root/.local/share/gem/ruby/3.1.0/bin in your PATH,
	  gem executables will not run.
Successfully installed bundler-2.2.21
1 gem installed
Successfully uninstalled bundler-2.2.21
WARNING:  You don't have /root/.local/share/gem/ruby/3.1.0/bin in your PATH,
	  gem executables will not run.
Successfully installed bundler-2.2.21
1 gem installed
Successfully uninstalled bundler-2.2.21
WARNING:  You don't have /root/.local/share/gem/ruby/3.1.0/bin in your PATH,
	  gem executables will not run.
Successfully installed bundler-2.2.21
1 gem installed
Successfully uninstalled bundler-2.2.21
WARNING:  You don't have /root/.local/share/gem/ruby/3.1.0/bin in your PATH,
	  gem executables will not run.
Successfully installed bundler-2.2.21
1 gem installed
Successfully uninstalled bundler-2.2.21
WARNING:  You don't have /root/.local/share/gem/ruby/3.1.0/bin in your PATH,
	  gem executables will not run.
Successfully installed bundler-2.2.21
1 gem installed
Successfully uninstalled bundler-2.2.21
WARNING:  You don't have /root/.local/share/gem/ruby/3.1.0/bin in your PATH,
	  gem executables will not run.
Successfully installed bundler-2.2.21
1 gem installed
Successfully uninstalled bundler-2.2.21
WARNING:  You don't have /root/.local/share/gem/ruby/3.1.0/bin in your PATH,
	  gem executables will not run.
Exception while verifying /usr/local/lib/ruby/gems/3.1.0/cache/bundler-2.2.21.gem
ERROR:  While executing gem ... (Zlib::BufError)
    buffer error

Related issues 1 (0 open1 closed)

Related to Ruby master - Bug #18277: buffer error (Zlib::BufError) in Zlib::Deflate#deflate when using MJITClosedk0kubun (Takashi Kokubun)Actions

Updated by k0kubun (Takashi Kokubun) over 2 years ago

  • Assignee set to k0kubun (Takashi Kokubun)

Updated by k0kubun (Takashi Kokubun) over 2 years ago

Do you know if we can let Bundler rubygems (you're right, it wasn't Bundler) show the backtrace? The first step to fix this is to increase reproduction probability, and the backtrace information might be helpful for reproducing the issue with a smaller script for it.

Updated by byroot (Jean Boussier) over 2 years ago

Do you know if we can let Bundler show the backtrace?

It's rubygems not bundler. I don't know how make it print the backtrace, but I can take a look. Worst case I can probably directly patch it to let the error bubble up.

Updated by byroot (Jean Boussier) over 2 years ago

Here you go:

/usr/local/lib/ruby/3.1.0/rubygems/package.rb:698:in `read': buffer error (Zlib::BufError)
	from /usr/local/lib/ruby/3.1.0/rubygems/package.rb:698:in `block in verify_gz'
	from /usr/local/lib/ruby/3.1.0/rubygems/package.rb:697:in `wrap'
	from /usr/local/lib/ruby/3.1.0/rubygems/package.rb:697:in `verify_gz'
	from /usr/local/lib/ruby/3.1.0/rubygems/package.rb:664:in `verify_entry'
	from /usr/local/lib/ruby/3.1.0/rubygems/package.rb:676:in `block in verify_files'
	from /usr/local/lib/ruby/3.1.0/rubygems/package/tar_reader.rb:65:in `each'
	from /usr/local/lib/ruby/3.1.0/rubygems/package.rb:675:in `verify_files'
	from /usr/local/lib/ruby/3.1.0/rubygems/package.rb:606:in `block (2 levels) in verify'
	from /usr/local/lib/ruby/3.1.0/rubygems/package/tar_reader.rb:27:in `new'
	from /usr/local/lib/ruby/3.1.0/rubygems/package.rb:603:in `block in verify'
	from /usr/local/lib/ruby/3.1.0/rubygems/package/file_source.rb:29:in `open'
	from /usr/local/lib/ruby/3.1.0/rubygems/package/file_source.rb:29:in `with_read_io'
	from /usr/local/lib/ruby/3.1.0/rubygems/package.rb:602:in `verify'
	from /usr/local/lib/ruby/3.1.0/rubygems/package.rb:583:in `spec'
	from /usr/local/lib/ruby/3.1.0/rubygems/installer.rb:269:in `spec'
	from /usr/local/lib/ruby/3.1.0/rubygems/installer.rb:719:in `verify_spec'
	from /usr/local/lib/ruby/3.1.0/rubygems/installer.rb:891:in `pre_install_checks'
	from /usr/local/lib/ruby/3.1.0/rubygems/installer.rb:286:in `install'
	from /usr/local/lib/ruby/3.1.0/rubygems/resolver/specification.rb:104:in `install'
	from /usr/local/lib/ruby/3.1.0/rubygems/request_set.rb:194:in `block in install'
	from /usr/local/lib/ruby/3.1.0/rubygems/request_set.rb:182:in `each'
	from /usr/local/lib/ruby/3.1.0/rubygems/request_set.rb:182:in `install'
	from /usr/local/lib/ruby/3.1.0/rubygems/commands/install_command.rb:210:in `install_gem'
	from /usr/local/lib/ruby/3.1.0/rubygems/commands/install_command.rb:226:in `block in install_gems'
	from /usr/local/lib/ruby/3.1.0/rubygems/commands/install_command.rb:219:in `each'
	from /usr/local/lib/ruby/3.1.0/rubygems/commands/install_command.rb:219:in `install_gems'
	from /usr/local/lib/ruby/3.1.0/rubygems/commands/install_command.rb:167:in `execute'
	from /usr/local/lib/ruby/3.1.0/rubygems/command.rb:323:in `invoke_with_build_args'
	from /usr/local/lib/ruby/3.1.0/rubygems/command_manager.rb:178:in `process_args'
	from /usr/local/lib/ruby/3.1.0/rubygems/command_manager.rb:147:in `run'
	from /usr/local/lib/ruby/3.1.0/rubygems/gem_runner.rb:53:in `run'
	from /usr/local/bin/gem:21:in `<main>'
Actions #5

Updated by byroot (Jean Boussier) over 2 years ago

  • Related to Bug #18277: buffer error (Zlib::BufError) in Zlib::Deflate#deflate when using MJIT added
Actions #6

Updated by hsbt (Hiroshi SHIBATA) over 1 year ago

  • Status changed from Open to Assigned

Updated by k0kubun (Takashi Kokubun) about 1 year ago

  • Status changed from Assigned to Closed

I replaced the entire implementation in [Feature #19420], and I haven't seen this issue with today's implementation.

Actions

Also available in: Atom PDF

Like0
Like0Like0Like0Like0Like0Like0Like0