Project

General

Profile

Actions
Copy link

Bug #14662

closed

Stack consistency error in 2.5.1

Added by jacobevelyn (Jacob Evelyn) about 6 years ago. Updated over 4 years ago.

Status:
Closed
Assignee:
-
Target version:
-
ruby -v:
2.5.1
Backport:
2.3: UNKNOWN, 2.4: UNKNOWN, 2.5: UNKNOWN
[ruby-core:86444]

Description

We've been trying to upgrade an app to Ruby 2.5 and are seeing interpreter crashes with Stack consistency error messages on both 2.5.0 and 2.5.1 when running our RSpec test suite. I've spent several days trying to pin this down but haven't been able to yet--it seems to happen nondeterministically about 10% of the time. I have not been able to produce these crashes on Ruby 2.4.4 or earlier (but because they're nondeterministic, it's possible I just got unlucky).

Because we randomize our test ordering, stack traces for the crashes have occurred in a handful of different places in gems we depend on, but if we run the tests in the same order (using pinned seeds) the crashes--when they occur, which is not every time, even with a pinned seed--will usually give the same stack trace.

The stack traces we've seen have pointed us to these lines in these versions of these gems:

https://github.com/jeremyevans/sequel/blob/d9104d2cf0611f749a16fe93c4171a1147dfd4b2/lib/sequel/dataset/query.rb#L1257

https://github.com/rails/rails/blob/813af4655f9bf3c712cf50205eebd337070cee52/activesupport/lib/active_support/ordered_options.rb#L52

https://github.com/rails/rails/blob/813af4655f9bf3c712cf50205eebd337070cee52/activesupport/lib/active_support/log_subscriber.rb#L109

https://github.com/rails/sprockets/blob/10dada6c30016ca89662c0aa48591f071b069420/lib/sprockets/mime.rb#L122

https://github.com/rspec/rspec-core/blob/3bef84bf4cce96a4417a6550d39f53390ec81439/lib/rspec/core/example_group.rb#L52

https://github.com/rails/rails/blob/813af4655f9bf3c712cf50205eebd337070cee52/activesupport/lib/active_support/notifications/instrumenter.rb#L80

In trying to pin this down, I've been able to reproduce with some regularity with a subset of around 100 of our tests, but not fewer. I don't know what about running all those tests triggers it -- some combination of code, or a garbage collection or timing issue. Based on which tests run (lots of complex behavior involving Queue and WeakRef) I suspected the issue might be related to https://bugs.ruby-lang.org/issues/13552, but when I checked out Ruby's git version 79d99551801812f2513dacc73b6dea716cc07096 (right before the changes for #13552), the crash--or a similar Stack consistency error--still occurred.

As I mentioned, the core dumps/stack traces vary but here's one of them from Ruby 2.5.1 (I'm happy to provide more if that's helpful):

/bundle/gems/sprockets-3.7.1/lib/sprockets/mime.rb:122: [BUG] Stack consistency error (sp: 960, bp: 959)
ruby 2.5.1p57 (2018-03-29 revision 63029) [x86_64-linux]
(snip)

Also, sometimes immediately after the Stack consistency error occurs and prints, a Segmentation fault occurs and prints the same stack trace. One additional thing I noticed is that the stack trace includes both the strings 2.5.1 and 2.5.0--I'm not sure if that's expected or the sign of a bad Ruby installation.

I know this isn't a particularly helpful bug report but I've unfortunately reached the limit of my expertise. Let me know if there's anything else I can provide or any additional investigation I can do!

Files

bug-14662.log (81 KB) bug-14662.log nobu (Nobuyoshi Nakada), 04/04/2018 10:02 AM
Actions
Copy link

Also available in: Atom PDF

Like0
Like0Like0Like0Like0Like0Like0