Bug #20616: <jemalloc>: Error in munmap(): Invalid argument

• Status: Feedback
• Assignee: -
• Target version: -
• ruby -v: 3.3.2 (2024-05-30 revision e5a195edf6) [aarch64-linux]
• Backport: 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN

[damuz91 (David Muñoz)]

Good day!

Description:

I am getting the following error randomly. I run the offending code every 10 minutes and some times it crashes and some it doesn't, I am clueless.

The error states:

<jemalloc>: Error in munmap(): Invalid argument
[BUG] Segmentation fault at 0x0000ffff69bfe000
ruby 3.3.2 (2024-05-30 revision e5a195edf6) [aarch64-linux]

I've published the backtrace: https://gist.github.com/damuz91/429bed215c7ff13ee33e3968d52a4729

My app runs Sidekiq which runs the offending code, i've submitted the code in Sidekiq but maintainer says it's not related to Sidekiq.

OS: Amazon Linux 2 aarch64
Ruby version: 3.3.2
Rails version: 7.1.3.4

The offending line states:

Jul 05 05:00:01 subsystems-sync sidekiq[11766]: c:0048 p:0049 s:0277 e:000276 METHOD /home/ec2-user/production/sync-backend/releases/370/app/services/scheduler/partial_sync_scheduler_job.rb:6

and my source code looks like this:

class Scheduler::PartialSyncSchedulerJob
  include Sidekiq::Job

  def perform(location_ids = [])
    locations = location_ids.any? ? Location.where(id: location_ids) : Location.enabled.should_sync.includes(:customer).where.not(customers: {service_object: ['xxx', 'yyy']})
    locations.each do |location| # <- Line 6
      if location.use_workflow
        PartialSync::Workflow.perform_async(location.id)
      else
        queue = location.customer.service_object.underscore.to_sym
        Sync::PartialSyncJob.set(queue: queue).perform_later(location.id)
      end
    end
  end

end

I am not sure where to start looking, I would really appreciate any insight to try to find out if this is indeed a bug in Ruby or if this is a jemalloc issue.
Thanks in advance.

[damuz91 (David Muñoz)]

• ruby -v changed from 3.3.2 to 3.3.2 (2024-05-30 revision e5a195edf6) [aarch64-linux]

[alanwu (Alan Wu)]

You could try running with the jemalloc MALLOC_CONF='abort:true' environment variable to get a crash when the munmap() error happens. That might get you more information with a different backtrace.

I would also check if the code runs close to using all available memory.

[alanwu (Alan Wu)]

• Status changed from Open to Feedback