Project

General

Profile

Actions

Bug #15472

closed

Invalid JSON data being sent from Net::HTTP in some cases with Ruby 2.6.0

Added by mseneadza (Michael Seneadza) almost 3 years ago. Updated almost 3 years ago.

Status:
Closed
Priority:
Normal
Assignee:
-
Target version:
-
[ruby-core:90749]

Description

It seems that invalid (JSON) data is being sent once the request's body passes a size threshold. I'm sending emails via the sparkpost.com API using the sparkpost_rails gem ( https://github.com/the-refinery/sparkpost_rails ). That gem uses Net::HTTP to make its API calls.

I upgraded my app from Ruby 2.5.3 to Ruby 2.6.0 on December 25th. Yesterday, on the 26th, my app attempted to send its nightly emails to users. I was flooded with these errors from the API:

{ "errors": [ { "message": "invalid data format\/type", "description": "Problems parsing request as json", "code": "1300" } ] }

After some debugging / trial & error I discovered that the length of the email messages seemed to be the problem. I haven't nailed down the exact maximum length that will work but I know that if the request body is 16,511 characters long the message will be sent. If it's 17,396 characters it will fail.

I've also tried replacing Net::HTTP with the rest_client gem. The full-sized messages get sent with no problems using RestClient.

So I can replace this (https://github.com/the-refinery/sparkpost_rails/blob/master/lib/sparkpost_rails/delivery_method.rb#L379):

    def post_to_api
      url = "https://api.sparkpost.com/api/v1/transmissions"

      uri = URI.parse(url)
      http = Net::HTTP.new(uri.host, uri.port)
      http.use_ssl = true

      request = Net::HTTP::Post.new(uri.path, @headers)
      request.body = JSON.generate(@data)
      http.request(request)
    end

with

    def post_to_api
      url = "https://api.sparkpost.com/api/v1/transmissions"

      RestClient.post(url, JSON.generate(@data), @headers)

    end

and my (full-length, much larger than 17,396 characters) messages get sent with no problem.

I also tried logging the actual HTTP requests using the httplog gem (https://github.com/trusche/httplog) in Ruby 2.5.3 and 2.6.0. Unfortunately that showed the request payloads were identical. But I suspect that gem's logging does some conversion that isn't actually happening in the real data being sent to the SparkPost API.

Actions #2

Updated by naruse (Yui NARUSE) almost 3 years ago

  • Backport changed from 2.4: UNKNOWN, 2.5: UNKNOWN, 2.6: UNKNOWN to 2.4: DONTNEED, 2.5: DONTNEED, 2.6: REQUIRED
Actions #3

Updated by naruse (Yui NARUSE) almost 3 years ago

  • Status changed from Open to Closed

Updated by naruse (Yui NARUSE) almost 3 years ago

  • Backport changed from 2.4: DONTNEED, 2.5: DONTNEED, 2.6: REQUIRED to 2.4: DONTNEED, 2.5: DONTNEED, 2.6: DONE

ruby_2_6 r66799 merged revision(s) 66582.

Actions

Also available in: Atom PDF