Project

General

Profile

Bug #15468

Net::Protocol::BufferedIO#write raises NoMethodError when sending large multi-byte string

Added by eitoball (Eito Katagiri) 26 days ago. Updated 9 days ago.

Status:
Closed
Priority:
Normal
Assignee:
-
Target version:
-
ruby -v:
ruby 2.6.0p0 (2018-12-25 revision 66547) [x86_64-linux]
[ruby-dev:50734]

Description

ruby-2.6.0で大きなJSON文字列をNet::HTTPでPOSTすると以下のように NoMothodError が発生するようになりました。

$ ruby -rnet/http -rjson -v -e "Net::HTTP.post(URI('http://httpbin.org/post'), { text: 'あ'*100_000 }.to_json, 'Content-Type' => 'application/json')"
ruby 2.6.0p0 (2018-12-25 revision 66547) [x86_64-linux]
Traceback (most recent call last):
        19: from -e:1:in `<main>'
        18: from lib/ruby/2.6.0/net/http.rb:500:in `post'
        17: from lib/ruby/2.6.0/net/http.rb:605:in `start'
        16: from lib/ruby/2.6.0/net/http.rb:920:in `start'
        15: from lib/ruby/2.6.0/net/http.rb:502:in `block in post'
        14: from lib/ruby/2.6.0/net/http.rb:1281:in `post'
        13: from lib/ruby/2.6.0/net/http.rb:1493:in `send_entity'
        12: from lib/ruby/2.6.0/net/http.rb:1479:in `request'
        11: from lib/ruby/2.6.0/net/http.rb:1506:in `transport_request'
        10: from lib/ruby/2.6.0/net/http.rb:1506:in `catch'
         9: from lib/ruby/2.6.0/net/http.rb:1507:in `block in transport_request'
         8: from lib/ruby/2.6.0/net/http/generic_request.rb:123:in `exec'
         7: from lib/ruby/2.6.0/net/http/generic_request.rb:189:in `send_request_with_body'
         6: from lib/ruby/2.6.0/net/protocol.rb:247:in `write'
         5: from lib/ruby/2.6.0/net/protocol.rb:265:in `writing'
         4: from lib/ruby/2.6.0/net/protocol.rb:248:in `block in write'
         3: from lib/ruby/2.6.0/net/protocol.rb:275:in `write0'
         2: from lib/ruby/2.6.0/net/protocol.rb:275:in `each_with_index'
         1: from lib/ruby/2.6.0/net/protocol.rb:275:in `each'
lib/ruby/2.6.0/net/protocol.rb:280:in `block in write0': undefined method `bytesize' for nil:NilClass (NoMethodError)

理由は、Net::Protocol::BufferedIO#write0で、String#[]で送信されていない文字列を求める際に文字列にマルチバイト文字が含まれているとIO#write_nonblockで返ってくる送信されたバイト数が文字数を超えるため、nilになるためだと考えています。https://github.com/ruby/ruby/pull/2058 を送ってみました。

Associated revisions

Revision 66582
Added by nobu (Nobuyoshi Nakada) 26 days ago

Fix Net::Protocol::BufferedIO#write when sending large multi-byte string

This commit should fix Net::Protocol::BufferedIO#write when sending
large multi-byte string like following example.

$ ruby -rnet/http -rjson -v -e "Net::HTTP.post(URI('http://httpbin.org/post'), { text: 'あ'*100_000 }.to_json, 'Content-Type' => 'application/json')"
ruby 2.6.0p0 (2018-12-25 revision 66547) [x86_64-linux]
Traceback (most recent call last):
        19: from -e:1:in `<main>'
        18: from lib/ruby/2.6.0/net/http.rb:500:in `post'
        17: from lib/ruby/2.6.0/net/http.rb:605:in `start'
        16: from lib/ruby/2.6.0/net/http.rb:920:in `start'
        15: from lib/ruby/2.6.0/net/http.rb:502:in `block in post'
        14: from lib/ruby/2.6.0/net/http.rb:1281:in `post'
        13: from lib/ruby/2.6.0/net/http.rb:1493:in `send_entity'
        12: from lib/ruby/2.6.0/net/http.rb:1479:in `request'
        11: from lib/ruby/2.6.0/net/http.rb:1506:in `transport_request'
        10: from lib/ruby/2.6.0/net/http.rb:1506:in `catch'
         9: from lib/ruby/2.6.0/net/http.rb:1507:in `block in transport_request'
         8: from lib/ruby/2.6.0/net/http/generic_request.rb:123:in `exec'
         7: from lib/ruby/2.6.0/net/http/generic_request.rb:189:in `send_request_with_body'
         6: from lib/ruby/2.6.0/net/protocol.rb:247:in `write'
         5: from lib/ruby/2.6.0/net/protocol.rb:265:in `writing'
         4: from lib/ruby/2.6.0/net/protocol.rb:248:in `block in write'
         3: from lib/ruby/2.6.0/net/protocol.rb:275:in `write0'
         2: from lib/ruby/2.6.0/net/protocol.rb:275:in `each_with_index'
         1: from lib/ruby/2.6.0/net/protocol.rb:275:in `each'
lib/ruby/2.6.0/net/protocol.rb:280:in `block in write0': undefined method `bytesize' for nil:NilClass (NoMethodError)

[Fix GH-2058]

From: Eito Katagiri eitoball@gmail.com

Revision 9548df04
Added by naruse (Yui NARUSE) 9 days ago

merge revision(s) 66582: [Backport #15468][Backport #15472]

    Fix Net::Protocol::BufferedIO#write when sending large multi-byte string

    This commit should fix Net::Protocol::BufferedIO#write when sending
    large multi-byte string like following example.

    ```
    $ ruby -rnet/http -rjson -v -e "Net::HTTP.post(URI('http://httpbin.org/post'), { text: '?'*100_000 }.to_json, 'Content-Type' => 'application/json')"
    ruby 2.6.0p0 (2018-12-25 revision 66547) [x86_64-linux]
    Traceback (most recent call last):
            19: from -e:1:in `<main>'
            18: from lib/ruby/2.6.0/net/http.rb:500:in `post'
            17: from lib/ruby/2.6.0/net/http.rb:605:in `start'
            16: from lib/ruby/2.6.0/net/http.rb:920:in `start'
            15: from lib/ruby/2.6.0/net/http.rb:502:in `block in post'
            14: from lib/ruby/2.6.0/net/http.rb:1281:in `post'
            13: from lib/ruby/2.6.0/net/http.rb:1493:in `send_entity'
            12: from lib/ruby/2.6.0/net/http.rb:1479:in `request'
            11: from lib/ruby/2.6.0/net/http.rb:1506:in `transport_request'
            10: from lib/ruby/2.6.0/net/http.rb:1506:in `catch'
             9: from lib/ruby/2.6.0/net/http.rb:1507:in `block in transport_request'
             8: from lib/ruby/2.6.0/net/http/generic_request.rb:123:in `exec'
             7: from lib/ruby/2.6.0/net/http/generic_request.rb:189:in `send_request_with_body'
             6: from lib/ruby/2.6.0/net/protocol.rb:247:in `write'
             5: from lib/ruby/2.6.0/net/protocol.rb:265:in `writing'
             4: from lib/ruby/2.6.0/net/protocol.rb:248:in `block in write'
             3: from lib/ruby/2.6.0/net/protocol.rb:275:in `write0'
             2: from lib/ruby/2.6.0/net/protocol.rb:275:in `each_with_index'
             1: from lib/ruby/2.6.0/net/protocol.rb:275:in `each'
    lib/ruby/2.6.0/net/protocol.rb:280:in `block in write0': undefined method `bytesize' for nil:NilClass (NoMethodError)
    ```

    [Fix GH-2058]

    From: Eito Katagiri <eitoball@gmail.com>

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/branches/ruby_2_6@66799 b2dd03c8-39d4-4d8f-98ff-823fe69b080e

Revision 66799
Added by naruse (Yui NARUSE) 9 days ago

merge revision(s) 66582: [Backport #15468][Backport #15472]

Fix Net::Protocol::BufferedIO#write when sending large multi-byte string

This commit should fix Net::Protocol::BufferedIO#write when sending
large multi-byte string like following example.

```
$ ruby -rnet/http -rjson -v -e "Net::HTTP.post(URI('http://httpbin.org/post'), { text: '?'*100_000 }.to_json, 'Content-Type' => 'application/json')"
ruby 2.6.0p0 (2018-12-25 revision 66547) [x86_64-linux]
Traceback (most recent call last):
        19: from -e:1:in `<main>'
        18: from lib/ruby/2.6.0/net/http.rb:500:in `post'
        17: from lib/ruby/2.6.0/net/http.rb:605:in `start'
        16: from lib/ruby/2.6.0/net/http.rb:920:in `start'
        15: from lib/ruby/2.6.0/net/http.rb:502:in `block in post'
        14: from lib/ruby/2.6.0/net/http.rb:1281:in `post'
        13: from lib/ruby/2.6.0/net/http.rb:1493:in `send_entity'
        12: from lib/ruby/2.6.0/net/http.rb:1479:in `request'
        11: from lib/ruby/2.6.0/net/http.rb:1506:in `transport_request'
        10: from lib/ruby/2.6.0/net/http.rb:1506:in `catch'
         9: from lib/ruby/2.6.0/net/http.rb:1507:in `block in transport_request'
         8: from lib/ruby/2.6.0/net/http/generic_request.rb:123:in `exec'
         7: from lib/ruby/2.6.0/net/http/generic_request.rb:189:in `send_request_with_body'
         6: from lib/ruby/2.6.0/net/protocol.rb:247:in `write'
         5: from lib/ruby/2.6.0/net/protocol.rb:265:in `writing'
         4: from lib/ruby/2.6.0/net/protocol.rb:248:in `block in write'
         3: from lib/ruby/2.6.0/net/protocol.rb:275:in `write0'
         2: from lib/ruby/2.6.0/net/protocol.rb:275:in `each_with_index'
         1: from lib/ruby/2.6.0/net/protocol.rb:275:in `each'
lib/ruby/2.6.0/net/protocol.rb:280:in `block in write0': undefined method `bytesize' for nil:NilClass (NoMethodError)
```

[Fix GH-2058]

From: Eito Katagiri <eitoball@gmail.com>

History

Updated by rafaelfranca (Rafael França) 12 days ago

Can we get this issue closed and the with right backport field values?

#2

Updated by nobu (Nobuyoshi Nakada) 11 days ago

  • Backport changed from 2.4: UNKNOWN, 2.5: UNKNOWN, 2.6: UNKNOWN to 2.4: UNKNOWN, 2.5: UNKNOWN, 2.6: REQUIRED
  • Status changed from Open to Closed
#3

Updated by hsbt (Hiroshi SHIBATA) 11 days ago

  • Backport changed from 2.4: UNKNOWN, 2.5: UNKNOWN, 2.6: REQUIRED to 2.4: DONTNEED, 2.5: DONTNEED, 2.6: REQUIRED

Updated by naruse (Yui NARUSE) 9 days ago

  • Backport changed from 2.4: DONTNEED, 2.5: DONTNEED, 2.6: REQUIRED to 2.4: DONTNEED, 2.5: DONTNEED, 2.6: DONE

ruby_2_6 r66799 merged revision(s) 66582.

Also available in: Atom PDF