Bug #8167

net/imap failing to parse IMAP response containing one BODYSTRUCTURE

Added by Nathan Mackinnon almost 2 years ago. Updated 8 months ago.

[ruby-core:53741]
Status:Closed
Priority:Normal
Assignee:Shugo Maeda
ruby -v:2.0.0p0 Backport:2.0.0: DONE, 2.1: DONTNEED

Description

Here is debugging output from Net::IMAP::ResponseParse::parse method:

@str: "* 29021 FETCH (RFC822.SIZE 3162 UID 113622 RFC822.HEADER {1155}\r\nReturn-path: <>\r\nEnvelope-to: info@xxxxxxxx.si\r\nDelivery-date: Tue, 26 Mar 2013 12:42:58 +0100\r\nReceived: from mail by xxxx.xxxxxxxxxxx.net with spam-scanned (Exim 4.76)\r\n\tid 1UKSHI-000Cwl-AR\r\n\tfor info@xxxxxxxx.si; Tue, 26 Mar 2013 12:42:58 +0100\r\nX-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on xxxx.xxxxxxxxxxx.net\r\nX-Spam-Level: **\r\nX-Spam-Status: No, score=2.1 required=7.0 tests=DKIM_ADSP_NXDOMAIN,RDNS_NONE\r\n\tautolearn=no version=3.3.1\r\nReceived: from xx.xxx.xxx.xx\r\n\tby xxxx.xxxxxxxxxxx.net with esmtp (Exim 4.76)\r\n\tid 1UKSHI-000Cwi-9j\r\n\tfor info@xxxxxxxx.si; Tue, 26 Mar 2013 12:42:56 +0100\r\nReceived: by xxxxxx.localdomain (Postfix)\r\n\tid 72725BEA64A; Tue, 26 Mar 2013 12:42:55 +0100 (CET)\r\nDate: Tue, 26 Mar 2013 12:42:55 +0100 (CET)\r\nFrom: MAILER-DAEMON@xxxxxx.localdomain (Mail Delivery System)\r\nSubject: Undelivered Mail Returned to Sender\r\nTo: info@xxxxxxxx.si\r\nAuto-Submitted: auto-replied\r\nMIME-Version: 1.0\r\nContent-Type: multipart/report; report-type=delivery-status;\r\n\tboundary=\"27797BEA649.1364298175/xxxxxx.localdomain\"\r\nMessage-Id: 20130326114255.72725BEA64A@xxxxxx.localdomain\r\n\r\n BODYSTRUCTURE ((\"text\" \"plain\" (\"charset\" \"us-ascii\") NIL \"Notification\" \"7bit\" 510 14 NIL NIL NIL NIL)(\"message\" \"delivery-status\" NIL NIL \"Delivery report\" \"7bit\" 410 NIL NIL NIL NIL)(\"text\" \"rfc822-headers\" (\"charset\" \"us-ascii\") NIL \"Undelivered Message Headers\" \"7bit\" 612 15 NIL NIL NIL NIL) \"report\" (\"report-type\" \"delivery-status\" \"boundary\" \"27797BEA649.1364298175/xxxxxx.localdomain\") NIL NIL NIL))\r\n"
@pos: 1390
@lex_state: EXPR_DATA
@token.symbol: SPACE
@token.value: " "
Net::IMAP::ResponseParseError: unexpected token SPACE (expected RPAR)

This error is observed on development machine with ruby 2.0.0-p0. Never reproduced on production installation running 1.9.3-p374.

I do not even know enough about IMAP standard to tell if message is actually invalid, or this is net/imap parser bug.

Associated revisions

Revision 40372
Added by Shugo Maeda almost 2 years ago

  • lib/net/imap.rb (body_type_msg): should accept
    message/delivery-status with extra data.
    [Bug #8167]

  • test/net/imap/test_imap_response_parser.rb: related test.

Revision 40372
Added by Shugo Maeda almost 2 years ago

  • lib/net/imap.rb (body_type_msg): should accept
    message/delivery-status with extra data.
    [Bug #8167]

  • test/net/imap/test_imap_response_parser.rb: related test.

Revision 46733
Added by Usaku NAKAMURA 8 months ago

merge revision(s) 40372: [Backport #8167]

* lib/net/imap.rb (body_type_msg): should accept
  message/delivery-status with extra data.
   [Bug #8167]

* test/net/imap/test_imap_response_parser.rb: related test.

History

#1 Updated by Bob Potter almost 2 years ago

It looks like the response is invalid at least according the RFC3501. The additional fields that the parser is blowing up on should only be included if the type is 'message/rfc822', in this case it is 'message/delivery-status'. It looks like java's parser also has issues with this type of message: https://social.technet.microsoft.com/forums/en-US/exchangesvrsecuremessaginglegacy/thread/a2c7df45-2768-4a67-9c61-b912b56281bf

It would be possible to make the imap parser more liberal to handle these cases but I'm not sure if that's desirable.

It looks like the changed that changed this behavior is r37240.

#2 Updated by Shugo Maeda almost 2 years ago

  • Status changed from Open to Assigned
  • Assignee set to Shugo Maeda

bpot (Bob Potter) wrote:

It looks like the response is invalid at least according the RFC3501. The additional fields that the parser is blowing up on should only be included if the type is 'message/rfc822', in this case it is 'message/delivery-status'. It looks like java's parser also has issues with this type of message: https://social.technet.microsoft.com/forums/en-US/exchangesvrsecuremessaginglegacy/thread/a2c7df45-2768-4a67-9c61-b912b56281bf

It would be possible to make the imap parser more liberal to handle these cases but I'm not sure if that's desirable.

I believe Net::IMAP should accept non-conformant input as long as the meaning is clear.

#3 Updated by Shugo Maeda almost 2 years ago

  • Status changed from Assigned to Closed
  • % Done changed from 0 to 100

This issue was solved with changeset r40372.
Nathan, thank you for reporting this issue.
Your contribution to Ruby is greatly appreciated.
May Ruby be with you.


  • lib/net/imap.rb (body_type_msg): should accept
    message/delivery-status with extra data.
    [Bug #8167]

  • test/net/imap/test_imap_response_parser.rb: related test.

#4 Updated by Usaku NAKAMURA 8 months ago

  • Backport set to 2.0.0: DONE, 2.1: DONTNEED

backported into ruby_2_0_0 at r46733.

Also available in: Atom PDF