Bug #4509: Net::IMAP::ResponseParseError: unexpected token CRLF (expected NUMBER)

• Status: Closed
• Assignee: shugo (Shugo Maeda)
• Target version: -
• ruby -v: 1.9.2-p136

[digger69 (Mark Nadig)]

=begin
To reproduce, create a free yahoo mail account. Using those credentials try this ruby code:

require 'net/imap'
Net::IMAP.debug = true
conn = Net::IMAP.new('imap.mail.yahoo.com', 143, false)
conn.instance_eval { send_command('ID ("GUID" "1")') }
conn.authenticate('LOGIN', <username>, <password>)
conn.select("INBOX")
uids = conn.uid_search(['ALL'])
conn.logout
conn.disconnect

You should have at least one welcome email and the response from the conn.uid_search(['ALL']) will return "* SEARCH 1 \r\n" The trailing space before the CRLF seems unanticipated by search_response and causes the 'unexpected token' downstream in number. Here is my patch:

 def search_response # line 2706 imap.rb
    token = match(T_ATOM)
    name = token.value.upcase
    token = lookahead
    if token.symbol == T_SPACE
      shift_token
      data = []
      while true
        token = lookahead

#begin patch - yahoo IMAP was returning pattern " SEARCH 1 2 \r\n so was doing push on CRLF
=begin before patch
case token.symbol
when T_CRLF
break
when T_SPACE
shift_token
end
data.push(number)
=end
case token.symbol
when T_CRLF
break
when T_SPACE
shift_token
else
data.push(number)
end
#end patch
end
else
data = []
end
return UntaggedResponse.new(name, data, @str)
end

I confirmed this code is unchanged in p180 so didn't test it there. I hope this helps.
=end

[digger69 (Mark Nadig)]

• File bug4509.txt bug4509.txt added

=begin
[not sure what happened to body of submission - putting in as update]

To reproduce, create a free yahoo mail account. Using those credentials try this ruby code:

require 'net/imap'
Net::IMAP.debug = true
conn = Net::IMAP.new('imap.mail.yahoo.com', 143, false)
conn.instance_eval { send_command('ID ("GUID" "1")') }
conn.authenticate('LOGIN', <username>, <password>)
conn.select("INBOX")
uids = conn.uid_search(['ALL'])
conn.logout
conn.disconnect

You should have at least one welcome email and the response from the conn.uid_search(['ALL']) will return "* SEARCH 1 \r\n" The trailing space before the CRLF seems unanticipated by search_response and causes the 'unexpected token' downstream in number. Here is my patch:

 def search_response # line 2706 imap.rb
    token = match(T_ATOM)
    name = token.value.upcase
    token = lookahead
    if token.symbol == T_SPACE
      shift_token
      data = []
      while true
        token = lookahead

#begin patch - yahoo IMAP was returning pattern " SEARCH 1 2 \r\n so was doing push on CRLF
=begin before patch
case token.symbol
when T_CRLF
break
when T_SPACE
shift_token
end
data.push(number)
=end
case token.symbol
when T_CRLF
break
when T_SPACE
shift_token
else
data.push(number)
end
#end patch
end
else
data = []
end
return UntaggedResponse.new(name, data, @str)
end

I confirmed this code is unchanged in p180 so didn't test it there. I hope this helps.
=end

[digger69 (Mark Nadig)]

=begin
=== Better steps to repro
I created a free account on yahoo to reproduce this error. Now, you can repro with these steps:
(({require 'net/imap'
conn = Net::IMAP.new('imap.mail.yahoo.com', 143, false)
conn.instance_eval { send_command('ID ("GUID" "1")') }
conn.authenticate('LOGIN', '[e-mail]', '!m@Pp@ssw0rd')
conn.select("INBOX")
uids = conn.uid_search(['ALL'])
conn.logout
conn.disconnect
}))

Patch:
def search_response # line 2706 imap.rb
token = match(T_ATOM)
name = token.value.upcase
token = lookahead
if token.symbol == T_SPACE
shift_token
data = []
while true
token = lookahead
#begin patch - yahoo IMAP was returning pattern " SEARCH 1 2 \r\n so was doing push on CRLF
=begin before patch
case token.symbol
when T_CRLF
break
when T_SPACE
shift_token
end
data.push(number)
=end
case token.symbol
when T_CRLF
break
when T_SPACE
shift_token
else
data.push(number)
end
#end patch
end
else
data = []
end
return UntaggedResponse.new(name, data, @str)
end

=end

[digger69 (Mark Nadig)]

=begin
(({def a # test - why "Error: file empty." in bug report}))
=end

[digger69 (Mark Nadig)]

=begin
=== Better steps to repro
I created a free account on yahoo to reproduce this error. Now, you can repro with these steps:

require 'net/imap'
conn = Net::IMAP.new('imap.mail.yahoo.com', 143, false)
conn.instance_eval { send_command('ID ("GUID" "1")') }
conn.authenticate('LOGIN', '[e-mail]', '!m@Pp@ssw0rd')
conn.select("INBOX")
uids = conn.uid_search(['ALL'])

This fails with "Net::IMAP::ResponseParseError: unexpected token CRLF (expected NUMBER)". I added the following patch locally and is working.

def search_response # line 2706 imap.rb
token = match(T_ATOM)
name = token.value.upcase
token = lookahead
if token.symbol == T_SPACE
shift_token
data = []
while true
token = lookahead
#begin patch - yahoo IMAP was returning pattern " SEARCH 1 2 \r\n so was doing push on CRLF
=begin before patch
case token.symbol
when T_CRLF
break
when T_SPACE
shift_token
end
data.push(number)
=end
case token.symbol
when T_CRLF
break
when T_SPACE
shift_token
else
data.push(number)
end
#end patch
end
else
data = []
end
return UntaggedResponse.new(name, data, @str)
end

=end

[ghazel (Greg Hazel)]

I can reproduce this bug on 1.8.7 as well, and the exact same patch fixes the problem.

[shugo (Shugo Maeda)]

• Assignee set to shugo (Shugo Maeda)

[shugo (Shugo Maeda)]

• Status changed from Open to Closed
• % Done changed from 0 to 100

This issue was solved with changeset r32114.
Mark, thank you for reporting this issue.
Your contribution to Ruby is greatly appreciated.
May Ruby be with you.

---

• lib/net/imap.rb (search_response): parses SEARCH responses from
  the Yahoo IMAP server correctly. patched by Mark Nadig. [Bug #4509]