Bug #2966: CSV reports illegal quoting on line ...

• Status: Rejected
• Assignee: JEG2 (James Gray)
• ruby -v: ruby 1.9.1p376 (2009-12-07 revision 26041) [i386-mswin32]

[sgoings (Seth Goings)]

=begin
I have a line of code:
@array = CSV.read("#{@dir}/#{@file}")

Which eventually reads line 24 (below) in file attached to this bug:
BBBP,004,/etc/release,"LSB_VERSION="core-2.0-noarch:core-3.0-noarch:core-2.0-x86_64:core-3.0-x86_64""

This raises a MalformedCSVError, "Illegal quoting on line #{lineno + 1}."

I understand the issue of the nested quotes and how the line is being parsed, but I don't see an issue with how this CSV line is written. Should this sort of line be parse-able with a csv library? If not, can/should the library fail nicely, instead of ruining the whole "read" call?
=end

[JEG2 (James Gray)]

• Category set to lib
• Status changed from Open to Rejected

=begin

Which eventually reads line 24

Unfortunately, that line is not valid CSV data. In CSV, quotes must be escaped (by doubling them) inside of a quoted field. Given that, the field would need to be: …,"LSB_VERSION=""core-2.0…

If not, can/should the library fail nicely, instead of ruining the whole "read" call?

Without being able to correctly parse that line, it's impossible to correctly parse any data that comes after it. You asked the library to read so data and it has become impossible to do so, so I feel the correct behavior is to raise an error. If you will be feeding CSV invalid data, you will need to rescue these errors and take appropriate actions.

=end

[sgoings (Seth Goings)]

=begin
Is there a reason why this is able to be read in FasterCSV (prior to 1.9) then?
=end

[JEG2 (James Gray)]

=begin

Is there a reason why this is able to be read in FasterCSV (prior to 1.9) then?

You have located a bug in the new non-regular expression parser used by FasterCSV. It's not parsable by older versions (that don't have the bug):

$ ruby -rubygems -ve 'gem("fastercsv", "=1.4.0"); require "faster_csv"; p FCSV.read(ARGV.shift)[0..24]' performance_log.csv.xls
ruby 1.8.6 (2010-02-05 patchlevel 399) [i686-darwin10.2.0]
/usr/local/lib/ruby/gems/1.8/gems/fastercsv-1.4.0/lib/faster_csv.rb:1650:in shift': Illegal quoting on line 24. (FasterCSV::MalformedCSVError) from /usr/local/lib/ruby/gems/1.8/gems/fastercsv-1.4.0/lib/faster_csv.rb:1568:in loop'
from /usr/local/lib/ruby/gems/1.8/gems/fastercsv-1.4.0/lib/faster_csv.rb:1568:in shift' from /usr/local/lib/ruby/gems/1.8/gems/fastercsv-1.4.0/lib/faster_csv.rb:1513:in each'
from /usr/local/lib/ruby/gems/1.8/gems/fastercsv-1.4.0/lib/faster_csv.rb:1524:in to_a' from /usr/local/lib/ruby/gems/1.8/gems/fastercsv-1.4.0/lib/faster_csv.rb:1524:in read'
from /usr/local/lib/ruby/gems/1.8/gems/fastercsv-1.4.0/lib/faster_csv.rb:1241:in read' from /usr/local/lib/ruby/gems/1.8/gems/fastercsv-1.4.0/lib/faster_csv.rb:1191:in open'
from /usr/local/lib/ruby/gems/1.8/gems/fastercsv-1.4.0/lib/faster_csv.rb:1241:in `read'
from -e:1

=end

[sgoings (Seth Goings)]

=begin
Ok thanks for your help!
=end

[JEG2 (James Gray)]

=begin
The bug you have found is fixed in the new FasterCSV 1.5.3 gem.
=end