Misc #20488: Document source file size restrictions

• Status: Closed
• Assignee: -

[kddnewton (Kevin Newton)]

I was hoping we might be able to decide on an official maximum size for a source file in terms of bytes/lines/columns/etc.

Ruby uses fixed integer sizes to represent line numbers, column numbers, offsets, etc. These can overflow with files that are too big. Sometimes they will fail with cryptic messages like:

ruby: negative string size (or size too big) (ArgumentError)

(That's for 2^31 "a"s in a row.) Note that for python for the same file you will get:

OverflowError: Parser column offset overflow - source line is too big

and for perl you will get:

Identifier too long at test.pl line 1.

For files with 2^32 newlines, Ruby just crashes on my machine with the current parser (prism finishes, but its newline counter overflows so it gets all the offsets wrong).

Would it be okay to say:

• maximum line: 31 bits
• maximum column: 32 bits
• maximum file byte size: 32 bits

This would also help with memory savings — I would like to only use 32 bits for offsets in the file, as opposed to the current 64 bits I'm using (which seems unnecessarily large).

[matz (Yukihiro Matsumoto)]

The limit will be declared (for CRuby):

• maximum line: signed 32 bits (eval uses negative line numbers)
• maximum column: unsigned 32 bits
• maximum file byte size: unsigned 32 bits

Matz.

[kddnewton (Kevin Newton)]

• Status changed from Open to Closed

Thank you!