Feature #13020
closedZlib.gzip and Zlib.gunzip
Description
I added Zlib.deflate/inflate [Feature #4180] before, but writing/reading gzip is still too complex.
It should have shorthand method.
Updated by naruse (Yui NARUSE) almost 8 years ago
- Status changed from Open to Closed
Applied in changeset r57035.
Zlib.gzip and Zlib.gunzip [Feature #13020]
Encode and Decode gzip data without creating files.
Updated by normalperson (Eric Wong) almost 8 years ago
naruse@airemix.jp wrote:
I added Zlib.deflate/inflate [Feature #4180] before, but writing/reading gzip is still too complex.
It should have shorthand method.
I like the convenience, but I think encouraging use of
potentially large String buffers is dangerous.
It causes memory problems and even opens up DoS attacks.
So, I think it would be better if it used IO-like object
instead, to discourage slurping.
Maybe make it like IO.copy_stream:
Zlib.gzip(src_io, dst_io)
Zlib.gunzip(src_io, dst_io)
Slurpers can still use StringIO if they want to.
In general, I am against non-streaming APIs and want to
encourage processing data in predictable sizes to avoid
fragmentation and OOM.
Updated by znz (Kazuhiro NISHIYAMA) almost 8 years ago
- Related to Bug #13021: `Zlib.gunzip` modifies argument String added
Updated by naruse (Yui NARUSE) almost 8 years ago
Eric Wong wrote:
naruse@airemix.jp wrote:
I added Zlib.deflate/inflate [Feature #4180] before, but writing/reading gzip is still too complex.
It should have shorthand method.I like the convenience, but I think encouraging use of
potentially large String buffers is dangerous.It causes memory problems and even opens up DoS attacks.
So, I think it would be better if it used IO-like object
instead, to discourage slurping.Maybe make it like IO.copy_stream:
Zlib.gzip(src_io, dst_io)
Zlib.gunzip(src_io, dst_io)Slurpers can still use StringIO if they want to.
In general, I am against non-streaming APIs and want to
encourage processing data in predictable sizes to avoid
fragmentation and OOM.
There's already Zlib.deflate/inflate, which are memory based API.
And gzip also uses deflate/inflate.
Note that gzip has the length of original data at the end of data. (isize=origsize / (2^32))
Anyway IO.copy_stream like API sounds reasonable.
Current API needs to change interface to use keyword arguments to avoid using 2nd argument...