Bug #11844: Please update unicode-licensed files (license issue)

• Status: Closed
• Assignee: naruse (Yui NARUSE)
• Target version: -
• ruby -v: 2.2.3
• Backport: 2.1: DONE, 2.2: DONE, 2.3: DONE

[zeha (Christian Hofstaedtler)]

Various files in enc/trans are old copies of Unicode.org files that forbid (re-)distribution. AFAICT, Unicode, Inc. has published newer versions of these files to fix this issue.
New versions could possibly be found on http://www.unicode.org/Public/MAPPINGS/OBSOLETE/EASTASIA/JIS/ and elsewhere under http://www.unicode.org/Public/

ext/json/generator/generator.c also has a restrictive Unicode, Inc. license blurb. Please fix this as well.

[duerst (Martin Dürst)]

Christian Hofstaedtler wrote:

Various files in enc/trans are old copies of Unicode.org files that forbid (re-)distribution. AFAICT, Unicode, Inc. has published newer versions of these files to fix this issue.

It looks like you have problems with the following text, e.g. in enc/trans/JIS/JISX0212%UCS.src:

#	Recipient is granted the right to make copies in any form for
#	internal distribution and to freely use the information supplied
#	in the creation of products supporting Unicode.  Unicode, Inc.
#	specifically excludes the right to re-distribute this file directly
#	to third parties or other organizations whether for profit or not.

However, this doesn't create any problems. We explicitly use the information to create products
(namely Ruby) supporting Unicode. We also don't distribute the file directly; there is an implementation-specific header at the start of the file, and another implementation-specific line at the end. Also, please note that at least some files (e.g. UCS%JISX0212.src) are changed much more, as the order is no longer the one claimed in the header ("The entries are in JIS X 0212 order"). In addition, each line comes with a '=' between the codepoints being converted, whereas http://www.unicode.org/Public/MAPPINGS/OBSOLETE/EASTASIA/JIS/JIS0212.TXT only has a tab between the codepoints, but comments on each line.

So at least for these files, what we do doesn't conflict with the conditions on these files at all. (Even if it did, the fact that Unicode changed the copyright to be more liberal is a strong indication that they wouldn't want to claim any violation anymore.)

New versions could possibly be found on http://www.unicode.org/Public/MAPPINGS/OBSOLETE/EASTASIA/JIS/ and elsewhere under http://www.unicode.org/Public/

I haven't checked whether they are available for all of our data. But that's not the problem. The problem is that our files are already changed quite a bit, and it would be difficult to make sure we apply exactly the same changes that we applied to the original files.

I think if this is really a problem for you (I hope that after the explanation above it isn't anymore), then we should think about modifying the header, not the data, to reduce the danger of regressions.

ext/json/generator/generator.c also has a restrictive Unicode, Inc. license blurb. Please fix this as well.

Restrictive? Have you actually read the blurb? Here's what it says:

 * Limitations on Rights to Redistribute This Code
 *
 * Unicode, Inc. hereby grants the right to freely use the information
 * supplied in this file in the creation of products supporting the
 * Unicode Standard, and to make copies of this file in any form
 * for internal or external distribution as long as this notice
 * remains attached.

So it's okay to make copies of that file in any form for internal or external distribution.
But it's not okay to remove (or change) the copyright notice. So we are exactly in compliance, and you shouldn't have any problems whatsoever with it.

[duerst (Martin Dürst)]

• Assignee set to akr (Akira Tanaka)

[akr (Akira Tanaka)]

• Assignee deleted (akr (Akira Tanaka))

I'm not the appropriate person responsible to this issue.

For example, enc/trans/JIS/JISX0212%UCS.src is added by naruse.

[duerst (Martin Dürst)]

• Status changed from Open to Feedback

As far as I have explained, there are no copyright problems. Set to feedback to confirm with original commenter.

[zeha (Christian Hofstaedtler)]

The non-free problem is with this line: "supplied in this file in the creation of products supporting the Unicode Standard" - which implies that you can't use this file to produce stuff not supporting / not complying with "the Unicode Standard" (which itself may change at any time). (This is a problem for redistributors that may opt to use the GPL for Ruby and also causes DFSG issues.)

That's one of the reasons the Unicode default copyright text has changed multiple times in the past (current version: http://unicode.org/copyright.html). But it's not clear if the new copyright applies to files carrying the old header. Given that Unicode, Inc has made available (some) new files, with the exact same data, but with the new copyright info, it's being assumed that one can't just take the old files and slap the new header onto them.

[zeha (Christian Hofstaedtler)]

Another thing - you're saying your copies of the files were modified. If the -data- inside them was modified, does that mean you're not compliant because you're not "supporting the Unicode Standard" as defined back then?

[duerst (Martin Dürst)]

• Assignee set to naruse (Yui NARUSE)

We have looked at this issue at today's developers' meeting

Christian Hofstaedtler wrote:

The non-free problem is with this line: "supplied in this file in the creation of products supporting the Unicode Standard" - which implies that you can't use this file to produce stuff not supporting / not complying with "the Unicode Standard" (which itself may change at any time). (This is a problem for redistributors that may opt to use the GPL for Ruby and also causes DFSG issues.)

Ok. We understand that it's not a problem for Ruby itself, but for redistributors. But we of course want to help redistributors if we can.

That's one of the reasons the Unicode default copyright text has changed multiple times in the past (current version: http://unicode.org/copyright.html). But it's not clear if the new copyright applies to files carrying the old header. Given that Unicode, Inc has made available (some) new files, with the exact same data, but with the new copyright info, it's being assumed that one can't just take the old files and slap the new header onto them.

Looking at the actual text in the copyright file, it says:
"""
EXHIBIT 1
UNICODE, INC. LICENSE AGREEMENT - DATA FILES AND SOFTWARE

Unicode Data Files include all data files under the directories http://www.unicode.org/Public/, http://www.unicode.org/reports/, and http://www.unicode.org/cldr/data/. Unicode Data Files do not include PDF online code charts under the directory http://www.unicode.org/Public/."
"""

The files concerned are under http://www.unicode.org/Public/MAPPINGS/, so the new copyright applies. Also, we noted that the actual files have been updated to mention the change to the copyright in the file itself. So we plan to appropriately change the files on our side, too.

[naruse (Yui NARUSE)]

• Status changed from Feedback to Closed

Applied in changeset r54129.

---

• enc/trans/JIS: update Unicode's notice. [Bug #11844]

[zeha (Christian Hofstaedtler)]

Thank you very much for fixing this issue in enc/trans/JIS. What about ext/json/generator/generator.c?

[hsbt (Hiroshi SHIBATA)]

What about ext/json/generator/generator.c

It's not original file of Ruby. Please report upstream. see https://github.com/flori/json

[usa (Usaku NAKAMURA)]

• Backport changed from 2.0.0: UNKNOWN, 2.1: UNKNOWN, 2.2: UNKNOWN to 2.1: REQUIRED, 2.2: REQUIRED, 2.3: REQUIRED

[usa (Usaku NAKAMURA)]

• Backport changed from 2.1: REQUIRED, 2.2: REQUIRED, 2.3: REQUIRED to 2.1: DONE, 2.2: REQUIRED, 2.3: REQUIRED

ruby_2_1 r54283 merged revision(s) 54129,54130,54132.

[nagachika (Tomoyuki Chikanaga)]

• Backport changed from 2.1: DONE, 2.2: REQUIRED, 2.3: REQUIRED to 2.1: DONE, 2.2: DONE, 2.3: REQUIRED

Backported into ruby_2_2 branch at r54334.

[naruse (Yui NARUSE)]

• Backport changed from 2.1: DONE, 2.2: DONE, 2.3: REQUIRED to 2.1: DONE, 2.2: DONE, 2.3: DONE

ruby_2_3 r54381 merged revision(s) 54129,54130,54132.

[jaruga (Jun Aruga)]

I could not find the license information for enc/trans/GB/*.src from the mapping page. [1]
So, I am asking to unicode.org about what is the correct license. [2]

[1] http://www.unicode.org/Public/MAPPINGS/OBSOLETE/EASTASIA/
[2] http://www.unicode.org/mail-arch/unicode-ml/y2016-m06/0078.html

[jaruga (Jun Aruga)]

Jun Aruga wrote:

I could not find the license information for enc/trans/GB/*.src from the mapping page. [1]
So, I am asking to unicode.org about what is the correct license. [2]

[1] http://www.unicode.org/Public/MAPPINGS/OBSOLETE/EASTASIA/
[2] http://www.unicode.org/mail-arch/unicode-ml/y2016-m06/0078.html

I am going to open/write new issue page for this kind of issue because above issue have already been closed.
Sorry for the confusing you, guys.

[jaruga (Jun Aruga)]

I am going to open/write new issue page for this kind of issue because above issue have already been closed.

Created, though it is focusing on LEGAL file.
https://bugs.ruby-lang.org/issues/12529

[duerst (Martin Dürst)]

• Related to Misc #12529: LEGAL file covering all the license information within Ruby added

[duerst (Martin Dürst)]

zeha (Christian Hofstaedtler) wrote:

The non-free problem is with this line: "supplied in this file in the creation of products supporting the Unicode Standard" - which implies that you can't use this file to produce stuff not supporting / not complying with "the Unicode Standard" (which itself may change at any time). (This is a problem for redistributors that may opt to use the GPL for Ruby and also causes DFSG issues.)

Just for the record, with r59715 it still says "supporting Unicode", but our understanding is that we used the original files to create a product (Ruby) supporting the Unicode standard, but the new files that were created are no longer under Unicode copyright, meaning that downstream third-party users should also be fine.