How GitHub Supports Vim License Detection, The Five Years Journey

Transcript

1. othree at COSCUP 2022 How GitHub Supports Vim License Detection The Five Years Journey
2. othree at COSCUP 2022 GitHub ⽀援 Vim License 的故事 歷時五年的開源貢獻
3. Who is othree • Web Developer • Mozillian (MozTW) • Blogger https://blog.othree.net • Speaker https://speakerdeck.com/othree • Vimmer • ❤ OSS
4. Before We Start • GitHub and Open Source License • Vim License
5. GitHub and Open Source License
9. License detection • Highlight selected License • Search by license • Only major licenses are supported • Detected by Licensee, a Ruby Gem
10. The Vim License
11. The Vim License • A special License only for Vim • GPL compatible according to Richard Stallman • Coupled with the Vim • Many Vim script uses Vim license, a.k.a “Same as Vim” • https://vimhelp.org/uganda.txt.html
12. VIM LICENSE I) There are no restrictions on distributing unmodified copies of Vim except that they must include this license text. You can also distribute unmodified parts of Vim, likewise unrestricted except that they must include this license text. You are also allowed to include executables that you made from the unmodified Vim sources, plus your own usage examples and Vim scripts. II) It is allowed to distribute a modified (or extended) version of Vim, including executables and/or source code, when the following four conditions are met: 1) This license text must be included unmodified. 2) The modified Vim must be distributed in one of the following five ways: a) If you make changes to Vim yourself, you must clearly describe in the distribution how to contact you. When the maintainer asks you (in any way) for a copy of the modified Vim you distributed, you must make your changes, including source code, available to the maintainer without fee. The maintainer reserves the right to include your changes in the official version of Vim. What the
13. VIM LICENSE I) There are no restrictions on distributing unmodified copies of Vim except that they must include this license text. You can also distribute unmodified parts of Vim, likewise unrestricted except that they must include this license text. You are also allowed to include executables that you made from the unmodified Vim sources, plus your own usage examples and Vim scripts. II) It is allowed to distribute a modified (or extended) version of Vim, including executables and/or source code, when the following four conditions are met: 1) This license text must be included unmodified. 2) The modified Vim must be distributed in one of the following five ways: a) If you make changes to Vim yourself, you must clearly describe in the distribution how to contact you. When the maintainer asks you (in any way) for a copy of the modified Vim you distributed, you must make your changes, including source code, available to the maintainer without fee. The maintainer reserves the right to include your changes in the official version of Vim. What the
14. II) It is allowed to distribute a modified (or extended) version of Vim, including executables and/or source code, when the following four conditions are met: 1) This license text must be included unmodified. 2) The modified Vim must be distributed in one of the following five ways: a) If you make changes to Vim yourself, you must clearly describe in the distribution how to contact you. When the maintainer asks you (in any way) for a copy of the modified Vim you distributed, you must make your changes, including source code, available to the maintainer without fee. The maintainer reserves the right to include your changes in the official version of Vim. What the maintainer will do with your changes and under what license they will be distributed is negotiable. If there has been no negotiation then this license, or a later version, also applies to your changes. The current maintainer is Bram Moolenaar <Bram@vim.org>. If this changes it will be announced in appropriate places (most likely vim.sf.net, www.vim.org and/or comp.editors). When it is completely impossible to contact the maintainer, the obligation to send him your changes ceases. Once the maintainer has confirmed that he has received your changes they will not have to be sent again. b) If you have received a modified Vim that was distributed as mentioned under a) you are allowed to further distribute it
15. a) If you make changes to Vim yourself, you must clearly describe in the distribution how to contact you. When the maintainer asks you (in any way) for a copy of the modified Vim you distributed, you must make your changes, including source code, available to the maintainer without fee. The maintainer reserves the right to include your changes in the official version of Vim. What the maintainer will do with your changes and under what license they will be distributed is negotiable. If there has been no negotiation then this license, or a later version, also applies to your changes. The current maintainer is Bram Moolenaar <Bram@vim.org>. If this changes it will be announced in appropriate places (most likely vim.sf.net, www.vim.org and/or comp.editors). When it is completely impossible to contact the maintainer, the obligation to send him your changes ceases. Once the maintainer has confirmed that he has received your changes they will not have to be sent again. b) If you have received a modified Vim that was distributed as mentioned under a) you are allowed to further distribute it unmodified, as mentioned at I). If you make additional changes the text under a) applies to those changes. c) Provide all the changes, including source code, with every copy of the modified Vim you distribute. This may be done in the form of a context diff. You can choose what license to use for new code you
16. received your changes they will not have to be sent again. b) If you have received a modified Vim that was distributed as mentioned under a) you are allowed to further distribute it unmodified, as mentioned at I). If you make additional changes the text under a) applies to those changes. c) Provide all the changes, including source code, with every copy of the modified Vim you distribute. This may be done in the form of a context diff. You can choose what license to use for new code you add. The changes and their license must not restrict others from making their own changes to the official version of Vim. d) When you have a modified Vim which includes changes as mentioned under c), you can distribute it without the source code for the changes if the following three conditions are met: - The license that applies to the changes permits you to distribute the changes to the Vim maintainer without fee or restriction, and permits the Vim maintainer to include the changes in the official version of Vim without fee or restriction. - You keep the changes for at least three years after last distributing the corresponding modified Vim. When the maintainer or someone who you distributed the modified Vim to asks you (in any way) for the changes within this period, you must make them available to him.
17. - You clearly describe in the distribution how to contact you. This contact information must remain valid for at least three years after last distributing the corresponding modified Vim, or as long as possible. e) When the GNU General Public License (GPL) applies to the changes, you can distribute the modified Vim under the GNU GPL version 2 or any later version. 3) A message must be added, at least in the output of the ":version" command and in the intro screen, such that the user of the modified Vim is able to see that it was modified. When distributing as mentioned under 2)e) adding the message is only required for as far as this does not conflict with the license used for the changes. 4) The contact information as required under 2)a) and 2)d) must not be removed or changed, except that the person himself can make corrections. III) If you distribute a modified version of Vim, you are encouraged to use the Vim license for your changes and make them available to the maintainer, including the source code. The preferred way to do this is by e-mail or by uploading the files to a server and e-mailing the URL. If the number of changes is small (e.g., a modified Makefile) e-mailing a context diff will do. The e-mail address to be used is
18. - You clearly describe in the distribution how to contact you. This contact information must remain valid for at least three years after last distributing the corresponding modified Vim, or as long as possible. e) When the GNU General Public License (GPL) applies to the changes, you can distribute the modified Vim under the GNU GPL version 2 or any later version. 3) A message must be added, at least in the output of the ":version" command and in the intro screen, such that the user of the modified Vim is able to see that it was modified. When distributing as mentioned under 2)e) adding the message is only required for as far as this does not conflict with the license used for the changes. 4) The contact information as required under 2)a) and 2)d) must not be removed or changed, except that the person himself can make corrections. III) If you distribute a modified version of Vim, you are encouraged to use the Vim license for your changes and make them available to the maintainer, including the source code. The preferred way to do this is by e-mail or by uploading the files to a server and e-mailing the URL. If the number of changes is small (e.g., a modified Makefile) e-mailing a context diff will do. The e-mail address to be used is
19. is able to see that it was modified. When distributing as mentioned under 2)e) adding the message is only required for as far as this does not conflict with the license used for the changes. 4) The contact information as required under 2)a) and 2)d) must not be removed or changed, except that the person himself can make corrections. III) If you distribute a modified version of Vim, you are encouraged to use the Vim license for your changes and make them available to the maintainer, including the source code. The preferred way to do this is by e-mail or by uploading the files to a server and e-mailing the URL. If the number of changes is small (e.g., a modified Makefile) e-mailing a context diff will do. The e-mail address to be used is <maintainer@vim.org> IV) It is not allowed to remove this license from the distribution of the Vim sources, parts of it or from a modified version. You may use this license for previous Vim releases instead of the license that they came with, at your option.
21. The Vim License • A lot of word Vim in the text • The Vim Maintainer
22. VIM LICENSE I) There are no restrictions on distributing unmodified copies of Vim except that they must include this license text. You can also distribute unmodified parts of Vim, likewise unrestricted except that they must include this license text. You are also allowed to include executables that you made from the unmodified Vim sources, plus your own usage examples and Vim scripts. II) It is allowed to distribute a modified (or extended) version of Vim, including executables and/or source code, when the following four conditions are met: 1) This license text must be included unmodified. 2) The modified Vim must be distributed in one of the following five ways: a) If you make changes to Vim yourself, you must clearly describe in the distribution how to contact you. When the maintainer asks you (in any way) for a copy of the modified Vim you distributed, you must make your changes, including source code, available to the maintainer without fee. The maintainer reserves the right to include your changes in the official version of Vim. What the maintainer will do with your changes and under what license they will be distributed is negotiable. If there has been no negotiation then this license, or a later version, also applies to your changes. The current maintainer is Bram Moolenaar <Bram@vim.org>. If this changes it will be announced in appropriate places (most likely vim.sf.net, www.vim.org and/or comp.editors). When it is completely impossible to contact the maintainer, the obligation to send him your changes ceases. Once the maintainer has confirmed that he has received your changes they will not have to be sent again. b) If you have received a modified Vim that was distributed as mentioned under a) you are allowed to further distribute it unmodified, as mentioned at I). If you make additional changes the text under a) applies to those changes. c) Provide all the changes, including source code, with every copy of the modified Vim you distribute. This may be done in the form of a context diff. You can choose what license to use for new code you add. The changes and their license must not restrict others from making their own changes to the official version of Vim. d) When you have a modified Vim which includes changes as mentioned under c), you can distribute it without the source code for the changes if the following three conditions are met: - The license that applies to the changes permits you to distribute the changes to the Vim maintainer without fee or restriction, and permits the Vim maintainer to include the changes in the official version of Vim without fee or restriction. - You keep the changes for at least three years after last distributing the corresponding modified Vim. When the maintainer or someone who you distributed the modified Vim to asks you (in any way) for the changes within this period, you must make them available to him. - You clearly describe in the distribution how to contact you. This contact information must remain valid for at least three years after last distributing the corresponding modified Vim, or as long as possible. e) When the GNU General Public License (GPL) applies to the changes, you can distribute the modified Vim under the GNU GPL version 2 or any later version. 3) A message must be added, at least in the output of the ":version" command and in the intro screen, such that the user of the modified Vim is able to see that it was modified. When distributing as mentioned under 2)e) adding the message is only required for as far as this does not conflict with the license used for the changes. 4) The contact information as required under 2)a) and 2)d) must not be removed or changed, except that the person himself can make corrections. III) If you distribute a modified version of Vim, you are encouraged to use the Vim license for your changes and make them available to the maintainer, including the source code. The preferred way to do this is by e-mail or by uploading the files to a server and e-mailing the URL. If the number of changes is small (e.g., a modified Makefile) e-mailing a context diff will do. The e-mail address to be used is <maintainer@vim.org> IV) It is not allowed to remove this license from the distribution of the Vim sources, parts of it or from a modified version. You may use this license for previous Vim releases instead of the license that they came with, at your option.
23. Motivation
24. Guide to Contribution
29. First Issue
39. Contribute Again
41. Requirements 1. The license must have an SPDX identifier If your license isn't registered with SPDX, please request that it be added 2. The license must be listed on one of the following approved lists of licenses: * List of OSI approved licenses * GNU's list of free licenses 
 (*note: the license must be listed in one of the three "free" categories*) * Open Definition's list of conformant licenses 3. The license must be used in at least *1,000* public repositories. This may be documented, for example, with a GitHub code search
42. 1. The license must have an SPDX identifier If your license isn't registered with SPDX, please request that it be added 2. The license must be listed on one of the following approved lists of licenses: * List of OSI approved licenses * GNU's list of free licenses 
 (*note: the license must be listed in one of the three "free" categories*) * Open Definition's list of conformant licenses 3. The license must be used in at least *1,000* public repositories. This may be documented, for example, with a GitHub code search
43. 1. The license must have an SPDX identifier If your license isn't registered with SPDX, please request that it be added 2. The license must be listed on one of the following approved lists of licenses: * List of OSI approved licenses * GNU's list of free licenses 
 (*note: the license must be listed in one of the three "free" categories*) * Open Definition's list of conformant licenses 3. The license must be used in at least *1,000* public repositories. This may be documented, for example, with a GitHub code search 4. 3 notable projects using the license must be identified. These
44. 2. The license must be listed on one of the following approved lists of licenses: * List of OSI approved licenses * GNU's list of free licenses 
 (*note: the license must be listed in one of the three "free" categories*) * Open Definition's list of conformant licenses 3. The license must be used in at least *1,000* public repositories. This may be documented, for example, with a GitHub code search 4. 3 notable projects using the license must be identified. These must have straightforward LICENSE files which serve as examples newcomers can follow and that could be detected by licensee if it knew about the license.
45. * GNU's list of free licenses 
 (*note: the license must be listed in one of the three "free" categories*) * Open Definition's list of conformant licenses 3. The license must be used in at least *1,000* public repositories. This may be documented, for example, with a GitHub code search 4. 3 notable projects using the license must be identified. These must have straightforward LICENSE files which serve as examples newcomers can follow and that could be detected by licensee if it knew about the license.
50. 3 Notable Projects • Vim • pathogen.vim by Tim Pope • TBD
55. Substitutions
56. Substitutions Vim → software? package? project? Vim Maintainer → softwareMaintainer? …etc
57. Should I?
59. Yes, But , But
60. Not The Vim Maintainer
61. Everything is Ready
63. The Licensee
64. Licensee • A Ruby Gem to detect under what license a project is distributed • Use data from choosealicense.com • Only detects picked licenses • Supports LICENSE, package fi le, statements in README, SPDX fi le …
65. The matching method • Exact match(hash) • Sørensen–Dice coe ff i cient (aka Dice) • Similarity of two sets(word set) • String length
67. The Issue • There are too many [project] in the license text • After replaced all substitutions to Vim. The con fi dence < 98
68. Possible solutions • Add tolerance to the speci fi ed license • Use regex if there are substitutions • Dynamic tolerance based on the substitutions
72. SPDX License List
73. Software Package Data Exchange (SPDX) • Open standard for communicating software bill of material information • From Linux Foundation • An ISO standard (ISO/IEC 5962:2021) • Example
74. SPDX License List • Most popular standard by SPDX • Standardize license full name, text, identi fi er • A canonical permanent URL for each license • Supported by most eco system(ex: npm, composer, cabal)
78. First Pull Request
83. The Result • Mike Linksvayer already create a PR to fi x the issue • Add extra tolerance to substitution fi elds
85. vim-license.dev
86. Vim License Gen • A web app help you generate Vim License Text fi le with custom project name
90. The 2nd/3rd Pull Request
91. • Add LICENSE fi le to the notable projects • vim • pathogen.vim
93. The Main Pull Request
94. 3 notable projects • Vim • pathogen.vim by Tim Pope • vim-license-gen (aka vim-license.dev)
96. Merged !!
99. Wait for Licensee Update
101. Wait for Licensee Release
103. Wait for GitHub to Upgrade
105. Wait and Wait • Give zero pressure to the GitHub • Wait and wait • Then one day, I test the Licensee again
108. The Cause • The change of the similarity algorithm • No test case for Vim License
109. Fix Licensee
110. The Solution • Update the algorithm or manual decision parameter • Modi fi ed a parameter from 10 to 13 • Add test case for Vim License
112. The Result • Mike Linksvayer already create a PR to fi x the issue • For each substitution fi eld, provide 2 ch space not counting to the penalty • Merged at Oct 2020
114. Wait Again
116. Feb 2021
117. GitHub Support
120. Wait Again and Again
122. Jan 2022
123. GitHub Search by License
125. Vim’s License Still Not Correct
128. GitHub Support
134. Summary
135. • 2 issues • 5 pull requests • 1 mailing-list thread • 1 new web app • 2 GitHub support request • ~ 5 years
136. Take away
137. • How to start a CONTRIBUTION and how to communicate • What is SPDX and SPDX License List • What is Licensee and how it work • The deep understand of Vim License
138. Thanks for You Listening
139. Questions?