Add gradation to license analysis

[david-a-wheeler]

Is your feature request related to a problem? Please describe.

Currently scorecard looks for a license, but it simply gives a 10 if there's a license found at all and a 0 if there's no license found at all. That doesn't give any gradation.

Describe the solution you'd like

Add more gradations for licenses. I think scorecard was intended for OSS projects, so I propose giving "10" for projects with a license known to be OSS per OSI or Free Software as defined by the FSF. Otherwise, give it a 5. For projects on GitHub you can just reuse the GitHub analysis, as GitHub has an API that provides license info. You can see an example of this use in the CII Best Practices' github_basic_detective and floss_license_detective.

Further discussion here: #1038

Describe alternatives you've considered

Different people have different views of copyleft, I don't think scorecard should force a particular view of whether copyleft or not-copyleft is better (less risk). You could argue either way, which to me suggests you shouldn't make the argument. Even those who prefer BSD-style licenses will often use the Linux kernel (GPL), because how you use software is very important. As a result, I think "is it approved by OSI or FSF" should be enough, don't force further gradations.

Additional context

Again, further discussion here: #1038

[shissam]

I'd like to see this implements AND to help on this. I did some experiments this weekend with scorecard and added a githubrepo client called clients/githubrepo/licenses.go that uses go-github/v38 API github.RepositoryLicense to obtain the license information known to github via the RESTAPI

curl -D fred.txt -H "Accept: application/vnd.github+json" -H "Authorization: Bearer <GITHUB_TOKEN>" 'https://api.github.com/repos/<owner>/<repo>/license'

My proposal

• introduce clients/licenses.go which embodies this github api (similar to clients/languages.go) - it would be plural as it may come down to the point where gitlab returns multiple licenses for a repo (I have a real-world example).
• This github client would populate a structure like the following:

type License struct {
        Key      string // RepositoryLicense.GetLicense().GetKey()
        Name     string // RepositoryLicense.GetLicense().GetName()
        Path     string // RepositoryLicense.GetName()
        Size     int    // RepositoryLicense.GetSize()
        SPDXId   string // RepositoryLicense.GetLicense().GetSPDXID()
        Type     string // RepositoryLicense.GetType()
}

where (example from ossf/scorecard:

{
 Key:apache-2.0
 Name:Apache License 2.0
 Path:LICENSE
 Size:11356
 SPDXId:Apache-2.0 
Type:file
}

• modify checks/raw/license.go to follow one of two possible paths:
  -- pathOne try the repoClient (as in c.RepoClient.ListLicenses()), if a license is reported by github USE that and return
  -- pathTwo if the repoClient returns nil, continue with the legacy logic and return what is always returned
• there are a number of advantages to this approach:
  -- speedup: for github, my test of c.RepoClient.ListLicenses() returns in less than 5 seconds, when compared to other repos with many files (like torvalds/linux this check takes over a minute
  -- extensibility: this test with github is promising, and gitlab does appear to have a api for getting this information (https://docs.gitlab.com/ee/api/templates/licenses.html)
  -- confidence: using the github api for 'sensing' which specific license is in use I assume is higher (more review on their algorithm which might use NLP)--I tried a license on my site which changed the work Apache to Apacje in the complete Apache License 2.0 file and github still sensed it was APL-2.0 (very cool).
  -- standardization: I am encouraged by the use of the SPDXId in the github API, if it is true that gitlab using the same standard identifier for licenses - this would be great for the community
  -- usability: using scorecard to retrieve the specific license detected would a) make this information more readily available to the end-user; and b) would support develop pipelines that need to sense the license (by name) as a risk management activity without having to move to other means to acquire such license information
• with this approach, propose changing checker.LicenseData{}:

type License struct {
        Key      string // from clients.License.Key
        Name     string // from clients.License.Name
        Size     int    // clients.License.Size (Size maybe not here but in type File struct
        SPDXId   string // clients.License.SPDXId
        Attribution     AttributionType // license sourced from LicenseRepoAPI or ScorecardEngine
}
type LicenseData struct {
        Files []File
        License []License
}

Design Decisions (not all are needed to get this moving)

• scoring: which would occur in checks/evaluation/license.go would not change at this time - a file is a file, both pathOne and pathTwo would continue to return this information
  -- possible future approachs suitable for this proposal: when:
  --- Attribution is LicenseRepoAPI (i.e., pathOne) then consult a map to convert the SPDXId to a score
  --- Attribution is ScorecardEngine (i.e., pathTwo) then continue to affix scorecard max, later as pathTwo is improved to regex licenses into SPDXIds then the map can be used in the evaluation.

Some particulary interesting github repos with possible licenses[1]

• github.com/Fivium/FOXopen
  {Key:gpl-3.0 Name:GNU General Public License v3.0 Path:LICENSE Size:35147 SPDXId:GPL-3.0 Type:file}
  --NB: this site actually confuses github showing two licenses on the website yet the API only returns the first, hence it appears based on this one example, the API will (for now) only return one license file
• github.com/twardoch/varfonts-ofl
  NB: github API returns a 404 - hence pathTwo continues detecting a presence which github did not
• github.com/mozilla/openbadges-bakery
  {Key:mpl-2.0 Name:Mozilla Public License 2.0 Path:LICENSE-MPL-2.0 Size:16726 SPDXId:MPL-2.0 Type:file}
• github.com/IQAndreas/markdown-licenses
  {Key:unlicense Name:The Unlicense Path:unlicense.md Size:1275 SPDXId:Unlicense Type:file}
  NB: notice the Unlicense
• github.com/johnjago/awesome-uncopyright
  {Key:cc0-1.0 Name:Creative Commons Zero v1.0 Universal Path:LICENSE Size:6515 SPDXId:CC0-1.0 Type:file}
• github.com/USEPA/open-source-projects
  {Key:mit Name:MIT License Path:license.md Size:1124 SPDXId:MIT Type:file}

[1] see these examples: https://github.com/Fivium/FOXopen/blob/master/LICENSE-THIRD-PARTY.md