You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Add a utility to map originator defined license strings to SPDX license format #106
Package managers or text files may declare a license string that may not be an SPDX license format. For example, many projects declare their license as BSD but it is unclear which BSD.
oss-review-toolkit has an implementation that comes reasonably close.
@nishakm Thank you! This is indeed useful but such a mapping would be most useful in the context of a certain package manifest and type and its actual raw license declaration data and IMHO in the context of license detection. As such the mappings from ORT are generic and would better if they were package-type specific (e.g. capturing the conventions of npms, Maven, etc).
But my main point is that I am not sure there is a place in this library where I could make use of this data. When you craft SPDX documents, you need to have already the proper normalized licenses ids and license expressions so there would not be a place to plug this in. This would have to be done/used before.
Therefore I think this is something that could be best used in two places:
as a list of aliases when parsing license expressions with the license-expression(https://github.com/nexB/license-expression/) library. This is something that is explicitly supported there and it would be great to have a list of aliases. This could be a list of generic aliases alright
as mappings used when you parse the declared licenses of package manifests in the scancode-toolkit. This probably would be another good place.
@nishakm Thank you! This is indeed useful but such a mapping would be most useful in the context of a certain package manifest and type and its actual raw license declaration data and IMHO in the context of license detection. As such the mappings from ORT are generic and would better if they were package-type specific (e.g. capturing the conventions of npms, Maven, etc).
Agreed. It would be a much larger undertaking then :)
But my main point is that I am not sure there is a place in this library where I could make use of this data. When you craft SPDX documents, you need to have already the proper normalized licenses ids and license expressions so there would not be a place to plug this in. This would have to be done/used before.
I was going to go off and create an independent python module for this purpose, but I was told by the SPDX folks that this repo would be a good central location for such a module.
Therefore I think this is something that could be best used in two places:
as a list of aliases when parsing license expressions with the license-expression(https://github.com/nexB/license-expression/) library. This is something that is explicitly supported there and it would be great to have a list of aliases. This could be a list of generic aliases alright
The use case is basically translating what looks like the appropriate license declared somewhere in the artifact into the license expression. So I'm not sure how a license expression parser would help here.
as mappings used when you parse the declared licenses of package manifests in the scancode-toolkit. This probably would be another good place.
I'd like it to be independent of scancode-toolkit because other projects who want the same thing can use it as well. But if there are already mappings in here, where in the project might I find it?
@nishakm how about creating language-agnostic license mappings in JSON / YAML format that are initially populated with the existing mappings from ORT / the ones that nexB has, and put them in a "neutral" place like probably a new repository at https://github.com/spdx?
@nishakm how about creating language-agnostic license mappings in JSON / YAML format that are initially populated with the existing mappings from ORT / the ones that nexB has, and put them in a "neutral" place like probably a new repository at https://github.com/spdx?
This sounds good to me if @kestewart and @goneall are OK with creating an independent repo under the spdx github namespace. I am not a license geek but I already feel like a 1:1 mapping is not going to get all the way there. There needs to be some kind of string formatting or downstream processing from there. But a 1:1 mapping to start off would be great!
how about creating language-agnostic license mappings in JSON / YAML format that are initially populated with the existing mappings from ORT / the ones that nexB has, and put them in a "neutral" place like probably a new repository at https://github.com/spdx?
I was exactly talking about this yesterday about this very ticket of @nishakm
I am not a license geek but I already feel like a 1:1 mapping is not going to get all the way there. There needs to be some kind of string formatting or downstream processing from there. But a 1:1 mapping to start off would be great!
The right approach would be indeed not to have a 1:1 mapping but something imho which would be this way:
Given these:
a license string and or structured data snippet (to account for npm old styles and Maven structures) as found in a package manifest.
a package manager type (e.g. a Package URL type)
Then we map to:
a license expression
some indication of confidence (say between 0 and 100) for the accuracy of this mapping
The use case is basically translating what looks like the appropriate license declared somewhere in the artifact into the license expression. So I'm not sure how a license expression parser would help here.
This would be important to validate that the license expressions are correct and in a canonical form. That's a something to add as a test of sorts
I recall discussing this on one of the SPDX calls and I do recall talking about adding the issue to the tools.
I also agree with the above comments that we should have a separate data mapping repo along with tools implementations that support the mapping. I don't recall the discussion precisely, but I don't think anyone had a concern with a neutral mapping repo - just a concern about adding it to the spec since the mapping may be updated quite frequently.
I like the idea of the mapping repo and can use this in the SPDX Java Tools as well.
Activity
pombredanne commentedon Jun 7, 2019
@nishakm Thank you! This is indeed useful but such a mapping would be most useful in the context of a certain package manifest and type and its actual raw license declaration data and IMHO in the context of license detection. As such the mappings from ORT are generic and would better if they were package-type specific (e.g. capturing the conventions of npms, Maven, etc).
But my main point is that I am not sure there is a place in this library where I could make use of this data. When you craft SPDX documents, you need to have already the proper normalized licenses ids and license expressions so there would not be a place to plug this in. This would have to be done/used before.
Therefore I think this is something that could be best used in two places:
as a list of aliases when parsing license expressions with the license-expression(https://github.com/nexB/license-expression/) library. This is something that is explicitly supported there and it would be great to have a list of aliases. This could be a list of generic aliases alright
as mappings used when you parse the declared licenses of package manifests in the scancode-toolkit. This probably would be another good place.
Feedback welcomed!
pombredanne commentedon Jun 7, 2019
@sschuberth btw, this mapping is not right: https://github.com/heremaps/oss-review-toolkit/blob/518e17c0b1385cc403960cfdfdff69e76240cb27/spdx-utils/src/main/kotlin/SpdxDeclaredLicenseMapping.kt#L120
"CDDL v1.0 / GPL v2 dual license" to (CDDL_1_0 and GPL_2_0_ONLY),should be instead an OR and not and AND IMHOsschuberth commentedon Jun 7, 2019
Thanks @pombredanne for the hint, @mnonnenmacher any comments?
nishakm commentedon Jun 8, 2019
Agreed. It would be a much larger undertaking then :)
The use case is basically translating what looks like the appropriate license declared somewhere in the artifact into the license expression. So I'm not sure how a license expression parser would help here.
I'd like it to be independent of scancode-toolkit because other projects who want the same thing can use it as well. But if there are already mappings in here, where in the project might I find it?
SpdxDeclaredLicenseMapping: Generally map "/" to "or"
SpdxDeclaredLicenseMapping: Generally map "/" to "or"
SpdxDeclaredLicenseMapping: Generally map "/" to "or"
sschuberth commentedon Sep 25, 2019
@nishakm how about creating language-agnostic license mappings in JSON / YAML format that are initially populated with the existing mappings from ORT / the ones that nexB has, and put them in a "neutral" place like probably a new repository at https://github.com/spdx?
nishakm commentedon Sep 25, 2019
This sounds good to me if @kestewart and @goneall are OK with creating an independent repo under the spdx github namespace. I am not a license geek but I already feel like a 1:1 mapping is not going to get all the way there. There needs to be some kind of string formatting or downstream processing from there. But a 1:1 mapping to start off would be great!
pombredanne commentedon Sep 25, 2019
@sschuberth re:
I was exactly talking about this yesterday about this very ticket of @nishakm
@nishakm re:
The right approach would be indeed not to have a 1:1 mapping but something imho which would be this way:
Given these:
Then we map to:
Let me create the repo :)
pombredanne commentedon Sep 25, 2019
@nishakm @sschuberth there it is: https://github.com/spdx/package-licenses-mapping
@nishakm I invited you there as a committer too.
pombredanne commentedon Sep 25, 2019
See spdx/package-licenses-mapping#1 which is the continuation for this ticket
pombredanne commentedon Sep 25, 2019
@nishakm one of the reason that having a separate repo is better is that it could be reused in many places beyond this Python tool repo.
nishakm commentedon Sep 25, 2019
It's what I asked for initially. It was suggested that I file an issue here :)
pombredanne commentedon Sep 25, 2019
@nishakm that's fine, you could have made it clear in the ticket
pombredanne commentedon Sep 25, 2019
@nishakm re:
This would be important to validate that the license expressions are correct and in a canonical form. That's a something to add as a test of sorts
goneall commentedon Sep 28, 2019
Just catching up on the issue .
I recall discussing this on one of the SPDX calls and I do recall talking about adding the issue to the tools.
I also agree with the above comments that we should have a separate data mapping repo along with tools implementations that support the mapping. I don't recall the discussion precisely, but I don't think anyone had a concern with a neutral mapping repo - just a concern about adding it to the spec since the mapping may be updated quite frequently.
I like the idea of the mapping repo and can use this in the SPDX Java Tools as well.