Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Generate Dash License Report #3

Merged
merged 3 commits into from
Sep 12, 2023

Conversation

wba2hi
Copy link
Contributor

@wba2hi wba2hi commented Sep 11, 2023

Closes: #2

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@@ -22,3 +44,40 @@ subprojects {
systemProperties = systemPropertiesMap
}
}

tasks.register("mergeDashFiles") {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would have prefered to put this into "dash.gradle.kts" to make it "re-usable" but unfortunately this task is only needed by the root project,... maybe it makes sense to think about a convention like naming this like dash.root.gradle.kts or so

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree as soon as it makes sense to extract this one. Currently it is still pretty clean.

unameOut="$(uname -s)"
case "${unameOut}" in
Linux*) GREP="grep";; # Linux
Darwin*) GREP="ggrep";; # Mac
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To test this on Mac ggrep needs to be installed

Comment on lines +50 to +57
./gradlew "$projectName":dependencies \
| ${GREP} -Poh "(?<=\-\-\- ).*" \
| ${GREP} -Pv "\([nc\*]\)" \
| ${GREP} -Pv "FAILED" \
| ${GREP} -Pv "project :[a-zA-Z0-9]+" \
| perl -pe 's/([\w\.\-]+):([\w\.\-]+):(?:[\w\.\-]+ -> )?([\w\.\-]+).*$/$1:$2:$3/gmi;t' \
| perl -pe 's/([\w\.\-]+):([\w\.\-]+) -> ([\w\.\-]+).*$/$1:$2:$3/gmi;t' \
| sort -u \
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

there is most probably a lot of potential to "optimize"/"shrink" these terms / number of lines. I tried some stuff, but always lead to something breaking. Reducing number of lines obviously also means increasing the complexity to understand each terms, so I left it for now.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fine for me for now :)

@@ -22,3 +44,40 @@ subprojects {
systemProperties = systemPropertiesMap
}
}

tasks.register("mergeDashFiles") {
Copy link
Contributor Author

@wba2hi wba2hi Sep 11, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • task "createDashFile" is registered per project and stores it's data in rootProject/build/oss/
  • task "mergeDashFiles" is registered on the rootProject, takes each of the project-specific "reports", aggregates them and also stores the aggregation in rootProject/build/oss/all

execute it using ./gradlew clean mergeDashFiles

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems like this could be a good code comment. Maybe add something like this in the next PR.

@wba2hi
Copy link
Contributor Author

wba2hi commented Sep 11, 2023

@Chrylo Ready for Review

@SebastianSchildt
Copy link
Contributor

Not sure the Dash data is "correct", at least there are a LOT of things where sources could not be found (i'd assign you in IP tracker, but so far your user seems not to be visible there)

As an example check this failed ticket: https://gitlab.eclipse.org/eclipsefdn/emo-team/iplab/-/issues/10457

vs. this working older one https://gitlab.eclipse.org/eclipsefdn/emo-team/iplab/-/issues/10217

Is this another "Kind" of dependency? or is our dash input not optimal?

So far there seems to be around 20 issues where sources are missing https://gitlab.eclipse.org/eclipsefdn/emo-team/iplab/-/issues/?sort=created_date&state=opened&author_username=sschildt&label_name%5B%5D=Help%20Wanted&first_page_size=100
, might become more, once Dash comes aorund to scan the other issues

(Check all issues opened in my name: https://gitlab.eclipse.org/eclipsefdn/emo-team/iplab/-/issues/?sort=created_date&state=opened&author_username=sschildt&first_page_size=100

@SebastianSchildt
Copy link
Contributor

Seems to be getting better now, wayne worked his magic and teached dash bot new tricks: https://gitlab.eclipse.org/eclipsefdn/emo-team/iplab/-/issues/10466 let's wait if it can churn through

@wba2hi
Copy link
Contributor Author

wba2hi commented Sep 12, 2023

Not sure the Dash data is "correct", at least there are a LOT of things where sources could not be found (i'd assign you in IP tracker, but so far your user seems not to be visible there)

It's just as Wayne stated. Some of the android sources are only available on the google maven repository. However for older versions of libraries google didn't upload sources at all. So there might still be some libraries and versions where the sources simply can not be found except maybe by scramming through the aosp sources directly.

I think some of the libraries have a "Android Software Development Kit" License (https://developer.android.com/studio/terms.html) which allows the use of said libraries, but is not open source (maps and navigation and stuff where google basically generates money with).

Is this another "Kind" of dependency? or is our dash input not optimal?

Unfortunately no and (most probably) no. Can't / won't guarantee 100% perfectness as I'm relying on code snippets here where I had to make adaptions to, but I checked them by sample and the results look "fine".

So far there seems to be around 20 issues where sources are missing https://gitlab.eclipse.org/eclipsefdn/emo-team/iplab/-/issues/?sort=created_date&state=opened&author_username=sschildt&label_name%5B%5D=Help%20Wanted&first_page_size=100 , might become more, once Dash comes aorund to scan the other issues

Either this filter does not work, or they are all identified by now? The filter produces no results for me

@SebastianSchildt
Copy link
Contributor

I feel - except following up on how the license clear in Eclipses backend - from a technical perspective this PR works

@Chrylo pls review and once happy give a positive review/ping me

@Chrylo
Copy link
Contributor

Chrylo commented Sep 12, 2023

@SebastianSchildt LGTM ;)

@SebastianSchildt SebastianSchildt merged commit 4fd6c30 into eclipse-kuksa:main Sep 12, 2023
3 checks passed
@Chrylo Chrylo deleted the feature-2 branch October 11, 2023 13:06
@Chrylo Chrylo mentioned this pull request Feb 22, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add Eclipse/Dash Licensing
3 participants