Merge pull request #522 from mozilla/local-glean-dev

Local glean dev

.buildconfig.yml

@@ -1,7 +1,7 @@
 libraryVersion: 21.2.0
 groupId: org.mozilla.telemetry
 projects:
-  glean-core:
+  glean:
     path: glean-core/android
     artifactId: glean
     publications:
@@ -12,7 +12,7 @@ projects:
     description: 'The Glean SDK is a modern approach for a Telemetry library and is part of the Glean project.'
   glean-sample-app:
     path: samples/android/app
-    artifactId: glean-core-sample-app
+    artifactId: glean-sample-app
     description: 'An app demoing how to use the Glean library to collect and send telemetry data.'
   glean-gradle-plugin:
     path: gradle-plugin

docs/SUMMARY.md

@@ -59,6 +59,9 @@
         - [Clearing metrics when disabling/enabling Glean](dev/core/internal/clearing.md).
         - [Payload format](dev/core/internal/payload.md).
         - [Directory structure](dev/core/internal/directory-structure.md)
+    - [Howtos](dev/howtos/index.md)
+        - [Development with android-components](dev/howtos/development-with-android-components.md)
+        - [Locally-published components in Fenix](dev/howtos/locally-published-components-in-fenix.md)
 - [API Documentation](api/index.md)
 - [Appendix](appendix/index.md)
   - [This Week in Glean](appendix/twig.md)

docs/code_coverage.md

@@ -19,7 +19,7 @@ Locally you can generate a coverage report with the following command:
 
 
 ```bash
-./gradlew -Pcoverage :glean-core:build
+./gradlew -Pcoverage :glean:build
 ```
 
 After that you'll find an HTML report at the following location:

docs/dev/howtos/development-with-android-components.md

@@ -0,0 +1,81 @@
+# Working on unreleased Glean code in android-components
+
+This is a companion to the [equivalent instructions for the android-components repository](https://mozilla-mobile.github.io/android-components/contributing/testing-components-inside-app).
+
+Modern Gradle supports [composite builds](https://docs.gradle.org/current/userguide/composite_builds.html), which allows to substitute on-disk projects for binary publications.  Composite builds transparently accomplish what is usually a frustrating loop of:
+1. change library
+1. publish library snapshot to the local Maven repository
+1. consume library snapshot in application
+
+## Preparation
+
+Clone the Glean SDK and android-components repositories:
+
+```sh
+git clone https://github.com/mozilla/glean
+git clone https://github.com/mozilla-mobile/android-components
+```
+
+## Cargo build targets
+
+By default when building Android Components using gradle, the gradle-cargo plugin will compile Glean for every possible platform,
+which might end up in failures.
+You can customize which targets are built in `glean/local.properties`
+([more information](https://github.com/ncalexan/rust-android-gradle/blob/master/README.md#specifying-local-targets)):
+
+```
+# For physical devices:
+rust.targets=arm
+
+# For unit tests:
+# rust.targets=darwin # Or linux-*, windows-* (* = x86/x64)
+
+# For emulator only:
+rust.targets=x86
+```
+
+## Substituting projects
+
+android-components has custom build logic for dealing with composite builds,
+so you should be able to configure it by simply adding the path to the Glean repo in the correct `local.properties` file:
+
+In `android-components/local.properties`:
+```groovy
+substitutions.glean.dir=../glean
+```
+
+If this doesn't seem to work, or if you need to configure composite builds for a project that does not contain this custom logic,
+add the following to `settings.gradle`:
+
+In `android-components/settings.gradle`:
+```groovy
+includeBuild('../glean') {
+  dependencySubstitution {
+    substitute module('org.mozilla.telemetry:glean') with project(':glean')
+    substitute module('org.mozilla.telemetry:glean-forUnitTests') with project(':glean')
+  }
+}
+```
+
+Composite builds will ensure the Glean SDK is build as part of tasks inside `android-components`.
+
+## Caveat
+
+There's a big gotcha with library substitutions: the Gradle build computes lazily, and AARs don't include their transitive dependencies' JNI libraries.
+This means that in `android-components`, `./gradlew :service-glean:assembleDebug` **does not** invoke `:glean:cargoBuild`,
+even though `:service-glean` depends on the substitution for `:glean` and even if the inputs to Cargo have changed!
+It's the final consumer of the `:service-glean` project (or publication) that will incorporate the JNI libraries.
+
+In practice that means _you should always be targeting something that produces an APK_: a test or a sample module.
+Then you should find that the `cargoBuild` tasks are invoked as you expect.
+
+Inside the `android-components` repository `./gradlew :samples-glean:connectedAndroidTest` should work.
+Other tests like `:service-glean:testDebugUnitTest` or `:support-sync-telemetry:testDebugUnitTest` will currently fail, because the JNI libraries are not included.
+
+
+## Notes
+
+This document is based on the equivalent documentation for application-services:
+[Development with the Reference Browser](https://github.com/mozilla/application-services/blob/master/docs/howtos/working-with-reference-browser.md)
+
+1. Transitive substitutions (as shown above) work but require newer Gradle versions (4.10+).

docs/dev/howtos/locally-published-components-in-fenix.md

@@ -0,0 +1,98 @@
+# Using locally-published Glean in Fenix
+
+> Note: This is a bit tedious, and you might like to try the substitution-based > approach documented in
+> [Working on unreleased Glean code in android-components](./development-with-android-components.md).
+> That approach is still fairly new, and the local-publishing approach in this document is necessary if it fails.
+
+> Note: This is Fenix-specific only in that some links on the page go to the `mozilla-mobile/fenix` repository,
+> however these steps should work for e.g. `reference-browser`, as well.
+> (Same goes for lockwise, or any other consumer of Glean, but they may use a different structure -- lockwise has no Dependencies.kt, for example)
+
+## Preparation
+
+Clone the Glean SDK, android-components and Fenix repositories:
+
+```sh
+git clone https://github.com/mozilla/glean
+git clone https://github.com/mozilla-mobile/android-components
+git clone https://github.com/mozilla-mobile/fenix/
+```
+
+## Local publishing
+
+
+1. Inside the `glean` repository root:
+    1. In [`.buildconfig.yml`][glean-yaml], change
+       `libraryVersion` to end in `-TESTING$N`[^1],
+       where `$N` is some number that you haven't used for this before.
+
+       Example: `libraryVersion: 22.0.0-TESTING1`
+    2. Check your `local.properties` file,
+       and add `rust.targets=x86` if you're testing on the emulator,
+       `rust.targets=arm` if you're testing on 32-bit arm (arm64 for 64-bit arm, etc).
+       This will make the build that's done in the next step much faster.
+    3. Run `./gradlew publishToMavenLocal`. This may take a few minutes.
+
+2. Inside the `android-components` repository root:
+    1. In [`.buildconfig.yml`][android-components-yaml], change
+       `componentsVersion` to end in `-TESTING$N`[^1],
+       where `$N` is some number that you haven't used for this before.
+
+       Example: `componentsVersion: 24.0.0-TESTING1`
+    2. Inside [`buildSrc/src/main/java/Dependencies.kt`][android-components-deps],
+       change `mozilla_glean` to reference the `libraryVersion` you published in step 2 part 1.
+
+       Example: `const val mozilla_glean = "22.0.0-TESTING1"`
+
+    3. Inside [`build.gradle`][android-components-build-gradle], add
+       `mavenLocal()` inside `allprojects { repositories { <here> } }`.
+
+    4. Inside the android-component's `local.properties` file, ensure
+       `substitutions.glean.dir` is *NOT* set.
+
+    5. Run `./gradlew publishToMavenLocal`.
+
+3. Inside the `fenix` repository root:
+    1. Inside [`build.gradle`][fenix-build-gradle-1], add
+       `mavenLocal()` inside `allprojects { repositories { <here> } }`.
+
+    2. Inside [`buildSrc/src/main/java/Dependencies.kt`][fenix-deps], change
+       `mozilla_android_components` to the version you defined in step 3 part 1.
+
+       Example: `const val mozilla_android_components = "24.0.0-TESTING1"`
+
+       In the same file change `mozilla_glean` to the version you defined in step 1 part 1.
+
+       Example: `const val mozilla_glean = "22.0.0-TESTING1"`
+
+You should now be able to build and run Fenix (assuming you could before all this).
+
+## Caveats
+
+1. This assumes you have followed the [android/rust build setup](../android/setup-android-build-environment.md)
+2. Make sure you're fully up to date in all repos, unless you know you need to not be.
+3. This omits the steps if changes needed because, e.g. Glean made a breaking change to an API used in android-components.
+   These should be understandable to fix, you usually should be able to find a PR with the fixes somewhere in the android-component's list of pending PRs
+   (or, failing that, a description of what to do in the Glean changelog).
+4. Ask in `#glean` slack (or `#telemetry` on mozilla IRC) if you get stuck.
+
+## Notes
+
+This document is based on the equivalent documentation for application-services:
+[Using locally-published components in Fenix](https://github.com/mozilla/application-services/blob/master/docs/howtos/locally-published-components-in-fenix.md)
+
+---
+
+[^1]: It doesn't have to end with `-TESTING$N`, it only needs to have the format `-someidentifier`.
+`-SNAPSHOT$N` is also very common to use, however without the numeric suffix, this has specific meaning to gradle,
+so we avoid it.
+Additionally, while the `$N` we have used in our running example has matched
+(e.g. all of the identifiers ended in `-TESTING1`, this is not required, so long as you match everything up correctly at the end).
+
+[glean-yaml]: https://github.com/mozilla/glean/blob/master/.buildconfig.yml#L1
+[android-components-yaml]: https://github.com/mozilla-mobile/android-components/blob/master/.buildconfig.yml#L1
+[android-components-deps]: https://github.com/mozilla-mobile/android-components/blob/50a2f28027f291bf1c6056d42b55e75ba3c050db/buildSrc/src/main/java/Dependencies.kt#L32
+[android-components-build-gradle]: https://github.com/mozilla-mobile/android-components/blob/b98206cf8de818499bdc87c00de942a41f8aa2fb/build.gradle#L28
+[fenix-build-gradle-1]: https://github.com/mozilla-mobile/fenix/blob/f897c2e295cd1b97d4024c7a9cb45dceb7a2fa89/build.gradle#L26
+[fenix-build-gradle-2]: https://github.com/mozilla-mobile/fenix/blob/f897c2e295cd1b97d4024c7a9cb45dceb7a2fa89/build.gradle#L6
+[fenix-deps]: https://github.com/mozilla-mobile/fenix/blob/8a330d413c1d55d14446abe3cfd57a5494884396/buildSrc/src/main/java/Dependencies.kt#L48

docs/dev/testing.md

@@ -51,7 +51,7 @@ The full Android test suite may be run from the command line with:
 ### From Android Studio
 
 To run the full Android test suite, in the "Gradle" pane, navigate to `glean-core` -> `Tasks` -> `verification` and double-click either `testDebugUnitTest` or `testReleaseUnitTest` (depending on whether you want to run in Debug or Release mode).
-You can save this task permanently by opening the task dropdown in the toolbar and selecting "Save glean.rs:glean-core:android [testDebugUnitTest] Configuration".
+You can save this task permanently by opening the task dropdown in the toolbar and selecting "Save glean.rs:glean:android [testDebugUnitTest] Configuration".
 
 To run a single Android test, navigate to the file containing the test, and right click on the green arrow in the left margin next to the test.  There you have a choice of running or debugging the test.
 

samples/android/app/build.gradle

@@ -32,7 +32,7 @@ android {
 }
 
 dependencies {
-    implementation project(':glean-core')
+    implementation project(':glean')
 
     implementation "org.jetbrains.kotlin:kotlin-stdlib-jdk7:$rootProject.versions.kotlin"
 

settings.gradle

@@ -11,6 +11,7 @@ buildscript {
         jcenter()
     }
 }
+rootProject.name = "glean"
 
 // See https://docs.gradle.org/current/userguide/publishing_maven.html#publishing_maven:deferred_configuration.
 enableFeaturePreview('STABLE_PUBLISHING')
@@ -31,6 +32,8 @@ def setupProject(name, projectProps) {
         if (project.name == name) {
             project.ext.description = description
             project.ext.artifactId = artifactId
+            // Expose the rest of the project properties, mostly for validation reasons.
+            project.ext.configProps = projectProps
         }
     }
 }
@@ -46,7 +49,7 @@ gradle.projectsLoaded { ->
     // Note that since this is set on "rootProject.ext", it will be "in scope" during the evaluation of all projects'
     // gradle files. This means that they can just access "config.<value>", and it'll function properly
     gradle.rootProject.ext.library = [
-            version: buildconfig.libraryVersion,
-            groupId: buildconfig.groupId,
+        version: buildconfig.libraryVersion,
+        groupId: buildconfig.groupId,
     ]
 }