For the full setup see [Setup the Android Build Environment](setup-android-build-environment.html).
@@ -20,10 +20,10 @@ All locations need to be updated on upgrades:
* `dev/android/setup-android-build-environment.md`
* CI configuration
* `.circleci/config.yml`
- * `sdkmanager 'build-tools;33.0.2'`
- * `image: circleci/android:2023.02.1`
+ * `sdkmanager 'build-tools;34.0.0'`
+ * `image: circleci/android:2024.01.1-browsers`
* `taskcluster/docker/linux/Dockerfile`.
- * `ENV ANDROID_BUILD_TOOLS "33.0.2"`
- * `ENV ANDROID_SDK_VERSION "9477386"`
- * `ENV ANDROID_PLATFORM_VERSION "33"`
- * `ENV ANDROID_NDK_VERSION "25.2.9519653"`
+ * `ENV ANDROID_BUILD_TOOLS "34.0.0"`
+ * `ENV ANDROID_SDK_VERSION "11076708"`
+ * `ENV ANDROID_PLATFORM_VERSION "34"`
+ * `ENV ANDROID_NDK_VERSION "26.2.11394342"`
diff --git a/docs/user/SUMMARY.md b/docs/user/SUMMARY.md
index e80322d865..3e74d9e3f7 100644
--- a/docs/user/SUMMARY.md
+++ b/docs/user/SUMMARY.md
@@ -73,6 +73,7 @@
- [Quantity](reference/metrics/quantity.md)
- [Rate](reference/metrics/rate.md)
- [Text](reference/metrics/text.md)
+ - [Object](reference/metrics/object.md)
- [Pings](reference/pings/index.md)
# SDK Specific Information
diff --git a/docs/user/reference/metrics/object.md b/docs/user/reference/metrics/object.md
new file mode 100644
index 0000000000..4c3a31b639
--- /dev/null
+++ b/docs/user/reference/metrics/object.md
@@ -0,0 +1,155 @@
+# Object
+
+Record structured data.
+
+{{#include ../../../shared/blockquote-warning.html}}
+
+## Recording API
+
+### `set`
+
+Sets an object metric to a specific value.
+
+{{#include ../../../shared/tab_header.md}}
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+**C++**
+
+Not yet implemented.
+
+**JavaScript**
+
+```js
+let balloons = [
+ { colour: "red", diameter: 5 },
+ { colour: "blue", diameter: 7 },
+];
+Glean.testOnly.balloons.set(balloons);
+```
+
+
+
+{{#include ../../../shared/tab_footer.md}}
+
+#### Limits
+
+* Only objects matching the specified structure will be recorded
+
+#### Recorded errors
+
+* [`invalid_value`](../../user/metrics/error-reporting.md): if the passed value doesn't match the predefined structure
+
+## Testing API
+
+### `testGetValue`
+
+Gets the recorded value for a given object metric.
+Returns the data as a JSON object if data is stored.
+Returns `null` if no data is stored.
+Has an optional argument to specify the name of the ping you wish to retrieve data from, except
+in Rust where it's required. `None` or no argument will default to the first value found for `send_in_pings`.
+
+{{#include ../../../shared/tab_header.md}}
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+**C++**
+
+Not yet implemented.
+
+**JavaScript**
+
+```js
+// testGetValue will throw a data error on invalid value.
+Assert.equal(
+ [
+ { colour: "red", diameter: 5 },
+ { colour: "blue", diameter: 7 },
+ ],
+ Glean.testOnly.balloons.testGetValue()
+);
+```
+
+
+
+{{#include ../../../shared/tab_footer.md}}
+
+### `testGetNumRecordedErrors`
+
+Gets the number of errors recorded for a given text metric.
+
+{{#include ../../../shared/tab_header.md}}
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+{{#include ../../../shared/tab_footer.md}}
+
+## Metric parameters
+
+Example text metric definition:
+
+```yaml
+party:
+ balloons:
+ type: object
+ description: A collection of balloons
+ bugs:
+ - https://bugzilla.mozilla.org/TODO
+ data_reviews:
+ - http://example.com/reviews
+ notification_emails:
+ - CHANGE-ME@example.com
+ expires: never
+ structure:
+ type: array
+ items:
+ type: object
+ properties:
+ colour:
+ type: string
+ diameter:
+ type: number
+```
+
+For a full reference on metrics parameters common to all metric types,
+refer to the [metrics YAML registry format](../yaml/metrics.md) reference page.
+
+## Data questions
+
diff --git a/docs/user/reference/yaml/metrics.md b/docs/user/reference/yaml/metrics.md
index 675aac515d..96c9d3f050 100644
--- a/docs/user/reference/yaml/metrics.md
+++ b/docs/user/reference/yaml/metrics.md
@@ -104,12 +104,13 @@ See the list of [supported metric types](../metrics/index.md).
{{#include ../../../shared/blockquote-warning.html}}
-##### Types must not be changed after release
+##### Types should not be changed after release
-> Once a metric is defined in a product, its `type` must not be changed.
-> The ingestion pipeline will not be able to handle such a change.
-> If a type change is required a new metric must be added with a new name and the new type.
-> This will require an additional data review, in which you can also reference the old data review.
+> Once a metric is defined in a product, its `type` should only be changed in rare circumstances.
+> It's better to rename the metric with the new type instead.
+> The ingestion pipeline will create a new column for a metric with a changed type.
+> Any new analysis will need to use the new column going forward.
+> The old column will still be populated with data from old clients.
#### `description`
diff --git a/docs/user/user/adding-glean-to-your-project/kotlin.md b/docs/user/user/adding-glean-to-your-project/kotlin.md
index 4c44961304..8ee0aac069 100644
--- a/docs/user/user/adding-glean-to-your-project/kotlin.md
+++ b/docs/user/user/adding-glean-to-your-project/kotlin.md
@@ -122,11 +122,14 @@ ext.gleanGenerateMarkdownDocs = true
apply plugin: "org.mozilla.telemetry.glean-gradle-plugin"
```
-{{#include ../../../shared/blockquote-warning.html}}
+{{#include ../../../shared/blockquote-info.html}}
-##### Earlier versions
+##### Rosetta 2 required on Apple Silicon
-> **Note:** Earlier versions of Glean used a Gradle script (`sdk_generator.gradle`) rather than a Gradle plugin. Its use is deprecated and projects should be updated to use the Gradle plugin as described above.
+> On Apple Silicon machines (M1/M2/M3 MacBooks and iMacs) Rosetta 2 is required for the bundled Python.
+> See the [Apple documentation about Rosetta 2](https://support.apple.com/en-us/HT211861)
+> and [Bug 1775420](https://bugzilla.mozilla.org/show_bug.cgi?id=1775420) for details.
+> You can install it with `softwareupdate --install-rosetta`
{{#include ../../../shared/blockquote-info.html}}
diff --git a/docs/user/user/pings/index.md b/docs/user/user/pings/index.md
index f12de0d83c..068ce88d90 100644
--- a/docs/user/user/pings/index.md
+++ b/docs/user/user/pings/index.md
@@ -16,11 +16,14 @@ that are assembled out of the box without any developer intervention.
Every ping payload has the following keys at the top-level:
- The [`ping_info` section](#the-ping_info-section) contains core metadata
- that is included in **every** ping.
+ that is included in every ping that doesn't set the
+ `metadata.include_info_sections` property to `false`.
- The [`client_info` section](#the-client_info-section) contains information that identifies the client.
- It is included in most pings (including all built-in pings), but may be excluded from pings
- where we don't want to connect client information with the other metrics in the ping.
+ It is included in every ping that doesn't set the
+ `metadata.include_info_sections` property to `false`.
+ When included, it contains a persistent client identifier `client_id`,
+ except when the `include_client_id` property is set to `false`.
The following keys are only present if any metrics or events were recorded for the given ping:
@@ -36,7 +39,8 @@ for more details for each metric type in the `metrics` and `events` section.
### The `ping_info` section
Metadata about the ping itself.
-This section is included in **every** ping.
+This section is included in every ping that doesn't set the
+`metadata.include_info_sections` property to `false`.
The following fields are included in the `ping_info` section.
Optional fields are marked accordingly.
@@ -103,6 +107,9 @@ The data is provided by the embedding application or automatically fetched by th
It is collected at initialization time and sent in every ping afterwards.
For historical reasons it contains metrics that are only useful on a certain platform.
+This section is included in every ping that doesn't set the
+`metadata.include_info_sections` property to `false`.
+
{{#include ../../../shared/blockquote-info.html}}
##### Additional metrics require a proposal
diff --git a/glean-core/Cargo.toml b/glean-core/Cargo.toml
index fbfe15909b..cdc2808ebb 100644
--- a/glean-core/Cargo.toml
+++ b/glean-core/Cargo.toml
@@ -1,6 +1,6 @@
[package]
name = "glean-core"
-version = "57.0.0"
+version = "58.0.0"
authors = ["Jan-Erik Rediger ", "The Glean Team "]
description = "A modern Telemetry library"
repository = "https://github.com/mozilla/glean"
@@ -21,7 +21,7 @@ include = [
rust-version = "1.66"
[package.metadata.glean]
-glean-parser = "11.0.1"
+glean-parser = "13.0.0"
[badges]
circle-ci = { repository = "mozilla/glean", branch = "main" }
diff --git a/glean-core/android-native/build.gradle b/glean-core/android-native/build.gradle
index fe95a840da..48769651f6 100644
--- a/glean-core/android-native/build.gradle
+++ b/glean-core/android-native/build.gradle
@@ -61,7 +61,7 @@ cargo {
module = '../bundle-android'
// The Android NDK API level to target.
- apiLevel = 21
+ apiLevel = rootProject.ext.build['minSdkVersion']
// Where Cargo writes its outputs.
targetDirectory = '../../target'
@@ -99,8 +99,18 @@ configurations {
}
dependencies {
- jnaForTest "net.java.dev.jna:jna:$versions.jna@jar"
- implementation "net.java.dev.jna:jna:$versions.jna@aar"
+ jnaForTest(libs.jna) {
+ artifact {
+ extension ="jar"
+ type = "jar"
+ }
+ }
+ implementation(libs.jna) {
+ artifact {
+ extension ="aar"
+ type = "aar"
+ }
+ }
}
afterEvaluate {
diff --git a/glean-core/android-native/dependency-licenses.xml b/glean-core/android-native/dependency-licenses.xml
index 25a22bdba5..57b510935a 100644
--- a/glean-core/android-native/dependency-licenses.xml
+++ b/glean-core/android-native/dependency-licenses.xml
@@ -39,6 +39,9 @@ the details of which are reproduced below.
Apache License 2.0: humantime
https://github.com/tailhook/humantime
+
+ Apache License 2.0: fd-lock
+ https://github.com/yoshuawuyts/fd-lock
Apache License 2.0: anyhow
https://github.com/dtolnay/anyhow
diff --git a/glean-core/android/build.gradle b/glean-core/android/build.gradle
index 6a513a8f1d..bf62a3af05 100644
--- a/glean-core/android/build.gradle
+++ b/glean-core/android/build.gradle
@@ -8,7 +8,7 @@
import groovy.json.JsonOutput
plugins {
- id "com.jetbrains.python.envs" version "0.0.26"
+ alias libs.plugins.gradle.python.envs
}
apply plugin: 'com.android.library'
@@ -98,7 +98,7 @@ android {
afterEvaluate {
if (project.hasProperty("coverage")) {
jacoco {
- toolVersion = "0.8.10"
+ toolVersion = libs.versions.jacoco
}
task jacocoTestReport(type: JacocoReport) {
@@ -148,18 +148,26 @@ configurations {
}
dependencies {
- jnaForTest "net.java.dev.jna:jna:$versions.jna@jar"
- implementation "net.java.dev.jna:jna:$versions.jna@aar"
+ jnaForTest(libs.jna) {
+ artifact {
+ extension ="jar"
+ type = "jar"
+ }
+ }
+ implementation(libs.jna) {
+ artifact {
+ extension ="aar"
+ type = "aar"
+ }
+ }
implementation project(":glean-native")
- // Note: the following version must be kept in sync
- implementation "org.jetbrains.kotlinx:kotlinx-coroutines-android:$versions.coroutines"
-
- implementation "androidx.annotation:annotation:$versions.androidx_annotation"
- implementation "androidx.lifecycle:lifecycle-common:$versions.androidx_lifecycle"
- implementation "androidx.lifecycle:lifecycle-process:$versions.androidx_lifecycle"
- implementation "androidx.work:work-runtime-ktx:$versions.androidx_work"
+ implementation libs.androidx.annotation
+ implementation libs.androidx.lifecycle.common
+ implementation libs.androidx.lifecycle.process
+ implementation libs.androidx.work
+ implementation libs.kotlinx.coroutines
// We need a compileOnly dependency on the following block of testing
// libraries in order to expose the GleanTestRule to applications/libraries
@@ -167,8 +175,8 @@ dependencies {
// We can't simply create a separate package otherwise we would need
// to provide a public API for the testing package to access the
// Glean internals, which is something we would not want to do.
- compileOnly "junit:junit:$versions.junit"
- compileOnly "androidx.work:work-testing:$versions.androidx_work"
+ compileOnly libs.junit
+ compileOnly libs.test.work
// For reasons unknown, resolving the jnaForTest configuration directly
// trips a nasty issue with the Android-Gradle plugin 3.2.1, like `Cannot
@@ -178,15 +186,15 @@ dependencies {
// causing it to be resolved. Cloning first dissociates the configuration,
// avoiding other configurations from being resolved. Tricky!
testImplementation files(configurations.jnaForTest.copyRecursive().files)
- testImplementation "androidx.test.ext:junit:$versions.androidx_junit"
- testImplementation "org.robolectric:robolectric:$versions.robolectric"
- testImplementation "org.mockito:mockito-core:$versions.mockito"
- testImplementation "androidx.test:core-ktx:$versions.androidx_test"
- testImplementation "com.squareup.okhttp3:mockwebserver:$versions.mockwebserver"
- testImplementation "androidx.work:work-testing:$versions.androidx_work"
-
- androidTestImplementation "androidx.test:runner:$versions.androidx_test"
- androidTestImplementation "androidx.test.espresso:espresso-core:$versions.androidx_espresso"
+ testImplementation libs.mockito
+ testImplementation libs.mockwebserver
+ testImplementation libs.robolectric
+ testImplementation libs.test.core
+ testImplementation libs.test.junit.ext
+ testImplementation libs.test.work
+
+ androidTestImplementation libs.test.espresso.core
+ androidTestImplementation libs.test.runner
}
evaluationDependsOn(":glean-native")
diff --git a/glean-core/android/dependency-licenses.xml b/glean-core/android/dependency-licenses.xml
index 25a22bdba5..57b510935a 100644
--- a/glean-core/android/dependency-licenses.xml
+++ b/glean-core/android/dependency-licenses.xml
@@ -39,6 +39,9 @@ the details of which are reproduced below.
Apache License 2.0: humantime
https://github.com/tailhook/humantime
+
+ Apache License 2.0: fd-lock
+ https://github.com/yoshuawuyts/fd-lock
Apache License 2.0: anyhow
https://github.com/dtolnay/anyhow
diff --git a/glean-core/android/src/main/java/mozilla/telemetry/glean/private/PingType.kt b/glean-core/android/src/main/java/mozilla/telemetry/glean/private/PingType.kt
index 349c3bdab5..19518b433c 100644
--- a/glean-core/android/src/main/java/mozilla/telemetry/glean/private/PingType.kt
+++ b/glean-core/android/src/main/java/mozilla/telemetry/glean/private/PingType.kt
@@ -49,6 +49,7 @@ class PingType (
includeClientId: Boolean,
sendIfEmpty: Boolean,
preciseTimestamps: Boolean,
+ includeInfoSections: Boolean,
val reasonCodes: List,
) where ReasonCodesEnum : Enum, ReasonCodesEnum : ReasonCode {
private var testCallback: ((ReasonCodesEnum?) -> Unit)? = null
@@ -60,6 +61,7 @@ class PingType (
includeClientId = includeClientId,
sendIfEmpty = sendIfEmpty,
preciseTimestamps = preciseTimestamps,
+ includeInfoSections = includeInfoSections,
reasonCodes = reasonCodes,
)
}
diff --git a/glean-core/android/src/test/java/mozilla/telemetry/glean/GleanTest.kt b/glean-core/android/src/test/java/mozilla/telemetry/glean/GleanTest.kt
index d68c1ba326..f6317dc116 100644
--- a/glean-core/android/src/test/java/mozilla/telemetry/glean/GleanTest.kt
+++ b/glean-core/android/src/test/java/mozilla/telemetry/glean/GleanTest.kt
@@ -479,6 +479,7 @@ class GleanTest {
includeClientId = true,
sendIfEmpty = false,
preciseTimestamps = true,
+ includeInfoSections = true,
reasonCodes = listOf(),
)
val stringMetric = StringMetricType(
@@ -845,6 +846,7 @@ class GleanTest {
includeClientId = true,
sendIfEmpty = false,
preciseTimestamps = true,
+ includeInfoSections = true,
reasonCodes = listOf(),
)
val stringMetric = StringMetricType(
diff --git a/glean-core/android/src/test/java/mozilla/telemetry/glean/debug/GleanDebugActivityTest.kt b/glean-core/android/src/test/java/mozilla/telemetry/glean/debug/GleanDebugActivityTest.kt
index 001c834702..e70ad9aa94 100644
--- a/glean-core/android/src/test/java/mozilla/telemetry/glean/debug/GleanDebugActivityTest.kt
+++ b/glean-core/android/src/test/java/mozilla/telemetry/glean/debug/GleanDebugActivityTest.kt
@@ -249,6 +249,7 @@ class GleanDebugActivityTest {
includeClientId = false,
sendIfEmpty = true,
preciseTimestamps = true,
+ includeInfoSections = true,
reasonCodes = listOf(),
)
diff --git a/glean-core/android/src/test/java/mozilla/telemetry/glean/pings/CustomPingTest.kt b/glean-core/android/src/test/java/mozilla/telemetry/glean/pings/CustomPingTest.kt
index 800c6df942..9b8d29d6cc 100644
--- a/glean-core/android/src/test/java/mozilla/telemetry/glean/pings/CustomPingTest.kt
+++ b/glean-core/android/src/test/java/mozilla/telemetry/glean/pings/CustomPingTest.kt
@@ -77,6 +77,7 @@ class CustomPingTest {
includeClientId = true,
sendIfEmpty = true,
preciseTimestamps = true,
+ includeInfoSections = true,
reasonCodes = emptyList(),
)
@@ -106,6 +107,7 @@ class CustomPingTest {
includeClientId = true,
sendIfEmpty = true,
preciseTimestamps = true,
+ includeInfoSections = true,
reasonCodes = emptyList(),
)
@@ -179,6 +181,7 @@ class CustomPingTest {
includeClientId = true,
sendIfEmpty = true,
preciseTimestamps = true,
+ includeInfoSections = true,
reasonCodes = emptyList(),
)
diff --git a/glean-core/android/src/test/java/mozilla/telemetry/glean/private/EventMetricTypeTest.kt b/glean-core/android/src/test/java/mozilla/telemetry/glean/private/EventMetricTypeTest.kt
index d42f0c92d9..ca1675452f 100644
--- a/glean-core/android/src/test/java/mozilla/telemetry/glean/private/EventMetricTypeTest.kt
+++ b/glean-core/android/src/test/java/mozilla/telemetry/glean/private/EventMetricTypeTest.kt
@@ -507,6 +507,7 @@ class EventMetricTypeTest {
includeClientId = true,
sendIfEmpty = false,
preciseTimestamps = true,
+ includeInfoSections = true,
reasonCodes = listOf(),
)
diff --git a/glean-core/android/src/test/java/mozilla/telemetry/glean/private/PingTypeTest.kt b/glean-core/android/src/test/java/mozilla/telemetry/glean/private/PingTypeTest.kt
index 10656a1c94..61e40b90c2 100644
--- a/glean-core/android/src/test/java/mozilla/telemetry/glean/private/PingTypeTest.kt
+++ b/glean-core/android/src/test/java/mozilla/telemetry/glean/private/PingTypeTest.kt
@@ -54,6 +54,7 @@ class PingTypeTest {
includeClientId = true,
sendIfEmpty = false,
preciseTimestamps = true,
+ includeInfoSections = true,
reasonCodes = listOf(),
)
@@ -120,6 +121,7 @@ class PingTypeTest {
includeClientId = true,
sendIfEmpty = false,
preciseTimestamps = true,
+ includeInfoSections = true,
reasonCodes = listOf(),
)
@@ -167,6 +169,7 @@ class PingTypeTest {
includeClientId = true,
sendIfEmpty = false,
preciseTimestamps = true,
+ includeInfoSections = true,
reasonCodes = listOf(),
)
@@ -214,6 +217,7 @@ class PingTypeTest {
includeClientId = false,
sendIfEmpty = false,
preciseTimestamps = true,
+ includeInfoSections = true,
reasonCodes = listOf(),
)
@@ -263,6 +267,7 @@ class PingTypeTest {
sendIfEmpty = false,
reasonCodes = listOf(),
preciseTimestamps = true,
+ includeInfoSections = true,
)
val counter = CounterMetricType(
diff --git a/glean-core/build/Cargo.toml b/glean-core/build/Cargo.toml
index 0c8d640634..bac7602cd6 100644
--- a/glean-core/build/Cargo.toml
+++ b/glean-core/build/Cargo.toml
@@ -1,6 +1,6 @@
[package]
name = "glean-build"
-version = "11.0.1"
+version = "13.0.0"
edition = "2021"
description = "Glean SDK Rust build helper"
repository = "https://github.com/mozilla/glean"
diff --git a/glean-core/build/src/lib.rs b/glean-core/build/src/lib.rs
index b1dadd9d09..f6f8115652 100644
--- a/glean-core/build/src/lib.rs
+++ b/glean-core/build/src/lib.rs
@@ -39,7 +39,7 @@ use std::env;
use xshell_venv::{Result, Shell, VirtualEnv};
-const GLEAN_PARSER_VERSION: &str = "11.0.1";
+const GLEAN_PARSER_VERSION: &str = "13.0.0";
/// A Glean Rust bindings generator.
pub struct Builder {
@@ -134,17 +134,21 @@ mod test {
#[test]
fn test_builder() {
- let package_root = PathBuf::from(env!("CARGO_MANIFEST_DIR"));
+ // We know this package's location, and it's up 2 folders from there.
+ let manifest_dir = PathBuf::from(env!("CARGO_MANIFEST_DIR"));
+ let package_root = manifest_dir.parent().unwrap().parent().unwrap();
+ // Ensure xshell-venv create the venv in the expected directory.
+ env::set_var(
+ "CARGO_TARGET_DIR",
+ package_root.join("target").display().to_string(),
+ );
+
// clean out the previous venv, if it exists
let venv_dir = package_root.join("target").join("venv-py3-glean_parser");
if venv_dir.exists() {
fs::remove_dir_all(venv_dir).unwrap();
}
let metrics_yaml = package_root
- .parent()
- .unwrap()
- .parent()
- .unwrap()
.join("samples")
.join("rust")
.join("metrics.yaml");
diff --git a/glean-core/ios/Glean/Metrics/Ping.swift b/glean-core/ios/Glean/Metrics/Ping.swift
index 867a7a2cdd..cfd4981c10 100644
--- a/glean-core/ios/Glean/Metrics/Ping.swift
+++ b/glean-core/ios/Glean/Metrics/Ping.swift
@@ -40,6 +40,7 @@ public class Ping {
includeClientId: Bool,
sendIfEmpty: Bool,
preciseTimestamps: Bool,
+ includeInfoSections: Bool,
reasonCodes: [String]
) {
self.name = name
@@ -49,6 +50,7 @@ public class Ping {
includeClientId,
sendIfEmpty,
preciseTimestamps,
+ includeInfoSections,
reasonCodes
)
}
diff --git a/glean-core/ios/GleanTests/GleanTests.swift b/glean-core/ios/GleanTests/GleanTests.swift
index 93542751b4..2b01219bc7 100644
--- a/glean-core/ios/GleanTests/GleanTests.swift
+++ b/glean-core/ios/GleanTests/GleanTests.swift
@@ -258,6 +258,7 @@ class GleanTests: XCTestCase {
includeClientId: true,
sendIfEmpty: false,
preciseTimestamps: true,
+ includeInfoSections: true,
reasonCodes: []
)
@@ -415,6 +416,7 @@ class GleanTests: XCTestCase {
includeClientId: true,
sendIfEmpty: false,
preciseTimestamps: true,
+ includeInfoSections: true,
reasonCodes: []
)
diff --git a/glean-core/ios/GleanTests/Metrics/PingTests.swift b/glean-core/ios/GleanTests/Metrics/PingTests.swift
index e693384de6..477ef0870c 100644
--- a/glean-core/ios/GleanTests/Metrics/PingTests.swift
+++ b/glean-core/ios/GleanTests/Metrics/PingTests.swift
@@ -40,6 +40,7 @@ class PingTests: XCTestCase {
includeClientId: true,
sendIfEmpty: false,
preciseTimestamps: true,
+ includeInfoSections: true,
reasonCodes: []
)
@@ -86,6 +87,7 @@ class PingTests: XCTestCase {
includeClientId: false,
sendIfEmpty: false,
preciseTimestamps: true,
+ includeInfoSections: true,
reasonCodes: []
)
@@ -122,6 +124,7 @@ class PingTests: XCTestCase {
includeClientId: false,
sendIfEmpty: true,
preciseTimestamps: true,
+ includeInfoSections: true,
reasonCodes: []
)
@@ -213,6 +216,7 @@ class PingTests: XCTestCase {
includeClientId: true,
sendIfEmpty: true,
preciseTimestamps: true,
+ includeInfoSections: true,
reasonCodes: ["was_tested"]
)
diff --git a/glean-core/ios/sdk_generator.sh b/glean-core/ios/sdk_generator.sh
index 78ca3cadd0..e1eaff06df 100755
--- a/glean-core/ios/sdk_generator.sh
+++ b/glean-core/ios/sdk_generator.sh
@@ -25,7 +25,7 @@
set -e
-GLEAN_PARSER_VERSION=11.0
+GLEAN_PARSER_VERSION=13.0
# CMDNAME is used in the usage text below.
# shellcheck disable=SC2034
diff --git a/glean-core/python/glean/__init__.py b/glean-core/python/glean/__init__.py
index e7f97a49f9..3b5953894a 100644
--- a/glean-core/python/glean/__init__.py
+++ b/glean-core/python/glean/__init__.py
@@ -31,7 +31,7 @@
__email__ = "glean-team@mozilla.com"
-GLEAN_PARSER_VERSION = "11.0.1"
+GLEAN_PARSER_VERSION = "13.0.0"
parser_version = VersionInfo.parse(GLEAN_PARSER_VERSION)
parser_version_next_major = parser_version.bump_major()
diff --git a/glean-core/python/glean/_loader.py b/glean-core/python/glean/_loader.py
index 70da8f02ca..e4390153a7 100644
--- a/glean-core/python/glean/_loader.py
+++ b/glean-core/python/glean/_loader.py
@@ -61,6 +61,7 @@
"reason_codes",
"send_in_pings",
"precise_timestamps",
+ "include_info_sections",
"time_unit",
]
diff --git a/glean-core/python/glean/metrics/ping.py b/glean-core/python/glean/metrics/ping.py
index 07a5ca3807..2372a93e14 100644
--- a/glean-core/python/glean/metrics/ping.py
+++ b/glean-core/python/glean/metrics/ping.py
@@ -16,6 +16,7 @@ def __init__(
include_client_id: bool,
send_if_empty: bool,
precise_timestamps: bool,
+ include_info_sections: bool,
reason_codes: List[str],
):
"""
@@ -26,7 +27,12 @@ def __init__(
"""
self._reason_codes = reason_codes
self._inner = GleanPingType(
- name, include_client_id, send_if_empty, precise_timestamps, reason_codes
+ name,
+ include_client_id,
+ send_if_empty,
+ precise_timestamps,
+ include_info_sections,
+ reason_codes,
)
self._test_callback = None # type: Optional[Callable[[Optional[str]], None]]
diff --git a/glean-core/python/requirements_dev.txt b/glean-core/python/requirements_dev.txt
index 0513ab5bf7..cbafca4c2c 100644
--- a/glean-core/python/requirements_dev.txt
+++ b/glean-core/python/requirements_dev.txt
@@ -1,5 +1,5 @@
auditwheel==6.0.0
-black==24.1.1
+black==24.2.0
coverage[toml]==7.2.2
flake8==7.0.0; python_version >= '3.8'
flake8-bugbear==24.2.6; python_version >= '3.8'
diff --git a/glean-core/python/tests/test_glean.py b/glean-core/python/tests/test_glean.py
index 010898d32c..2d74daee41 100644
--- a/glean-core/python/tests/test_glean.py
+++ b/glean-core/python/tests/test_glean.py
@@ -279,6 +279,7 @@ def test_dont_schedule_pings_if_metrics_disabled(safe_httpserver):
include_client_id=True,
send_if_empty=False,
precise_timestamps=True,
+ include_info_sections=True,
reason_codes=[],
)
@@ -299,6 +300,7 @@ def test_dont_schedule_pings_if_there_is_no_ping_content(safe_httpserver):
include_client_id=True,
send_if_empty=False,
precise_timestamps=True,
+ include_info_sections=True,
reason_codes=[],
)
@@ -365,6 +367,7 @@ def test_ping_collection_must_happen_after_currently_scheduled_metrics_recording
include_client_id=True,
send_if_empty=False,
precise_timestamps=True,
+ include_info_sections=True,
reason_codes=[],
)
string_metric = StringMetricType(
@@ -694,6 +697,7 @@ def broken_process(*args, **kwargs):
include_client_id=True,
send_if_empty=True,
precise_timestamps=True,
+ include_info_sections=True,
reason_codes=[],
)
@@ -766,6 +770,7 @@ def test_presubmit_makes_a_valid_ping(tmpdir, ping_schema_url, monkeypatch):
include_client_id=True,
send_if_empty=True,
precise_timestamps=True,
+ include_info_sections=True,
reason_codes=[],
)
@@ -828,6 +833,7 @@ def test_flipping_upload_enabled_respects_order_of_events(tmpdir, monkeypatch):
include_client_id=True,
send_if_empty=True,
precise_timestamps=True,
+ include_info_sections=True,
reason_codes=[],
)
@@ -936,6 +942,7 @@ def test_sending_of_custom_pings(safe_httpserver):
include_client_id=True,
send_if_empty=False,
precise_timestamps=True,
+ include_info_sections=True,
reason_codes=[],
)
@@ -1019,6 +1026,7 @@ def test_glean_shutdown(safe_httpserver):
include_client_id=True,
send_if_empty=False,
precise_timestamps=False,
+ include_info_sections=True,
reason_codes=[],
)
diff --git a/glean-core/rlb/Cargo.toml b/glean-core/rlb/Cargo.toml
index 19baea550d..d4b2c36e69 100644
--- a/glean-core/rlb/Cargo.toml
+++ b/glean-core/rlb/Cargo.toml
@@ -1,6 +1,6 @@
[package]
name = "glean"
-version = "57.0.0"
+version = "58.0.0"
authors = ["Jan-Erik Rediger ", "The Glean Team "]
description = "Glean SDK Rust language bindings"
repository = "https://github.com/mozilla/glean"
@@ -23,27 +23,22 @@ maintenance = { status = "actively-developed" }
[dependencies.glean-core]
path = ".."
-version = "57.0.0"
+version = "58.0.0"
[dependencies]
-crossbeam-channel = "0.5"
inherent = "1"
log = "0.4.8"
once_cell = "1.18.0"
-thiserror = "1.0.4"
-serde_json = "1.0.44"
-serde = { version = "1.0.104", features = ["derive"] }
-uuid = { version = "1.0", features = ["v4"] }
-chrono = { version = "0.4.10", features = ["serde"] }
-time = "0.1.40"
whatsys = "0.3.0"
[dev-dependencies]
+crossbeam-channel = "0.5"
env_logger = { version = "0.10.0", default-features = false, features = ["humantime"] }
-tempfile = "3.1.0"
-jsonschema-valid = "0.5.0"
flate2 = "1.0.19"
+jsonschema-valid = "0.5.0"
libc = "0.2"
+serde_json = "1.0.44"
+tempfile = "3.1.0"
[features]
preinit_million_queue = ["glean-core/preinit_million_queue"]
diff --git a/glean-core/rlb/examples/crashing-threads.rs b/glean-core/rlb/examples/crashing-threads.rs
index c1afcbf680..5d165c4fda 100644
--- a/glean-core/rlb/examples/crashing-threads.rs
+++ b/glean-core/rlb/examples/crashing-threads.rs
@@ -88,7 +88,7 @@ pub mod glean_metrics {
#[allow(non_upper_case_globals)]
pub static PrototypePing: Lazy =
- Lazy::new(|| PingType::new("prototype", true, true, true, vec![]));
+ Lazy::new(|| PingType::new("prototype", true, true, true, true, vec![]));
fn main() {
env_logger::init();
diff --git a/glean-core/rlb/examples/long-running.rs b/glean-core/rlb/examples/long-running.rs
index 018ce07f0b..a1c9836530 100644
--- a/glean-core/rlb/examples/long-running.rs
+++ b/glean-core/rlb/examples/long-running.rs
@@ -33,12 +33,7 @@ pub mod glean_metrics {
struct FakeUploader;
impl net::PingUploader for FakeUploader {
- fn upload(
- &self,
- _url: String,
- _body: Vec,
- _headers: Vec<(String, String)>,
- ) -> net::UploadResult {
+ fn upload(&self, _upload_request: net::PingUploadRequest) -> net::UploadResult {
thread::sleep(time::Duration::from_millis(100));
net::UploadResult::http_status(200)
}
@@ -46,7 +41,7 @@ impl net::PingUploader for FakeUploader {
#[allow(non_upper_case_globals)]
pub static PrototypePing: Lazy =
- Lazy::new(|| PingType::new("prototype", true, true, true, vec![]));
+ Lazy::new(|| PingType::new("prototype", true, true, true, true, vec![]));
fn main() {
env_logger::init();
diff --git a/glean-core/rlb/examples/prototype.rs b/glean-core/rlb/examples/prototype.rs
index a38154b946..00f3f98b90 100644
--- a/glean-core/rlb/examples/prototype.rs
+++ b/glean-core/rlb/examples/prototype.rs
@@ -29,7 +29,7 @@ pub mod glean_metrics {
#[allow(non_upper_case_globals)]
pub static PrototypePing: Lazy =
- Lazy::new(|| PingType::new("prototype", true, true, true, vec![]));
+ Lazy::new(|| PingType::new("prototype", true, true, true, true, vec![]));
fn main() {
env_logger::init();
diff --git a/glean-core/rlb/src/lib.rs b/glean-core/rlb/src/lib.rs
index 538b8c590d..5c5b945a95 100644
--- a/glean-core/rlb/src/lib.rs
+++ b/glean-core/rlb/src/lib.rs
@@ -23,7 +23,7 @@
//! let cfg = ConfigurationBuilder::new(true, "/tmp/data", "org.mozilla.glean_core.example").build();
//! glean::initialize(cfg, ClientInfoMetrics::unknown());
//!
-//! let prototype_ping = PingType::new("prototype", true, true, true, vec!());
+//! let prototype_ping = PingType::new("prototype", true, true, true, true, vec!());
//!
//! prototype_ping.submit(None);
//! ```
diff --git a/glean-core/rlb/src/net/http_uploader.rs b/glean-core/rlb/src/net/http_uploader.rs
index 4646fe61b4..4ca1687acf 100644
--- a/glean-core/rlb/src/net/http_uploader.rs
+++ b/glean-core/rlb/src/net/http_uploader.rs
@@ -2,7 +2,7 @@
// License, v. 2.0. If a copy of the MPL was not distributed with this
// file, You can obtain one at https://mozilla.org/MPL/2.0/.
-use crate::net::{PingUploader, UploadResult};
+use crate::net::{PingUploadRequest, PingUploader, UploadResult};
/// A simple mechanism to upload pings over HTTPS.
#[derive(Debug)]
@@ -13,12 +13,9 @@ impl PingUploader for HttpUploader {
///
/// # Arguments
///
- /// * `url` - the URL path to upload the data to.
- /// * `body` - the serialized text data to send.
- /// * `headers` - a vector of tuples containing the headers to send with
- /// the request, i.e. (Name, Value).
- fn upload(&self, url: String, _body: Vec, _headers: Vec<(String, String)>) -> UploadResult {
- log::debug!("TODO bug 1675468: submitting to {:?}", url);
+ /// * `upload_request` - the requested upload.
+ fn upload(&self, upload_request: PingUploadRequest) -> UploadResult {
+ log::debug!("TODO bug 1675468: submitting to {:?}", upload_request.url);
UploadResult::http_status(200)
}
}
diff --git a/glean-core/rlb/src/net/mod.rs b/glean-core/rlb/src/net/mod.rs
index 5571d30e67..5546078e63 100644
--- a/glean-core/rlb/src/net/mod.rs
+++ b/glean-core/rlb/src/net/mod.rs
@@ -19,6 +19,20 @@ use thread_state::{AtomicState, State};
mod http_uploader;
+/// Everything you need to request a ping to be uploaded.
+pub struct PingUploadRequest {
+ /// The URL the Glean SDK expects you to use to upload the ping.
+ pub url: String,
+ /// The body, already content-encoded, for upload.
+ pub body: Vec,
+ /// The HTTP headers, including any Content-Encoding.
+ pub headers: Vec<(String, String)>,
+ /// Whether the body has {client|ping}_info sections in it.
+ pub body_has_info_sections: bool,
+ /// The name (aka doctype) of the ping.
+ pub ping_name: String,
+}
+
/// A description of a component used to upload pings.
pub trait PingUploader: std::fmt::Debug + Send + Sync {
/// Uploads a ping to a server.
@@ -29,7 +43,7 @@ pub trait PingUploader: std::fmt::Debug + Send + Sync {
/// * `body` - the serialized text data to send.
/// * `headers` - a vector of tuples containing the headers to send with
/// the request, i.e. (Name, Value).
- fn upload(&self, url: String, body: Vec, headers: Vec<(String, String)>) -> UploadResult;
+ fn upload(&self, upload_request: PingUploadRequest) -> UploadResult;
}
/// The logic for uploading pings: this leaves the actual upload mechanism as
@@ -105,7 +119,14 @@ impl UploadManager {
let upload_url = format!("{}{}", inner.server_endpoint, request.path);
let headers: Vec<(String, String)> =
request.headers.into_iter().collect();
- let result = inner.uploader.upload(upload_url, request.body, headers);
+ let upload_request = PingUploadRequest {
+ url: upload_url,
+ body: request.body,
+ headers,
+ body_has_info_sections: request.body_has_info_sections,
+ ping_name: request.ping_name,
+ };
+ let result = inner.uploader.upload(upload_request);
// Process the upload response.
match glean_core::glean_process_ping_upload_response(doc_id, result) {
UploadTaskAction::Next => (),
diff --git a/glean-core/rlb/src/private/mod.rs b/glean-core/rlb/src/private/mod.rs
index 8a5c304193..575707cf59 100644
--- a/glean-core/rlb/src/private/mod.rs
+++ b/glean-core/rlb/src/private/mod.rs
@@ -5,6 +5,7 @@
//! The different metric types supported by the Glean SDK to handle data.
mod event;
+mod object;
mod ping;
pub use event::EventMetric;
@@ -26,6 +27,7 @@ pub use glean_core::UrlMetric;
pub use glean_core::UuidMetric;
pub use glean_core::{AllowLabeled, LabeledMetric};
pub use glean_core::{Datetime, DatetimeMetric};
+pub use object::ObjectMetric;
pub use ping::PingType;
// Re-export types that are used by the glean_parser-generated code.
diff --git a/glean-core/rlb/src/private/object.rs b/glean-core/rlb/src/private/object.rs
new file mode 100644
index 0000000000..f7403ec889
--- /dev/null
+++ b/glean-core/rlb/src/private/object.rs
@@ -0,0 +1,192 @@
+// This Source Code Form is subject to the terms of the Mozilla Public
+// License, v. 2.0. If a copy of the MPL was not distributed with this
+// file, You can obtain one at https://mozilla.org/MPL/2.0/.
+
+use std::marker::PhantomData;
+
+use glean_core::metrics::JsonValue;
+use glean_core::traits;
+
+use crate::ErrorType;
+
+// We need to wrap the glean-core type: otherwise if we try to implement
+// the trait for the metric in `glean_core::metrics` we hit error[E0117]:
+// only traits defined in the current crate can be implemented for arbitrary
+// types.
+
+/// Developer-facing API for recording object metrics.
+///
+/// Instances of this class type are automatically generated by the parsers
+/// at build time, allowing developers to record values that were previously
+/// registered in the metrics.yaml file.
+#[derive(Clone)]
+pub struct ObjectMetric {
+ pub(crate) inner: glean_core::metrics::ObjectMetric,
+ object_type: PhantomData,
+}
+
+impl ObjectMetric {
+ /// The public constructor used by automatically generated metrics.
+ pub fn new(meta: glean_core::CommonMetricData) -> Self {
+ let inner = glean_core::metrics::ObjectMetric::new(meta);
+ Self {
+ inner,
+ object_type: PhantomData,
+ }
+ }
+
+ /// Sets to the specified structure.
+ ///
+ /// # Arguments
+ ///
+ /// * `object` - the object to set.
+ pub fn set(&self, object: K) {
+ let obj = object
+ .into_serialized_object()
+ .expect("failed to serialize object. This should be impossible.");
+ self.inner.set(obj);
+ }
+
+ /// Sets to the specified structure.
+ ///
+ /// Parses the passed JSON string.
+ /// If it can't be parsed into a valid object it records an invalid value error.
+ ///
+ /// # Arguments
+ ///
+ /// * `object` - JSON representation of the object to set.
+ pub fn set_string(&self, object: String) {
+ let data = match K::from_str(&object) {
+ Ok(data) => data,
+ Err(_) => {
+ self.inner.record_schema_error();
+ return;
+ }
+ };
+ self.set(data)
+ }
+
+ /// **Test-only API (exported for FFI purposes).**
+ ///
+ /// Gets the currently stored value as JSON-encoded string.
+ ///
+ /// This doesn't clear the stored value.
+ pub fn test_get_value<'a, S: Into>>(&self, ping_name: S) -> Option {
+ let ping_name = ping_name.into().map(|s| s.to_string());
+ self.inner.test_get_value(ping_name)
+ }
+
+ /// **Exported for test purposes.**
+ ///
+ /// Gets the number of recorded errors for the given metric and error type.
+ ///
+ /// # Arguments
+ ///
+ /// * `error` - The type of error
+ ///
+ /// # Returns
+ ///
+ /// The number of errors reported.
+ pub fn test_get_num_recorded_errors(&self, error: ErrorType) -> i32 {
+ self.inner.test_get_num_recorded_errors(error)
+ }
+}
+
+#[cfg(test)]
+mod test {
+ use super::*;
+ use crate::common_test::{lock_test, new_glean};
+ use crate::CommonMetricData;
+
+ use serde_json::json;
+
+ #[test]
+ fn simple_array() {
+ let _lock = lock_test();
+ let _t = new_glean(None, true);
+
+ type SimpleArray = Vec;
+
+ let metric: ObjectMetric = ObjectMetric::new(CommonMetricData {
+ name: "object".into(),
+ category: "test".into(),
+ send_in_pings: vec!["test1".into()],
+ ..Default::default()
+ });
+
+ let arr = SimpleArray::from([1, 2, 3]);
+ metric.set(arr);
+
+ let data = metric.test_get_value(None).expect("no object recorded");
+ let expected = json!([1, 2, 3]);
+ assert_eq!(expected, data);
+ }
+
+ #[test]
+ fn complex_nested_object() {
+ let _lock = lock_test();
+ let _t = new_glean(None, true);
+
+ type BalloonsObject = Vec;
+
+ #[derive(
+ Debug, Hash, Eq, PartialEq, traits::__serde::Deserialize, traits::__serde::Serialize,
+ )]
+ #[serde(crate = "traits::__serde")]
+ #[serde(deny_unknown_fields)]
+ struct BalloonsObjectItem {
+ #[serde(skip_serializing_if = "Option::is_none")]
+ colour: Option,
+ #[serde(skip_serializing_if = "Option::is_none")]
+ diameter: Option,
+ }
+
+ let metric: ObjectMetric = ObjectMetric::new(CommonMetricData {
+ name: "object".into(),
+ category: "test".into(),
+ send_in_pings: vec!["test1".into()],
+ ..Default::default()
+ });
+
+ let balloons = BalloonsObject::from([
+ BalloonsObjectItem {
+ colour: Some("red".to_string()),
+ diameter: Some(5),
+ },
+ BalloonsObjectItem {
+ colour: Some("green".to_string()),
+ diameter: None,
+ },
+ ]);
+ metric.set(balloons);
+
+ let data = metric.test_get_value(None).expect("no object recorded");
+ let expected = json!([
+ { "colour": "red", "diameter": 5 },
+ { "colour": "green" },
+ ]);
+ assert_eq!(expected, data);
+ }
+
+ #[test]
+ fn set_string_api() {
+ let _lock = lock_test();
+ let _t = new_glean(None, true);
+
+ type SimpleArray = Vec;
+
+ let metric: ObjectMetric = ObjectMetric::new(CommonMetricData {
+ name: "object".into(),
+ category: "test".into(),
+ send_in_pings: vec!["test1".into()],
+ ..Default::default()
+ });
+
+ let arr_str = String::from("[1, 2, 3]");
+ metric.set_string(arr_str);
+
+ let data = metric.test_get_value(None).expect("no object recorded");
+ let expected = json!([1, 2, 3]);
+ assert_eq!(expected, data);
+ }
+}
diff --git a/glean-core/rlb/src/private/ping.rs b/glean-core/rlb/src/private/ping.rs
index c9c68a10a2..6c126992bc 100644
--- a/glean-core/rlb/src/private/ping.rs
+++ b/glean-core/rlb/src/private/ping.rs
@@ -33,6 +33,7 @@ impl PingType {
include_client_id: bool,
send_if_empty: bool,
precise_timestamps: bool,
+ include_info_sections: bool,
reason_codes: Vec,
) -> Self {
let inner = glean_core::metrics::PingType::new(
@@ -40,6 +41,7 @@ impl PingType {
include_client_id,
send_if_empty,
precise_timestamps,
+ include_info_sections,
reason_codes,
);
diff --git a/glean-core/rlb/src/test.rs b/glean-core/rlb/src/test.rs
index cb41b49b66..16d6d05447 100644
--- a/glean-core/rlb/src/test.rs
+++ b/glean-core/rlb/src/test.rs
@@ -22,22 +22,16 @@ use crate::common_test::{lock_test, new_glean, GLOBAL_APPLICATION_ID};
fn send_a_ping() {
let _lock = lock_test();
- let (s, r) = crossbeam_channel::bounded::(1);
+ let (s, r) = crossbeam_channel::bounded::(1);
- // Define a fake uploader that reports back the submission URL
- // using a crossbeam channel.
+ // Define a fake uploader that reports back the ping upload request.
#[derive(Debug)]
pub struct FakeUploader {
- sender: crossbeam_channel::Sender,
+ sender: crossbeam_channel::Sender,
}
impl net::PingUploader for FakeUploader {
- fn upload(
- &self,
- url: String,
- _body: Vec,
- _headers: Vec<(String, String)>,
- ) -> net::UploadResult {
- self.sender.send(url).unwrap();
+ fn upload(&self, upload_request: net::PingUploadRequest) -> net::UploadResult {
+ self.sender.send(upload_request).unwrap();
net::UploadResult::http_status(200)
}
}
@@ -55,12 +49,54 @@ fn send_a_ping() {
// Define a new ping and submit it.
const PING_NAME: &str = "test-ping";
- let custom_ping = private::PingType::new(PING_NAME, true, true, true, vec![]);
+ let custom_ping = private::PingType::new(PING_NAME, true, true, true, true, vec![]);
custom_ping.submit(None);
// Wait for the ping to arrive.
- let url = r.recv().unwrap();
- assert!(url.contains(PING_NAME));
+ let upload_request = r.recv().unwrap();
+ assert!(upload_request.body_has_info_sections);
+ assert_eq!(upload_request.ping_name, PING_NAME);
+ assert!(upload_request.url.contains(PING_NAME));
+}
+
+#[test]
+fn send_a_ping_without_info_sections() {
+ let _lock = lock_test();
+
+ let (s, r) = crossbeam_channel::bounded::(1);
+
+ // Define a fake uploader that reports back the ping upload request.
+ #[derive(Debug)]
+ pub struct FakeUploader {
+ sender: crossbeam_channel::Sender,
+ }
+ impl net::PingUploader for FakeUploader {
+ fn upload(&self, upload_request: net::PingUploadRequest) -> net::UploadResult {
+ self.sender.send(upload_request).unwrap();
+ net::UploadResult::http_status(200)
+ }
+ }
+
+ // Create a custom configuration to use a fake uploader.
+ let dir = tempfile::tempdir().unwrap();
+ let tmpname = dir.path().to_path_buf();
+
+ let cfg = ConfigurationBuilder::new(true, tmpname, GLOBAL_APPLICATION_ID)
+ .with_server_endpoint("invalid-test-host")
+ .with_uploader(FakeUploader { sender: s })
+ .build();
+
+ let _t = new_glean(Some(cfg), true);
+
+ // Define a new ping and submit it.
+ const PING_NAME: &str = "noinfo-ping";
+ let custom_ping = private::PingType::new(PING_NAME, true, true, true, false, vec![]);
+ custom_ping.submit(None);
+
+ // Wait for the ping to arrive.
+ let upload_request = r.recv().unwrap();
+ assert!(!upload_request.body_has_info_sections);
+ assert_eq!(upload_request.ping_name, PING_NAME);
}
#[test]
@@ -190,13 +226,8 @@ fn sending_of_foreground_background_pings() {
sender: crossbeam_channel::Sender,
}
impl net::PingUploader for FakeUploader {
- fn upload(
- &self,
- url: String,
- _body: Vec,
- _headers: Vec<(String, String)>,
- ) -> net::UploadResult {
- self.sender.send(url).unwrap();
+ fn upload(&self, upload_request: net::PingUploadRequest) -> net::UploadResult {
+ self.sender.send(upload_request.url).unwrap();
net::UploadResult::http_status(200)
}
}
@@ -263,13 +294,8 @@ fn sending_of_startup_baseline_ping() {
sender: crossbeam_channel::Sender,
}
impl net::PingUploader for FakeUploader {
- fn upload(
- &self,
- url: String,
- _body: Vec,
- _headers: Vec<(String, String)>,
- ) -> net::UploadResult {
- self.sender.send(url).unwrap();
+ fn upload(&self, upload_request: net::PingUploadRequest) -> net::UploadResult {
+ self.sender.send(upload_request.url).unwrap();
net::UploadResult::http_status(200)
}
}
@@ -315,13 +341,8 @@ fn no_dirty_baseline_on_clean_shutdowns() {
sender: crossbeam_channel::Sender,
}
impl net::PingUploader for FakeUploader {
- fn upload(
- &self,
- url: String,
- _body: Vec,
- _headers: Vec<(String, String)>,
- ) -> net::UploadResult {
- self.sender.send(url).unwrap();
+ fn upload(&self, upload_request: net::PingUploadRequest) -> net::UploadResult {
+ self.sender.send(upload_request.url).unwrap();
net::UploadResult::http_status(200)
}
}
@@ -543,12 +564,8 @@ fn ping_collection_must_happen_after_concurrently_scheduled_metrics_recordings()
sender: crossbeam_channel::Sender<(String, JsonValue)>,
}
impl net::PingUploader for FakeUploader {
- fn upload(
- &self,
- url: String,
- body: Vec,
- _headers: Vec<(String, String)>,
- ) -> net::UploadResult {
+ fn upload(&self, upload_request: net::PingUploadRequest) -> net::UploadResult {
+ let net::PingUploadRequest { body, url, .. } = upload_request;
// Decode the gzipped body.
let mut gzip_decoder = GzDecoder::new(&body[..]);
let mut s = String::with_capacity(body.len());
@@ -577,7 +594,7 @@ fn ping_collection_must_happen_after_concurrently_scheduled_metrics_recordings()
);
let ping_name = "custom_ping_1";
- let ping = private::PingType::new(ping_name, true, false, true, vec![]);
+ let ping = private::PingType::new(ping_name, true, false, true, true, vec![]);
let metric = private::StringMetric::new(CommonMetricData {
name: "string_metric".into(),
category: "telemetry".into(),
@@ -681,13 +698,8 @@ fn sending_deletion_ping_if_disabled_outside_of_run() {
sender: crossbeam_channel::Sender,
}
impl net::PingUploader for FakeUploader {
- fn upload(
- &self,
- url: String,
- _body: Vec,
- _headers: Vec<(String, String)>,
- ) -> net::UploadResult {
- self.sender.send(url).unwrap();
+ fn upload(&self, upload_request: net::PingUploadRequest) -> net::UploadResult {
+ self.sender.send(upload_request.url).unwrap();
net::UploadResult::http_status(200)
}
}
@@ -731,13 +743,8 @@ fn no_sending_of_deletion_ping_if_unchanged_outside_of_run() {
sender: crossbeam_channel::Sender,
}
impl net::PingUploader for FakeUploader {
- fn upload(
- &self,
- url: String,
- _body: Vec,
- _headers: Vec<(String, String)>,
- ) -> net::UploadResult {
- self.sender.send(url).unwrap();
+ fn upload(&self, upload_request: net::PingUploadRequest) -> net::UploadResult {
+ self.sender.send(upload_request.url).unwrap();
net::UploadResult::http_status(200)
}
}
@@ -779,12 +786,8 @@ fn deletion_request_ping_contains_experimentation_id() {
sender: crossbeam_channel::Sender,
}
impl net::PingUploader for FakeUploader {
- fn upload(
- &self,
- _url: String,
- body: Vec,
- _headers: Vec<(String, String)>,
- ) -> net::UploadResult {
+ fn upload(&self, upload_request: net::PingUploadRequest) -> net::UploadResult {
+ let body = upload_request.body;
let mut gzip_decoder = GzDecoder::new(&body[..]);
let mut body_str = String::with_capacity(body.len());
let data: JsonValue = gzip_decoder
@@ -847,12 +850,8 @@ fn test_sending_of_startup_baseline_ping_with_application_lifetime_metric() {
sender: crossbeam_channel::Sender<(String, JsonValue)>,
}
impl net::PingUploader for FakeUploader {
- fn upload(
- &self,
- url: String,
- body: Vec,
- _headers: Vec<(String, String)>,
- ) -> net::UploadResult {
+ fn upload(&self, upload_request: net::PingUploadRequest) -> net::UploadResult {
+ let net::PingUploadRequest { url, body, .. } = upload_request;
// Decode the gzipped body.
let mut gzip_decoder = GzDecoder::new(&body[..]);
let mut s = String::with_capacity(body.len());
@@ -932,13 +931,8 @@ fn setting_debug_view_tag_before_initialization_should_not_crash() {
sender: crossbeam_channel::Sender>,
}
impl net::PingUploader for FakeUploader {
- fn upload(
- &self,
- _url: String,
- _body: Vec,
- headers: Vec<(String, String)>,
- ) -> net::UploadResult {
- self.sender.send(headers).unwrap();
+ fn upload(&self, upload_request: net::PingUploadRequest) -> net::UploadResult {
+ self.sender.send(upload_request.headers).unwrap();
net::UploadResult::http_status(200)
}
}
@@ -983,13 +977,8 @@ fn setting_source_tags_before_initialization_should_not_crash() {
sender: crossbeam_channel::Sender>,
}
impl net::PingUploader for FakeUploader {
- fn upload(
- &self,
- _url: String,
- _body: Vec,
- headers: Vec<(String, String)>,
- ) -> net::UploadResult {
- self.sender.send(headers).unwrap();
+ fn upload(&self, upload_request: net::PingUploadRequest) -> net::UploadResult {
+ self.sender.send(upload_request.headers).unwrap();
net::UploadResult::http_status(200)
}
}
@@ -1033,13 +1022,8 @@ fn setting_source_tags_after_initialization_should_not_crash() {
sender: crossbeam_channel::Sender>,
}
impl net::PingUploader for FakeUploader {
- fn upload(
- &self,
- _url: String,
- _body: Vec,
- headers: Vec<(String, String)>,
- ) -> net::UploadResult {
- self.sender.send(headers).unwrap();
+ fn upload(&self, upload_request: net::PingUploadRequest) -> net::UploadResult {
+ self.sender.send(upload_request.headers).unwrap();
net::UploadResult::http_status(200)
}
}
@@ -1097,13 +1081,8 @@ fn flipping_upload_enabled_respects_order_of_events() {
sender: crossbeam_channel::Sender,
}
impl net::PingUploader for FakeUploader {
- fn upload(
- &self,
- url: String,
- _body: Vec,
- _headers: Vec<(String, String)>,
- ) -> net::UploadResult {
- self.sender.send(url).unwrap();
+ fn upload(&self, upload_request: net::PingUploadRequest) -> net::UploadResult {
+ self.sender.send(upload_request.url).unwrap();
net::UploadResult::http_status(200)
}
}
@@ -1118,7 +1097,7 @@ fn flipping_upload_enabled_respects_order_of_events() {
.build();
// We create a ping and a metric before we initialize Glean
- let sample_ping = PingType::new("sample-ping-1", true, false, true, vec![]);
+ let sample_ping = PingType::new("sample-ping-1", true, false, true, true, vec![]);
let metric = private::StringMetric::new(CommonMetricData {
name: "string_metric".into(),
category: "telemetry".into(),
@@ -1155,19 +1134,14 @@ fn registering_pings_before_init_must_work() {
sender: crossbeam_channel::Sender,
}
impl net::PingUploader for FakeUploader {
- fn upload(
- &self,
- url: String,
- _body: Vec,
- _headers: Vec<(String, String)>,
- ) -> net::UploadResult {
- self.sender.send(url).unwrap();
+ fn upload(&self, upload_request: net::PingUploadRequest) -> net::UploadResult {
+ self.sender.send(upload_request.url).unwrap();
net::UploadResult::http_status(200)
}
}
// Create a custom ping and attempt its registration.
- let sample_ping = PingType::new("pre-register", true, true, true, vec![]);
+ let sample_ping = PingType::new("pre-register", true, true, true, true, vec![]);
// Create a custom configuration to use a fake uploader.
let dir = tempfile::tempdir().unwrap();
@@ -1201,13 +1175,8 @@ fn test_a_ping_before_submission() {
sender: crossbeam_channel::Sender,
}
impl net::PingUploader for FakeUploader {
- fn upload(
- &self,
- url: String,
- _body: Vec,
- _headers: Vec<(String, String)>,
- ) -> net::UploadResult {
- self.sender.send(url).unwrap();
+ fn upload(&self, upload_request: net::PingUploadRequest) -> net::UploadResult {
+ self.sender.send(upload_request.url).unwrap();
net::UploadResult::http_status(200)
}
}
@@ -1224,7 +1193,7 @@ fn test_a_ping_before_submission() {
let _t = new_glean(Some(cfg), true);
// Create a custom ping and register it.
- let sample_ping = PingType::new("custom1", true, true, true, vec![]);
+ let sample_ping = PingType::new("custom1", true, true, true, true, vec![]);
let metric = CounterMetric::new(CommonMetricData {
name: "counter_metric".into(),
@@ -1308,12 +1277,7 @@ fn signaling_done() {
counter: Arc>>,
}
impl net::PingUploader for FakeUploader {
- fn upload(
- &self,
- _url: String,
- _body: Vec,
- _headers: Vec<(String, String)>,
- ) -> net::UploadResult {
+ fn upload(&self, _upload_request: net::PingUploadRequest) -> net::UploadResult {
let mut map = self.counter.lock().unwrap();
*map.entry(thread::current().id()).or_insert(0) += 1;
@@ -1346,7 +1310,7 @@ fn signaling_done() {
// Define a new ping and submit it.
const PING_NAME: &str = "test-ping";
- let custom_ping = private::PingType::new(PING_NAME, true, true, true, vec![]);
+ let custom_ping = private::PingType::new(PING_NAME, true, true, true, true, vec![]);
custom_ping.submit(None);
custom_ping.submit(None);
@@ -1385,17 +1349,12 @@ fn configure_ping_throttling() {
done: Arc,
}
impl net::PingUploader for FakeUploader {
- fn upload(
- &self,
- url: String,
- _body: Vec,
- _headers: Vec<(String, String)>,
- ) -> net::UploadResult {
+ fn upload(&self, upload_request: net::PingUploadRequest) -> net::UploadResult {
if self.done.load(std::sync::atomic::Ordering::SeqCst) {
// If we've outlived the test, just lie.
return net::UploadResult::http_status(200);
}
- self.sender.send(url).unwrap();
+ self.sender.send(upload_request.url).unwrap();
net::UploadResult::http_status(200)
}
}
@@ -1422,7 +1381,7 @@ fn configure_ping_throttling() {
// Define a new ping.
const PING_NAME: &str = "test-ping";
- let custom_ping = private::PingType::new(PING_NAME, true, true, true, vec![]);
+ let custom_ping = private::PingType::new(PING_NAME, true, true, true, true, vec![]);
// Submit and receive it `pings_per_interval` times.
for _ in 0..pings_per_interval {
diff --git a/glean-core/rlb/tests/init_fails.rs b/glean-core/rlb/tests/init_fails.rs
index def5acc4b9..a0c23ca277 100644
--- a/glean-core/rlb/tests/init_fails.rs
+++ b/glean-core/rlb/tests/init_fails.rs
@@ -43,7 +43,7 @@ mod pings {
#[allow(non_upper_case_globals)]
pub static validation: Lazy =
- Lazy::new(|| glean::private::PingType::new("validation", true, true, true, vec![]));
+ Lazy::new(|| glean::private::PingType::new("validation", true, true, true, true, vec![]));
}
/// Test scenario: Glean initialization fails.
diff --git a/glean-core/rlb/tests/never_init.rs b/glean-core/rlb/tests/never_init.rs
index 3df472ee31..0d0d3768ff 100644
--- a/glean-core/rlb/tests/never_init.rs
+++ b/glean-core/rlb/tests/never_init.rs
@@ -39,7 +39,7 @@ mod pings {
#[allow(non_upper_case_globals)]
pub static validation: Lazy =
- Lazy::new(|| glean::private::PingType::new("validation", true, true, true, vec![]));
+ Lazy::new(|| glean::private::PingType::new("validation", true, true, true, true, vec![]));
}
/// Test scenario: Glean is never initialized.
diff --git a/glean-core/rlb/tests/no_time_to_init.rs b/glean-core/rlb/tests/no_time_to_init.rs
index 763835f2f3..c312b397af 100644
--- a/glean-core/rlb/tests/no_time_to_init.rs
+++ b/glean-core/rlb/tests/no_time_to_init.rs
@@ -41,7 +41,7 @@ mod pings {
#[allow(non_upper_case_globals)]
pub static validation: Lazy =
- Lazy::new(|| glean::private::PingType::new("validation", true, true, true, vec![]));
+ Lazy::new(|| glean::private::PingType::new("validation", true, true, true, true, vec![]));
}
/// Test scenario: Glean initialization fails.
diff --git a/glean-core/rlb/tests/schema.rs b/glean-core/rlb/tests/schema.rs
index 0a1bf4d2e8..01a2108b3c 100644
--- a/glean-core/rlb/tests/schema.rs
+++ b/glean-core/rlb/tests/schema.rs
@@ -10,7 +10,7 @@ use glean_core::TextMetric;
use jsonschema_valid::{self, schemas::Draft};
use serde_json::Value;
-use glean::net::UploadResult;
+use glean::net::{PingUploadRequest, UploadResult};
use glean::private::*;
use glean::{
traits, ClientInfoMetrics, CommonMetricData, ConfigurationBuilder, HistogramType, MemoryUnit,
@@ -60,13 +60,8 @@ fn validate_against_schema() {
sender: crossbeam_channel::Sender>,
}
impl glean::net::PingUploader for ValidatingUploader {
- fn upload(
- &self,
- _url: String,
- body: Vec,
- _headers: Vec<(String, String)>,
- ) -> UploadResult {
- self.sender.send(body).unwrap();
+ fn upload(&self, ping_request: PingUploadRequest) -> UploadResult {
+ self.sender.send(ping_request.body).unwrap();
UploadResult::http_status(200)
}
}
@@ -176,7 +171,7 @@ fn validate_against_schema() {
text_metric.set("loooooong text".repeat(100));
// Define a new ping and submit it.
- let custom_ping = glean::private::PingType::new(PING_NAME, true, true, true, vec![]);
+ let custom_ping = glean::private::PingType::new(PING_NAME, true, true, true, true, vec![]);
custom_ping.submit(None);
// Wait for the ping to arrive.
diff --git a/glean-core/rlb/tests/simple.rs b/glean-core/rlb/tests/simple.rs
index 3685d44faa..3baa4df14e 100644
--- a/glean-core/rlb/tests/simple.rs
+++ b/glean-core/rlb/tests/simple.rs
@@ -41,7 +41,7 @@ mod pings {
#[allow(non_upper_case_globals)]
pub static validation: Lazy =
- Lazy::new(|| glean::private::PingType::new("validation", true, true, true, vec![]));
+ Lazy::new(|| glean::private::PingType::new("validation", true, true, true, true, vec![]));
}
/// Test scenario: A clean run
diff --git a/glean-core/rlb/tests/upload_timing.rs b/glean-core/rlb/tests/upload_timing.rs
index 9e77fc3eb5..ba0eee3402 100644
--- a/glean-core/rlb/tests/upload_timing.rs
+++ b/glean-core/rlb/tests/upload_timing.rs
@@ -97,7 +97,7 @@ mod pings {
#[allow(non_upper_case_globals)]
pub static validation: Lazy =
- Lazy::new(|| glean::private::PingType::new("validation", true, true, true, vec![]));
+ Lazy::new(|| glean::private::PingType::new("validation", true, true, true, true, vec![]));
}
// Define a fake uploader that sleeps.
@@ -108,13 +108,9 @@ struct FakeUploader {
}
impl net::PingUploader for FakeUploader {
- fn upload(
- &self,
- _url: String,
- body: Vec,
- _headers: Vec<(String, String)>,
- ) -> net::UploadResult {
+ fn upload(&self, upload_request: net::PingUploadRequest) -> net::UploadResult {
let calls = self.calls.fetch_add(1, Ordering::SeqCst);
+ let body = upload_request.body;
let decode = |body: Vec| {
let mut gzip_decoder = GzDecoder::new(&body[..]);
let mut s = String::with_capacity(body.len());
diff --git a/glean-core/src/core/mod.rs b/glean-core/src/core/mod.rs
index 5a8dd56cde..85691b496c 100644
--- a/glean-core/src/core/mod.rs
+++ b/glean-core/src/core/mod.rs
@@ -122,7 +122,7 @@ where
/// experimentation_id: None,
/// };
/// let mut glean = Glean::new(cfg).unwrap();
-/// let ping = PingType::new("sample", true, false, true, vec![]);
+/// let ping = PingType::new("sample", true, false, true, true, vec![]);
/// glean.register_ping_type(&ping);
///
/// let call_counter: CounterMetric = CounterMetric::new(CommonMetricData {
diff --git a/glean-core/src/glean.udl b/glean-core/src/glean.udl
index e531f64a26..e31de42e70 100644
--- a/glean-core/src/glean.udl
+++ b/glean-core/src/glean.udl
@@ -201,6 +201,10 @@ dictionary PingRequest {
sequence body;
// A map with all the headers to be sent with the request.
record headers;
+ // Whether the body has {client|ping}_info sections.
+ boolean body_has_info_sections;
+ // The ping's name. Likely also somewhere in `path`.
+ string ping_name;
};
// An enum representing the possible upload tasks to be performed by an uploader.
@@ -287,7 +291,7 @@ enum ErrorType {
};
interface PingType {
- constructor(string name, boolean include_client_id, boolean send_if_empty, boolean precise_timestamps, sequence reason_codes);
+ constructor(string name, boolean include_client_id, boolean send_if_empty, boolean precise_timestamps, boolean include_info_sections, sequence reason_codes);
void submit(optional string? reason = null);
};
diff --git a/glean-core/src/internal_pings.rs b/glean-core/src/internal_pings.rs
index 076a1f7485..07c3849006 100644
--- a/glean-core/src/internal_pings.rs
+++ b/glean-core/src/internal_pings.rs
@@ -26,6 +26,7 @@ impl InternalPings {
true,
true,
true,
+ true,
vec![
"active".to_string(),
"dirty_startup".to_string(),
@@ -37,6 +38,7 @@ impl InternalPings {
true,
false,
true,
+ true,
vec![
"overdue".to_string(),
"reschedule".to_string(),
@@ -50,6 +52,7 @@ impl InternalPings {
true,
false,
true,
+ true,
vec![
"startup".to_string(),
"inactive".to_string(),
@@ -61,6 +64,7 @@ impl InternalPings {
true,
true,
true,
+ true,
vec!["at_init".to_string(), "set_upload_enabled".to_string()],
),
}
diff --git a/glean-core/src/lib.rs b/glean-core/src/lib.rs
index a54e57a95b..b7f9d73beb 100644
--- a/glean-core/src/lib.rs
+++ b/glean-core/src/lib.rs
@@ -91,12 +91,12 @@ pub(crate) const DELETION_REQUEST_PINGS_DIRECTORY: &str = "deletion_request";
static INITIALIZE_CALLED: AtomicBool = AtomicBool::new(false);
/// Keep track of the debug features before Glean is initialized.
-static PRE_INIT_DEBUG_VIEW_TAG: OnceCell> = OnceCell::new();
+static PRE_INIT_DEBUG_VIEW_TAG: Mutex = Mutex::new(String::new());
static PRE_INIT_LOG_PINGS: AtomicBool = AtomicBool::new(false);
-static PRE_INIT_SOURCE_TAGS: OnceCell>> = OnceCell::new();
+static PRE_INIT_SOURCE_TAGS: Mutex> = Mutex::new(Vec::new());
/// Keep track of pings registered before Glean is initialized.
-static PRE_INIT_PING_REGISTRATION: OnceCell>> = OnceCell::new();
+static PRE_INIT_PING_REGISTRATION: Mutex> = Mutex::new(Vec::new());
/// Global singleton of the handles of the glean.init threads.
/// For joining. For tests.
@@ -396,11 +396,9 @@ fn initialize_inner(
core::with_glean_mut(|glean| {
// The debug view tag might have been set before initialize,
// get the cached value and set it.
- if let Some(tag) = PRE_INIT_DEBUG_VIEW_TAG.get() {
- let lock = tag.try_lock();
- if let Ok(ref debug_tag) = lock {
- glean.set_debug_view_tag(debug_tag);
- }
+ let debug_tag = PRE_INIT_DEBUG_VIEW_TAG.lock().unwrap();
+ if debug_tag.len() > 0 {
+ glean.set_debug_view_tag(&debug_tag);
}
// The log pings debug option might have been set before initialize,
@@ -412,11 +410,9 @@ fn initialize_inner(
// The source tags might have been set before initialize,
// get the cached value and set them.
- if let Some(tags) = PRE_INIT_SOURCE_TAGS.get() {
- let lock = tags.try_lock();
- if let Ok(ref source_tags) = lock {
- glean.set_source_tags(source_tags.to_vec());
- }
+ let source_tags = PRE_INIT_SOURCE_TAGS.lock().unwrap();
+ if source_tags.len() > 0 {
+ glean.set_source_tags(source_tags.to_vec());
}
// Get the current value of the dirty flag so we know whether to
@@ -428,13 +424,9 @@ fn initialize_inner(
// Perform registration of pings that were attempted to be
// registered before init.
- if let Some(tags) = PRE_INIT_PING_REGISTRATION.get() {
- let lock = tags.try_lock();
- if let Ok(pings) = lock {
- for ping in &*pings {
- glean.register_ping_type(ping);
- }
- }
+ let pings = PRE_INIT_PING_REGISTRATION.lock().unwrap();
+ for ping in pings.iter() {
+ glean.register_ping_type(ping);
}
// If this is the first time ever the Glean SDK runs, make sure to set
@@ -861,7 +853,7 @@ pub(crate) fn register_ping_type(ping: &PingType) {
// if ping registration is attempted before Glean initializes.
// This state is kept across Glean resets, which should only ever happen in test mode.
// It's a set and keeping them around forever should not have much of an impact.
- let m = PRE_INIT_PING_REGISTRATION.get_or_init(Default::default);
+ let m = &PRE_INIT_PING_REGISTRATION;
let mut lock = m.lock().unwrap();
lock.push(ping.clone());
}
@@ -956,7 +948,7 @@ pub fn glean_set_debug_view_tag(tag: String) -> bool {
true
} else {
// Glean has not been initialized yet. Cache the provided tag value.
- let m = PRE_INIT_DEBUG_VIEW_TAG.get_or_init(Default::default);
+ let m = &PRE_INIT_DEBUG_VIEW_TAG;
let mut lock = m.lock().unwrap();
*lock = tag;
// When setting the debug view tag before initialization,
@@ -984,7 +976,7 @@ pub fn glean_set_source_tags(tags: Vec) -> bool {
true
} else {
// Glean has not been initialized yet. Cache the provided source tags.
- let m = PRE_INIT_SOURCE_TAGS.get_or_init(Default::default);
+ let m = &PRE_INIT_SOURCE_TAGS;
let mut lock = m.lock().unwrap();
*lock = tags;
// When setting the source tags before initialization,
diff --git a/glean-core/src/lib_unit_tests.rs b/glean-core/src/lib_unit_tests.rs
index 0fc85b4602..7d34e2613f 100644
--- a/glean-core/src/lib_unit_tests.rs
+++ b/glean-core/src/lib_unit_tests.rs
@@ -423,6 +423,7 @@ fn correct_order() {
Jwe("eyJhbGciOiJSU0EtT0FFUCIsImVuYyI6IkEyNTZHQ00ifQ.OKOawDo13gRp2ojaHV7LFpZcgV7T6DVZKTyKOMTYUmKoTCVJRgckCL9kiMT03JGeipsEdY3mx_etLbbWSrFr05kLzcSr4qKAq7YN7e9jwQRb23nfa6c9d-StnImGyFDbSv04uVuxIp5Zms1gNxKKK2Da14B8S4rzVRltdYwam_lDp5XnZAYpQdb76FdIKLaVmqgfwX7XWRxv2322i-vDxRfqNzo_tETKzpVLzfiwQyeyPGLBIO56YJ7eObdv0je81860ppamavo35UgoRdbYaBcoh9QcfylQr66oc6vFWXRcZ_ZT2LawVCWTIy3brGPi6UklfCpIMfIjf7iGdXKHzg.48V1_ALb6US04U3b.5eym8TW_c8SuK0ltJ3rpYIzOeDQz7TALvtu6UG9oMo4vpzs9tX_EFShS8iB7j6jiSdiwkIr3ajwQzaBtQD_A.XFBoMYUZodetZdvTiFvSkQ".into()),
Rate(0, 0),
Text(long_string),
+ Object("{}".into()),
];
for metric in all_metrics {
@@ -451,6 +452,7 @@ fn correct_order() {
Rate(..) => assert_eq!(14, disc),
Url(..) => assert_eq!(15, disc),
Text(..) => assert_eq!(16, disc),
+ Object(..) => assert_eq!(17, disc),
}
}
}
diff --git a/glean-core/src/metrics/boolean.rs b/glean-core/src/metrics/boolean.rs
index 71ed2372c2..ade4a22bfc 100644
--- a/glean-core/src/metrics/boolean.rs
+++ b/glean-core/src/metrics/boolean.rs
@@ -74,7 +74,6 @@ impl BooleanMetric {
///
/// # Arguments
///
- /// * `glean` - the Glean instance this metric belongs to.
/// * `value` - the value to set.
pub fn set(&self, value: bool) {
let metric = self.clone();
@@ -106,6 +105,15 @@ impl BooleanMetric {
/// Gets the currently stored value as an integer.
///
/// This doesn't clear the stored value.
+ ///
+ /// # Arguments
+ ///
+ /// * `ping_name` - the optional name of the ping to retrieve the metric
+ /// for. Defaults to the first value in `send_in_pings`.
+ ///
+ /// # Returns
+ ///
+ /// The stored value or `None` if nothing stored.
pub fn test_get_value(&self, ping_name: Option) -> Option {
crate::block_on_dispatcher();
crate::core::with_glean(|glean| self.get_value(glean, ping_name.as_deref()))
@@ -118,8 +126,6 @@ impl BooleanMetric {
/// # Arguments
///
/// * `error` - The type of error
- /// * `ping_name` - represents the optional name of the ping to retrieve the
- /// metric for. inner to the first value in `send_in_pings`.
///
/// # Returns
///
diff --git a/glean-core/src/metrics/counter.rs b/glean-core/src/metrics/counter.rs
index 8f0a01cc3e..7e262c7d68 100644
--- a/glean-core/src/metrics/counter.rs
+++ b/glean-core/src/metrics/counter.rs
@@ -105,7 +105,6 @@ impl CounterMetric {
///
/// # Arguments
///
- /// * `glean` - The Glean instance this metric belongs to.
/// * `amount` - The amount to increase by. Should be positive.
///
/// ## Notes
@@ -143,6 +142,15 @@ impl CounterMetric {
/// Gets the currently stored value as an integer.
///
/// This doesn't clear the stored value.
+ ///
+ /// # Arguments
+ ///
+ /// * `ping_name` - the optional name of the ping to retrieve the metric
+ /// for. Defaults to the first value in `send_in_pings`.
+ ///
+ /// # Returns
+ ///
+ /// The stored value or `None` if nothing stored.
pub fn test_get_value(&self, ping_name: Option) -> Option {
crate::block_on_dispatcher();
crate::core::with_glean(|glean| self.get_value(glean, ping_name.as_deref()))
@@ -155,8 +163,6 @@ impl CounterMetric {
/// # Arguments
///
/// * `error` - The type of error
- /// * `ping_name` - represents the optional name of the ping to retrieve the
- /// metric for. inner to the first value in `send_in_pings`.
///
/// # Returns
///
diff --git a/glean-core/src/metrics/custom_distribution.rs b/glean-core/src/metrics/custom_distribution.rs
index 929e4863ec..d14be78dc0 100644
--- a/glean-core/src/metrics/custom_distribution.rs
+++ b/glean-core/src/metrics/custom_distribution.rs
@@ -194,6 +194,15 @@ impl CustomDistributionMetric {
/// Gets the currently stored value as an integer.
///
/// This doesn't clear the stored value.
+ ///
+ /// # Arguments
+ ///
+ /// * `ping_name` - the optional name of the ping to retrieve the metric
+ /// for. Defaults to the first value in `send_in_pings`.
+ ///
+ /// # Returns
+ ///
+ /// The stored value or `None` if nothing stored.
pub fn test_get_value(&self, ping_name: Option) -> Option {
crate::block_on_dispatcher();
crate::core::with_glean(|glean| self.get_value(glean, ping_name.as_deref()))
@@ -206,8 +215,6 @@ impl CustomDistributionMetric {
/// # Arguments
///
/// * `error` - The type of error
- /// * `ping_name` - represents the optional name of the ping to retrieve the
- /// metric for. inner to the first value in `send_in_pings`.
///
/// # Returns
///
diff --git a/glean-core/src/metrics/datetime.rs b/glean-core/src/metrics/datetime.rs
index 3ef846a32c..e04f7fc051 100644
--- a/glean-core/src/metrics/datetime.rs
+++ b/glean-core/src/metrics/datetime.rs
@@ -262,8 +262,8 @@ impl DatetimeMetric {
///
/// # Arguments
///
- /// * `glean` - the Glean instance this metric belongs to.
- /// * `storage_name` - the storage name to look into.
+ /// * `ping_name` - the optional name of the ping to retrieve the metric
+ /// for. Defaults to the first value in `send_in_pings`.
///
/// # Returns
///
@@ -284,8 +284,8 @@ impl DatetimeMetric {
///
/// # Arguments
///
- /// * `glean` - the Glean instance this metric belongs to.
- /// * `storage_name` - the storage name to look into.
+ /// * `ping_name` - the optional name of the ping to retrieve the metric
+ /// for. Defaults to the first value in `send_in_pings`.
///
/// # Returns
///
@@ -311,8 +311,6 @@ impl DatetimeMetric {
/// # Arguments
///
/// * `error` - The type of error
- /// * `ping_name` - represents the optional name of the ping to retrieve the
- /// metric for. inner to the first value in `send_in_pings`.
///
/// # Returns
///
diff --git a/glean-core/src/metrics/denominator.rs b/glean-core/src/metrics/denominator.rs
index fb80874924..3083d6e78a 100644
--- a/glean-core/src/metrics/denominator.rs
+++ b/glean-core/src/metrics/denominator.rs
@@ -91,6 +91,15 @@ impl DenominatorMetric {
/// Gets the currently stored value as an integer.
///
/// This doesn't clear the stored value.
+ ///
+ /// # Arguments
+ ///
+ /// * `ping_name` - the optional name of the ping to retrieve the metric
+ /// for. Defaults to the first value in `send_in_pings`.
+ ///
+ /// # Returns
+ ///
+ /// The stored value or `None` if nothing stored.
pub fn test_get_value(&self, ping_name: Option) -> Option {
crate::block_on_dispatcher();
crate::core::with_glean(|glean| self.get_value(glean, ping_name.as_deref()))
@@ -124,8 +133,6 @@ impl DenominatorMetric {
/// # Arguments
///
/// * `error` - The type of error
- /// * `ping_name` - the optional name of the ping to retrieve the metric
- /// for. inner to the first value in `send_in_pings`.
///
/// # Returns
///
diff --git a/glean-core/src/metrics/event.rs b/glean-core/src/metrics/event.rs
index 5ad6e6d50c..c7aefd9cd6 100644
--- a/glean-core/src/metrics/event.rs
+++ b/glean-core/src/metrics/event.rs
@@ -185,6 +185,11 @@ impl EventMetric {
/// Get the vector of currently stored events for this event metric.
///
/// This doesn't clear the stored value.
+ ///
+ /// # Arguments
+ ///
+ /// * `ping_name` - the optional name of the ping to retrieve the metric
+ /// for. Defaults to the first value in `send_in_pings`.
pub fn test_get_value(&self, ping_name: Option) -> Option> {
crate::block_on_dispatcher();
crate::core::with_glean(|glean| self.get_value(glean, ping_name.as_deref()))
@@ -197,8 +202,6 @@ impl EventMetric {
/// # Arguments
///
/// * `error` - The type of error
- /// * `ping_name` - represents the optional name of the ping to retrieve the
- /// metric for. inner to the first value in `send_in_pings`.
///
/// # Returns
///
diff --git a/glean-core/src/metrics/experiment.rs b/glean-core/src/metrics/experiment.rs
index 23e6c41ce2..5695bf942e 100644
--- a/glean-core/src/metrics/experiment.rs
+++ b/glean-core/src/metrics/experiment.rs
@@ -195,6 +195,15 @@ impl ExperimentMetric {
/// the RecordedExperiment.
///
/// This doesn't clear the stored value.
+ ///
+ /// # Arguments
+ ///
+ /// * `ping_name` - the optional name of the ping to retrieve the metric
+ /// for. Defaults to the first value in `send_in_pings`.
+ ///
+ /// # Returns
+ ///
+ /// The stored value or `None` if nothing stored.
pub fn test_get_value(&self, glean: &Glean) -> Option {
match StorageManager.snapshot_metric_for_test(
glean.storage(),
diff --git a/glean-core/src/metrics/labeled.rs b/glean-core/src/metrics/labeled.rs
index fa3e6a6a75..f9f6a28880 100644
--- a/glean-core/src/metrics/labeled.rs
+++ b/glean-core/src/metrics/labeled.rs
@@ -205,8 +205,6 @@ where
/// # Arguments
///
/// * `error` - The type of error
- /// * `ping_name` - represents the optional name of the ping to retrieve the
- /// metric for. Defaults to the first value in `send_in_pings`.
///
/// # Returns
///
diff --git a/glean-core/src/metrics/memory_distribution.rs b/glean-core/src/metrics/memory_distribution.rs
index ac9eda1a90..7b5e5ee192 100644
--- a/glean-core/src/metrics/memory_distribution.rs
+++ b/glean-core/src/metrics/memory_distribution.rs
@@ -254,6 +254,15 @@ impl MemoryDistributionMetric {
/// Gets the currently stored value.
///
/// This doesn't clear the stored value.
+ ///
+ /// # Arguments
+ ///
+ /// * `ping_name` - the optional name of the ping to retrieve the metric
+ /// for. Defaults to the first value in `send_in_pings`.
+ ///
+ /// # Returns
+ ///
+ /// The stored value or `None` if nothing stored.
pub fn test_get_value(&self, ping_name: Option) -> Option {
crate::block_on_dispatcher();
crate::core::with_glean(|glean| self.get_value(glean, ping_name.as_deref()))
@@ -266,8 +275,6 @@ impl MemoryDistributionMetric {
/// # Arguments
///
/// * `error` - The type of error
- /// * `ping_name` - represents the optional name of the ping to retrieve the
- /// metric for. Defaults to the first value in `send_in_pings`.
///
/// # Returns
///
diff --git a/glean-core/src/metrics/mod.rs b/glean-core/src/metrics/mod.rs
index 43253b9aa7..92001efd2a 100644
--- a/glean-core/src/metrics/mod.rs
+++ b/glean-core/src/metrics/mod.rs
@@ -9,7 +9,8 @@ use std::sync::atomic::Ordering;
use chrono::{DateTime, FixedOffset};
use serde::{Deserialize, Serialize};
-use serde_json::{json, Value as JsonValue};
+use serde_json::json;
+pub use serde_json::Value as JsonValue;
mod boolean;
mod counter;
@@ -23,6 +24,7 @@ mod memory_distribution;
mod memory_unit;
mod metrics_enabled_config;
mod numerator;
+mod object;
mod ping;
mod quantity;
mod rate;
@@ -54,6 +56,7 @@ pub use self::labeled::{LabeledBoolean, LabeledCounter, LabeledMetric, LabeledSt
pub use self::memory_distribution::MemoryDistributionMetric;
pub use self::memory_unit::MemoryUnit;
pub use self::numerator::NumeratorMetric;
+pub use self::object::ObjectMetric;
pub use self::ping::PingType;
pub use self::quantity::QuantityMetric;
pub use self::rate::{Rate, RateMetric};
@@ -141,6 +144,8 @@ pub enum Metric {
Url(String),
/// A Text metric. See [`TextMetric`] for more information.
Text(String),
+ /// An Object metric. See [`ObjectMetric`] for more information.
+ Object(String),
}
/// A [`MetricType`] describes common behavior across all metrics.
@@ -251,6 +256,7 @@ impl Metric {
Metric::MemoryDistribution(_) => "memory_distribution",
Metric::Jwe(_) => "jwe",
Metric::Text(_) => "text",
+ Metric::Object(_) => "object",
}
}
@@ -280,6 +286,9 @@ impl Metric {
Metric::MemoryDistribution(hist) => json!(memory_distribution::snapshot(hist)),
Metric::Jwe(s) => json!(s),
Metric::Text(s) => json!(s),
+ Metric::Object(s) => {
+ serde_json::from_str(s).expect("object storage should have been json")
+ }
}
}
}
diff --git a/glean-core/src/metrics/numerator.rs b/glean-core/src/metrics/numerator.rs
index 3c340cab1d..de29338a5c 100644
--- a/glean-core/src/metrics/numerator.rs
+++ b/glean-core/src/metrics/numerator.rs
@@ -55,12 +55,16 @@ impl NumeratorMetric {
///
/// Gets the currently stored value as a pair of integers.
///
+ /// This doesn't clear the stored value.
+ ///
/// # Arguments
///
/// * `ping_name` - the optional name of the ping to retrieve the metric
/// for. Defaults to the first value in `send_in_pings`.
///
- /// This doesn't clear the stored value.
+ /// # Returns
+ ///
+ /// The stored value or `None` if nothing stored.
pub fn test_get_value(&self, ping_name: Option) -> Option {
crate::block_on_dispatcher();
crate::core::with_glean(|glean| self.get_value(glean, ping_name.as_deref()))
@@ -82,8 +86,6 @@ impl NumeratorMetric {
/// # Arguments
///
/// * `error` - The type of error
- /// * `ping_name` - the optional name of the ping to retrieve the metric
- /// for. Defaults to the first value in `send_in_pings`.
///
/// # Returns
///
diff --git a/glean-core/src/metrics/object.rs b/glean-core/src/metrics/object.rs
new file mode 100644
index 0000000000..6071e2b33a
--- /dev/null
+++ b/glean-core/src/metrics/object.rs
@@ -0,0 +1,135 @@
+// This Source Code Form is subject to the terms of the Mozilla Public
+// License, v. 2.0. If a copy of the MPL was not distributed with this
+// file, You can obtain one at https://mozilla.org/MPL/2.0/.
+
+use std::sync::Arc;
+
+use crate::common_metric_data::CommonMetricDataInternal;
+use crate::error_recording::{record_error, test_get_num_recorded_errors, ErrorType};
+use crate::metrics::JsonValue;
+use crate::metrics::Metric;
+use crate::metrics::MetricType;
+use crate::storage::StorageManager;
+use crate::CommonMetricData;
+use crate::Glean;
+
+/// An object metric.
+///
+/// Record structured data.
+/// The value must adhere to a predefined structure and is serialized into JSON.
+#[derive(Clone, Debug)]
+pub struct ObjectMetric {
+ meta: Arc,
+}
+
+impl MetricType for ObjectMetric {
+ fn meta(&self) -> &CommonMetricDataInternal {
+ &self.meta
+ }
+}
+
+// IMPORTANT:
+//
+// When changing this implementation, make sure all the operations are
+// also declared in the related trait in `../traits/`.
+impl ObjectMetric {
+ /// Creates a new object metric.
+ pub fn new(meta: CommonMetricData) -> Self {
+ Self {
+ meta: Arc::new(meta.into()),
+ }
+ }
+
+ /// Sets to the specified structure.
+ ///
+ /// # Arguments
+ ///
+ /// * `glean` - the Glean instance this metric belongs to.
+ /// * `value` - the value to set.
+ #[doc(hidden)]
+ pub fn set_sync(&self, glean: &Glean, value: JsonValue) {
+ let value = Metric::Object(serde_json::to_string(&value).unwrap());
+ glean.storage().record(glean, &self.meta, &value)
+ }
+
+ /// Sets to the specified structure.
+ ///
+ /// No additional verification is done.
+ /// The shape needs to be externally verified.
+ ///
+ /// # Arguments
+ ///
+ /// * `value` - the value to set.
+ pub fn set(&self, value: JsonValue) {
+ let metric = self.clone();
+ crate::launch_with_glean(move |glean| metric.set_sync(glean, value))
+ }
+
+ /// Record an `InvalidValue` error for this metric.
+ ///
+ /// Only to be used by the RLB.
+ // TODO(bug 1691073): This can probably go once we have a more generic mechanism to record
+ // errors
+ pub fn record_schema_error(&self) {
+ let metric = self.clone();
+ crate::launch_with_glean(move |glean| {
+ let msg = "Value did not match predefined schema";
+ record_error(glean, &metric.meta, ErrorType::InvalidValue, msg, None);
+ });
+ }
+
+ /// Get current value
+ #[doc(hidden)]
+ pub fn get_value<'a, S: Into>>(
+ &self,
+ glean: &Glean,
+ ping_name: S,
+ ) -> Option {
+ let queried_ping_name = ping_name
+ .into()
+ .unwrap_or_else(|| &self.meta().inner.send_in_pings[0]);
+
+ match StorageManager.snapshot_metric_for_test(
+ glean.storage(),
+ queried_ping_name,
+ &self.meta.identifier(glean),
+ self.meta.inner.lifetime,
+ ) {
+ Some(Metric::Object(o)) => Some(o),
+ _ => None,
+ }
+ }
+
+ /// **Test-only API (exported for FFI purposes).**
+ ///
+ /// Gets the currently stored value as JSON.
+ ///
+ /// This doesn't clear the stored value.
+ pub fn test_get_value(&self, ping_name: Option) -> Option {
+ crate::block_on_dispatcher();
+ let value = crate::core::with_glean(|glean| self.get_value(glean, ping_name.as_deref()));
+ // We only store valid JSON
+ value.map(|val| serde_json::from_str(&val).unwrap())
+ }
+
+ /// **Exported for test purposes.**
+ ///
+ /// Gets the number of recorded errors for the given metric and error type.
+ ///
+ /// # Arguments
+ ///
+ /// * `error` - The type of error
+ /// * `ping_name` - represents the optional name of the ping to retrieve the
+ /// metric for. inner to the first value in `send_in_pings`.
+ ///
+ /// # Returns
+ ///
+ /// The number of errors reported.
+ pub fn test_get_num_recorded_errors(&self, error: ErrorType) -> i32 {
+ crate::block_on_dispatcher();
+
+ crate::core::with_glean(|glean| {
+ test_get_num_recorded_errors(glean, self.meta(), error).unwrap_or(0)
+ })
+ }
+}
diff --git a/glean-core/src/metrics/ping.rs b/glean-core/src/metrics/ping.rs
index dc37d76a45..e60284b1e2 100644
--- a/glean-core/src/metrics/ping.rs
+++ b/glean-core/src/metrics/ping.rs
@@ -6,6 +6,7 @@ use std::fmt;
use std::sync::Arc;
use crate::ping::PingMaker;
+use crate::upload::PingPayload;
use crate::Glean;
use uuid::Uuid;
@@ -26,6 +27,8 @@ struct InnerPing {
pub send_if_empty: bool,
/// Whether to use millisecond-precise start/end times.
pub precise_timestamps: bool,
+ /// Whether to include the {client|ping}_info sections on assembly.
+ pub include_info_sections: bool,
/// The "reason" codes that this ping can send
pub reason_codes: Vec,
}
@@ -37,6 +40,7 @@ impl fmt::Debug for PingType {
.field("include_client_id", &self.0.include_client_id)
.field("send_if_empty", &self.0.send_if_empty)
.field("precise_timestamps", &self.0.precise_timestamps)
+ .field("include_info_sections", &self.0.include_info_sections)
.field("reason_codes", &self.0.reason_codes)
.finish()
}
@@ -61,6 +65,7 @@ impl PingType {
include_client_id: bool,
send_if_empty: bool,
precise_timestamps: bool,
+ include_info_sections: bool,
reason_codes: Vec,
) -> Self {
let this = Self(Arc::new(InnerPing {
@@ -68,6 +73,7 @@ impl PingType {
include_client_id,
send_if_empty,
precise_timestamps,
+ include_info_sections,
reason_codes,
}));
@@ -94,6 +100,10 @@ impl PingType {
self.0.precise_timestamps
}
+ pub(crate) fn include_info_sections(&self) -> bool {
+ self.0.include_info_sections
+ }
+
/// Submits the ping for eventual uploading.
///
/// The ping content is assembled as soon as possible, but upload is not
@@ -186,13 +196,17 @@ impl PingType {
// so both scenarios should be impossible.
let content =
::serde_json::to_string(&ping.content).expect("ping serialization failed");
- glean.upload_manager.enqueue_ping(
- glean,
- ping.doc_id,
- ping.url_path,
- &content,
- Some(ping.headers),
- );
+ // TODO: Shouldn't we consolidate on a single collected Ping representation?
+ let ping = PingPayload {
+ document_id: ping.doc_id.to_string(),
+ upload_path: ping.url_path.to_string(),
+ json_body: content,
+ headers: Some(ping.headers),
+ body_has_info_sections: self.0.include_info_sections,
+ ping_name: self.0.name.to_string(),
+ };
+
+ glean.upload_manager.enqueue_ping(glean, ping);
return true;
}
diff --git a/glean-core/src/metrics/quantity.rs b/glean-core/src/metrics/quantity.rs
index c59d3a4a21..92216625d6 100644
--- a/glean-core/src/metrics/quantity.rs
+++ b/glean-core/src/metrics/quantity.rs
@@ -98,6 +98,15 @@ impl QuantityMetric {
/// Gets the currently stored value as an integer.
///
/// This doesn't clear the stored value.
+ ///
+ /// # Arguments
+ ///
+ /// * `ping_name` - the optional name of the ping to retrieve the metric
+ /// for. Defaults to the first value in `send_in_pings`.
+ ///
+ /// # Returns
+ ///
+ /// The stored value or `None` if nothing stored.
pub fn test_get_value(&self, ping_name: Option) -> Option {
crate::block_on_dispatcher();
crate::core::with_glean(|glean| self.get_value(glean, ping_name.as_deref()))
@@ -110,8 +119,6 @@ impl QuantityMetric {
/// # Arguments
///
/// * `error` - The type of error
- /// * `ping_name` - represents the optional name of the ping to retrieve the
- /// metric for. Defaults to the first value in `send_in_pings`.
///
/// # Returns
///
diff --git a/glean-core/src/metrics/rate.rs b/glean-core/src/metrics/rate.rs
index ba7f085b55..843d35002e 100644
--- a/glean-core/src/metrics/rate.rs
+++ b/glean-core/src/metrics/rate.rs
@@ -141,6 +141,15 @@ impl RateMetric {
/// Gets the currently stored value as a pair of integers.
///
/// This doesn't clear the stored value.
+ ///
+ /// # Arguments
+ ///
+ /// * `ping_name` - the optional name of the ping to retrieve the metric
+ /// for. Defaults to the first value in `send_in_pings`.
+ ///
+ /// # Returns
+ ///
+ /// The stored value or `None` if nothing stored.
pub fn test_get_value(&self, ping_name: Option) -> Option {
crate::block_on_dispatcher();
crate::core::with_glean(|glean| self.get_value(glean, ping_name.as_deref()))
@@ -175,8 +184,6 @@ impl RateMetric {
/// # Arguments
///
/// * `error` - The type of error
- /// * `ping_name` - represents the optional name of the ping to retrieve the
- /// metric for. Defaults to the first value in `send_in_pings`.
///
/// # Returns
///
diff --git a/glean-core/src/metrics/string.rs b/glean-core/src/metrics/string.rs
index 5ed7b2c7f1..4aa30a8d7e 100644
--- a/glean-core/src/metrics/string.rs
+++ b/glean-core/src/metrics/string.rs
@@ -112,6 +112,15 @@ impl StringMetric {
/// Gets the currently stored value as a string.
///
/// This doesn't clear the stored value.
+ ///
+ /// # Arguments
+ ///
+ /// * `ping_name` - the optional name of the ping to retrieve the metric
+ /// for. Defaults to the first value in `send_in_pings`.
+ ///
+ /// # Returns
+ ///
+ /// The stored value or `None` if nothing stored.
pub fn test_get_value(&self, ping_name: Option) -> Option {
crate::block_on_dispatcher();
crate::core::with_glean(|glean| self.get_value(glean, ping_name.as_deref()))
@@ -124,8 +133,6 @@ impl StringMetric {
/// # Arguments
///
/// * `error` - The type of error
- /// * `ping_name` - represents the optional name of the ping to retrieve the
- /// metric for. Defaults to the first value in `send_in_pings`.
///
/// # Returns
///
diff --git a/glean-core/src/metrics/string_list.rs b/glean-core/src/metrics/string_list.rs
index 75b2df7f80..cd4e71b885 100644
--- a/glean-core/src/metrics/string_list.rs
+++ b/glean-core/src/metrics/string_list.rs
@@ -171,6 +171,15 @@ impl StringListMetric {
/// Gets the currently-stored values.
///
/// This doesn't clear the stored value.
+ ///
+ /// # Arguments
+ ///
+ /// * `ping_name` - the optional name of the ping to retrieve the metric
+ /// for. Defaults to the first value in `send_in_pings`.
+ ///
+ /// # Returns
+ ///
+ /// The stored value or `None` if nothing stored.
pub fn test_get_value(&self, ping_name: Option) -> Option> {
crate::block_on_dispatcher();
crate::core::with_glean(|glean| self.get_value(glean, ping_name.as_deref()))
@@ -183,8 +192,6 @@ impl StringListMetric {
/// # Arguments
///
/// * `error` - The type of error
- /// * `ping_name` - represents the optional name of the ping to retrieve the
- /// metric for. Defaults to the first value in `send_in_pings`.
///
/// # Returns
///
diff --git a/glean-core/src/metrics/text.rs b/glean-core/src/metrics/text.rs
index 06ad5c0d78..baa8e88d75 100644
--- a/glean-core/src/metrics/text.rs
+++ b/glean-core/src/metrics/text.rs
@@ -116,6 +116,15 @@ impl TextMetric {
/// Gets the currently stored value as a string.
///
/// This doesn't clear the stored value.
+ ///
+ /// # Arguments
+ ///
+ /// * `ping_name` - the optional name of the ping to retrieve the metric
+ /// for. Defaults to the first value in `send_in_pings`.
+ ///
+ /// # Returns
+ ///
+ /// The stored value or `None` if nothing stored.
pub fn test_get_value(&self, ping_name: Option) -> Option {
crate::block_on_dispatcher();
crate::core::with_glean(|glean| self.get_value(glean, ping_name.as_deref()))
@@ -128,8 +137,6 @@ impl TextMetric {
/// # Arguments
///
/// * `error` - The type of error
- /// * `ping_name` - represents the optional name of the ping to retrieve the
- /// metric for. Defaults to the first value in `send_in_pings`.
///
/// # Returns
///
diff --git a/glean-core/src/metrics/timespan.rs b/glean-core/src/metrics/timespan.rs
index b4d3bd5902..ee63fb52f8 100644
--- a/glean-core/src/metrics/timespan.rs
+++ b/glean-core/src/metrics/timespan.rs
@@ -253,6 +253,15 @@ impl TimespanMetric {
/// Gets the currently stored value as an integer.
///
/// This doesn't clear the stored value.
+ ///
+ /// # Arguments
+ ///
+ /// * `ping_name` - the optional name of the ping to retrieve the metric
+ /// for. Defaults to the first value in `send_in_pings`.
+ ///
+ /// # Returns
+ ///
+ /// The stored value or `None` if nothing stored.
pub fn test_get_value(&self, ping_name: Option) -> Option {
crate::block_on_dispatcher();
crate::core::with_glean(|glean| {
@@ -292,8 +301,6 @@ impl TimespanMetric {
/// # Arguments
///
/// * `error` - The type of error
- /// * `ping_name` - represents the optional name of the ping to retrieve the
- /// metric for. Defaults to the first value in `send_in_pings`.
///
/// # Returns
///
diff --git a/glean-core/src/metrics/timing_distribution.rs b/glean-core/src/metrics/timing_distribution.rs
index e339ef8882..076942b1d3 100644
--- a/glean-core/src/metrics/timing_distribution.rs
+++ b/glean-core/src/metrics/timing_distribution.rs
@@ -464,6 +464,15 @@ impl TimingDistributionMetric {
/// Gets the currently stored value as an integer.
///
/// This doesn't clear the stored value.
+ ///
+ /// # Arguments
+ ///
+ /// * `ping_name` - the optional name of the ping to retrieve the metric
+ /// for. Defaults to the first value in `send_in_pings`.
+ ///
+ /// # Returns
+ ///
+ /// The stored value or `None` if nothing stored.
pub fn test_get_value(&self, ping_name: Option) -> Option {
crate::block_on_dispatcher();
crate::core::with_glean(|glean| self.get_value(glean, ping_name.as_deref()))
@@ -476,8 +485,6 @@ impl TimingDistributionMetric {
/// # Arguments
///
/// * `error` - The type of error
- /// * `ping_name` - represents the optional name of the ping to retrieve the
- /// metric for. Defaults to the first value in `send_in_pings`.
///
/// # Returns
///
diff --git a/glean-core/src/metrics/url.rs b/glean-core/src/metrics/url.rs
index c9eb824a3e..48b3f9e7ae 100644
--- a/glean-core/src/metrics/url.rs
+++ b/glean-core/src/metrics/url.rs
@@ -131,6 +131,15 @@ impl UrlMetric {
/// Gets the currently stored value as a string.
///
/// This doesn't clear the stored value.
+ ///
+ /// # Arguments
+ ///
+ /// * `ping_name` - the optional name of the ping to retrieve the metric
+ /// for. Defaults to the first value in `send_in_pings`.
+ ///
+ /// # Returns
+ ///
+ /// The stored value or `None` if nothing stored.
pub fn test_get_value(&self, ping_name: Option) -> Option {
crate::block_on_dispatcher();
crate::core::with_glean(|glean| self.get_value(glean, ping_name.as_deref()))
@@ -143,8 +152,6 @@ impl UrlMetric {
/// # Arguments
///
/// * `error` - The type of error
- /// * `ping_name` - represents the optional name of the ping to retrieve the
- /// metric for. Defaults to the first value in `send_in_pings`.
///
/// # Returns
///
diff --git a/glean-core/src/metrics/uuid.rs b/glean-core/src/metrics/uuid.rs
index e78d15ad3b..77d0f82320 100644
--- a/glean-core/src/metrics/uuid.rs
+++ b/glean-core/src/metrics/uuid.rs
@@ -128,6 +128,15 @@ impl UuidMetric {
/// Gets the currently stored value as a string.
///
/// This doesn't clear the stored value.
+ ///
+ /// # Arguments
+ ///
+ /// * `ping_name` - the optional name of the ping to retrieve the metric
+ /// for. Defaults to the first value in `send_in_pings`.
+ ///
+ /// # Returns
+ ///
+ /// The stored value or `None` if nothing stored.
pub fn test_get_value(&self, ping_name: Option) -> Option {
crate::block_on_dispatcher();
crate::core::with_glean(|glean| {
@@ -143,8 +152,6 @@ impl UuidMetric {
/// # Arguments
///
/// * `error` - The type of error
- /// * `ping_name` - represents the optional name of the ping to retrieve the
- /// metric for. Defaults to the first value in `send_in_pings`.
///
/// # Returns
///
diff --git a/glean-core/src/ping/mod.rs b/glean-core/src/ping/mod.rs
index c22a890aa2..d1a67ae360 100644
--- a/glean-core/src/ping/mod.rs
+++ b/glean-core/src/ping/mod.rs
@@ -14,7 +14,7 @@ use serde_json::{json, Value as JsonValue};
use crate::common_metric_data::{CommonMetricData, Lifetime};
use crate::metrics::{CounterMetric, DatetimeMetric, Metric, MetricType, PingType, TimeUnit};
use crate::storage::{StorageManager, INTERNAL_STORAGE};
-use crate::upload::HeaderMap;
+use crate::upload::{HeaderMap, PingMetadata};
use crate::util::{get_iso_time_string, local_now_with_offset};
use crate::{Glean, Result, DELETION_REQUEST_PINGS_DIRECTORY, PENDING_PINGS_DIRECTORY};
@@ -30,6 +30,8 @@ pub struct Ping<'a> {
pub content: JsonValue,
/// The headers to upload with the payload.
pub headers: HeaderMap,
+ /// Whether the content contains {client|ping}_info sections.
+ pub includes_info_sections: bool,
}
/// Collect a ping's data, assemble it into its full payload and store it on disk.
@@ -237,9 +239,9 @@ impl PingMaker {
.snapshot_as_json(glean, ping.name(), true);
// Due to the way the experimentation identifier could link datasets that are intentionally unlinked,
- // it will not be included in pings that specifically exclude the Glean client-id and those pings that
- // should not be sent if empty.
- if (!ping.include_client_id() || !ping.send_if_empty())
+ // it will not be included in pings that specifically exclude the Glean client-id, those pings that
+ // should not be sent if empty, or pings that exclude the {client|ping}_info sections wholesale.
+ if (!ping.include_client_id() || !ping.send_if_empty() || !ping.include_info_sections())
&& glean.test_get_experimentation_id().is_some()
&& metrics_data.is_some()
{
@@ -285,13 +287,18 @@ impl PingMaker {
TimeUnit::Minute
};
- let ping_info = self.get_ping_info(glean, ping.name(), reason, precision);
- let client_info = self.get_client_info(glean, ping.include_client_id());
+ let mut json = if ping.include_info_sections() {
+ let ping_info = self.get_ping_info(glean, ping.name(), reason, precision);
+ let client_info = self.get_client_info(glean, ping.include_client_id());
+
+ json!({
+ "ping_info": ping_info,
+ "client_info": client_info
+ })
+ } else {
+ json!({})
+ };
- let mut json = json!({
- "ping_info": ping_info,
- "client_info": client_info
- });
let json_obj = json.as_object_mut()?;
if let Some(metrics_data) = metrics_data {
json_obj.insert("metrics".to_string(), metrics_data);
@@ -306,6 +313,7 @@ impl PingMaker {
doc_id,
url_path,
headers: self.get_headers(glean),
+ includes_info_sections: ping.include_info_sections(),
})
}
@@ -355,11 +363,17 @@ impl PingMaker {
file.write_all(ping.url_path.as_bytes())?;
file.write_all(b"\n")?;
file.write_all(::serde_json::to_string(&ping.content)?.as_bytes())?;
- if !ping.headers.is_empty() {
- file.write_all(b"\n{\"headers\":")?;
- file.write_all(::serde_json::to_string(&ping.headers)?.as_bytes())?;
- file.write_all(b"}")?;
- }
+ file.write_all(b"\n")?;
+ let metadata = PingMetadata {
+ // We don't actually need to clone the headers except to match PingMetadata's ownership.
+ // But since we're going to write a file to disk in a sec,
+ // and HeaderMaps tend to have only like two things in them, tops,
+ // the cost is bearable.
+ headers: Some(ping.headers.clone()),
+ body_has_info_sections: Some(ping.includes_info_sections),
+ ping_name: Some(ping.name.to_string()),
+ };
+ file.write_all(::serde_json::to_string(&metadata)?.as_bytes())?;
}
if let Err(e) = std::fs::rename(&temp_ping_path, &ping_path) {
diff --git a/glean-core/src/traits/mod.rs b/glean-core/src/traits/mod.rs
index c4bcf7cdd6..4115609fdd 100644
--- a/glean-core/src/traits/mod.rs
+++ b/glean-core/src/traits/mod.rs
@@ -7,6 +7,10 @@
//! Individual metric types implement this trait to expose the specific metrics API.
//! It can be used by wrapping implementations to guarantee API conformance.
+/// Re-export for use in generated code.
+#[doc(hidden)]
+pub extern crate serde as __serde;
+
mod boolean;
mod counter;
mod custom_distribution;
@@ -15,6 +19,7 @@ mod event;
mod labeled;
mod memory_distribution;
mod numerator;
+mod object;
mod ping;
mod quantity;
mod rate;
@@ -37,6 +42,7 @@ pub use self::event::NoExtraKeys;
pub use self::labeled::Labeled;
pub use self::memory_distribution::MemoryDistribution;
pub use self::numerator::Numerator;
+pub use self::object::{ObjectError, ObjectSerialize};
pub use self::ping::Ping;
pub use self::quantity::Quantity;
pub use self::rate::Rate;
diff --git a/glean-core/src/traits/object.rs b/glean-core/src/traits/object.rs
new file mode 100644
index 0000000000..c579efeac7
--- /dev/null
+++ b/glean-core/src/traits/object.rs
@@ -0,0 +1,53 @@
+// This Source Code Form is subject to the terms of the Mozilla Public
+// License, v. 2.0. If a copy of the MPL was not distributed with this
+// file, You can obtain one at https://mozilla.org/MPL/2.0/.
+
+use std::fmt::Display;
+
+use serde::{Deserialize, Serialize};
+use serde_json::Value as JsonValue;
+
+/// This type represents all possible errors that can occur when serializing or deserializing an object from/to JSON.
+#[derive(Debug)]
+pub struct ObjectError(serde_json::Error);
+
+impl Display for ObjectError {
+ fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
+ Display::fmt(&self.0, f)
+ }
+}
+
+impl std::error::Error for ObjectError {}
+
+/// An object that can be serialized into JSON.
+///
+/// Objects are defined by their structure in the metrics definition.
+///
+/// This is essentially a wrapper around serde's `Serialize`/`Deserialize`,
+/// but in a way we can name it for our JSON (de)serialization.
+pub trait ObjectSerialize {
+ /// Deserialize the object from its JSON representation.
+ ///
+ /// Returns an error if deserialization fails.
+ /// This should not happen for glean_parser-generated and later serialized objects.
+ fn from_str(obj: &str) -> Result
+ where
+ Self: Sized;
+
+ /// Serialize this object into a JSON string.
+ fn into_serialized_object(self) -> Result;
+}
+
+impl ObjectSerialize for V
+where
+ V: Serialize,
+ V: for<'de> Deserialize<'de>,
+{
+ fn from_str(obj: &str) -> Result {
+ serde_json::from_str(obj).map_err(ObjectError)
+ }
+
+ fn into_serialized_object(self) -> Result {
+ serde_json::to_value(self).map_err(ObjectError)
+ }
+}
diff --git a/glean-core/src/upload/directory.rs b/glean-core/src/upload/directory.rs
index a78bbf0bdb..706550fe6c 100644
--- a/glean-core/src/upload/directory.rs
+++ b/glean-core/src/upload/directory.rs
@@ -9,15 +9,28 @@ use std::fs::{self, File};
use std::io::{BufRead, BufReader};
use std::path::{Path, PathBuf};
-use serde::Deserialize;
+use serde::{Deserialize, Serialize};
use uuid::Uuid;
use super::request::HeaderMap;
use crate::{DELETION_REQUEST_PINGS_DIRECTORY, PENDING_PINGS_DIRECTORY};
/// A representation of the data extracted from a ping file,
-/// this will contain the document_id, path, JSON encoded body of a ping and the persisted headers.
-pub type PingPayload = (String, String, String, Option);
+#[derive(Clone, Debug, Default)]
+pub struct PingPayload {
+ /// The ping's doc_id.
+ pub document_id: String,
+ /// The path to upload the ping to.
+ pub upload_path: String,
+ /// The ping body as JSON-encoded string.
+ pub json_body: String,
+ /// HTTP headers to include in the upload request.
+ pub headers: Option,
+ /// Whether the ping body contains {client|ping}_info
+ pub body_has_info_sections: bool,
+ /// The ping's name. (Also likely in the upload_path.)
+ pub ping_name: String,
+}
/// A struct to hold the result of scanning all pings directories.
#[derive(Clone, Debug, Default)]
@@ -62,20 +75,28 @@ fn get_file_name_as_str(path: &Path) -> Option<&str> {
}
}
+/// A ping's metadata, as (optionally) represented on disk.
+///
+/// Anything that isn't the upload path or the ping body.
+#[derive(Default, Deserialize, Serialize)]
+pub struct PingMetadata {
+ /// HTTP headers to include when uploading the ping.
+ pub headers: Option,
+ /// Whether the body has {client|ping}_info sections.
+ pub body_has_info_sections: Option,
+ /// The name of the ping.
+ pub ping_name: Option,
+}
+
/// Processes a ping's metadata.
///
/// The metadata is an optional third line in the ping file,
/// currently it contains only additonal headers to be added to each ping request.
/// Therefore, we will process the contents of this line
/// and return a HeaderMap of the persisted headers.
-fn process_metadata(path: &str, metadata: &str) -> Option {
- #[derive(Deserialize)]
- struct PingMetadata {
- pub headers: HeaderMap,
- }
-
+fn process_metadata(path: &str, metadata: &str) -> Option {
if let Ok(metadata) = serde_json::from_str::(metadata) {
- return Some(metadata.headers);
+ return Some(metadata);
} else {
log::warn!("Error while parsing ping metadata: {}", path);
}
@@ -171,8 +192,23 @@ impl PingDirectoryManager {
if let (Some(Ok(path)), Some(Ok(body)), Ok(metadata)) =
(lines.next(), lines.next(), lines.next().transpose())
{
- let headers = metadata.and_then(|m| process_metadata(&path, &m));
- return Some((document_id.into(), path, body, headers));
+ let PingMetadata {
+ headers,
+ body_has_info_sections,
+ ping_name,
+ } = metadata
+ .and_then(|m| process_metadata(&path, &m))
+ .unwrap_or_default();
+ let ping_name =
+ ping_name.unwrap_or_else(|| path.split('/').nth(3).unwrap_or("").into());
+ return Some(PingPayload {
+ document_id: document_id.into(),
+ upload_path: path,
+ json_body: body,
+ headers,
+ body_has_info_sections: body_has_info_sections.unwrap_or(true),
+ ping_name,
+ });
} else {
log::warn!(
"Error processing ping file: {}. Ping file is not formatted as expected.",
@@ -303,7 +339,7 @@ mod test {
let (mut glean, dir) = new_glean(None);
// Register a ping for testing
- let ping_type = PingType::new("test", true, true, true, vec![]);
+ let ping_type = PingType::new("test", true, true, true, true, vec![]);
glean.register_ping_type(&ping_type);
// Submit the ping to populate the pending_pings directory
@@ -320,7 +356,8 @@ mod test {
// Verify request was returned for the "test" ping
let ping = &data.pending_pings[0].1;
- let request_ping_type = ping.1.split('/').nth(3).unwrap();
+ let request_ping_type = ping.upload_path.split('/').nth(3).unwrap();
+ assert_eq!(request_ping_type, ping.ping_name);
assert_eq!(request_ping_type, "test");
}
@@ -329,7 +366,7 @@ mod test {
let (mut glean, dir) = new_glean(None);
// Register a ping for testing
- let ping_type = PingType::new("test", true, true, true, vec![]);
+ let ping_type = PingType::new("test", true, true, true, true, vec![]);
glean.register_ping_type(&ping_type);
// Submit the ping to populate the pending_pings directory
@@ -352,7 +389,8 @@ mod test {
// Verify request was returned for the "test" ping
let ping = &data.pending_pings[0].1;
- let request_ping_type = ping.1.split('/').nth(3).unwrap();
+ let request_ping_type = ping.upload_path.split('/').nth(3).unwrap();
+ assert_eq!(request_ping_type, ping.ping_name);
assert_eq!(request_ping_type, "test");
// Verify that file was indeed deleted
@@ -364,7 +402,7 @@ mod test {
let (mut glean, dir) = new_glean(None);
// Register a ping for testing
- let ping_type = PingType::new("test", true, true, true, vec![]);
+ let ping_type = PingType::new("test", true, true, true, true, vec![]);
glean.register_ping_type(&ping_type);
// Submit the ping to populate the pending_pings directory
@@ -387,7 +425,8 @@ mod test {
// Verify request was returned for the "test" ping
let ping = &data.pending_pings[0].1;
- let request_ping_type = ping.1.split('/').nth(3).unwrap();
+ let request_ping_type = ping.upload_path.split('/').nth(3).unwrap();
+ assert_eq!(request_ping_type, ping.ping_name);
assert_eq!(request_ping_type, "test");
// Verify that file was indeed deleted
@@ -414,7 +453,8 @@ mod test {
// Verify request was returned for the "deletion-request" ping
let ping = &data.deletion_request_pings[0].1;
- let request_ping_type = ping.1.split('/').nth(3).unwrap();
+ let request_ping_type = ping.upload_path.split('/').nth(3).unwrap();
+ assert_eq!(request_ping_type, ping.ping_name);
assert_eq!(request_ping_type, "deletion-request");
}
}
diff --git a/glean-core/src/upload/mod.rs b/glean-core/src/upload/mod.rs
index d764dcd29e..e51a9d9508 100644
--- a/glean-core/src/upload/mod.rs
+++ b/glean-core/src/upload/mod.rs
@@ -30,6 +30,7 @@ use directory::{PingDirectoryManager, PingPayloadsByDirectory};
use policy::Policy;
use request::create_date_header_value;
+pub use directory::{PingMetadata, PingPayload};
pub use request::{HeaderMap, PingRequest};
pub use result::{UploadResult, UploadTaskAction};
@@ -322,21 +323,24 @@ impl PingUploadManager {
///
/// Returns the `PingRequest` or `None` if unable to build,
/// in which case it will delete the ping file and record an error.
- fn build_ping_request(
- &self,
- glean: &Glean,
- document_id: &str,
- path: &str,
- body: &str,
- headers: Option,
- ) -> Option {
+ fn build_ping_request(&self, glean: &Glean, ping: PingPayload) -> Option {
+ let PingPayload {
+ document_id,
+ upload_path: path,
+ json_body: body,
+ headers,
+ body_has_info_sections,
+ ping_name,
+ } = ping;
let mut request = PingRequest::builder(
&self.language_binding_name,
self.policy.max_ping_body_size(),
)
- .document_id(document_id)
+ .document_id(&document_id)
.path(path)
- .body(body);
+ .body(body)
+ .body_has_info_sections(body_has_info_sections)
+ .ping_name(ping_name);
if let Some(headers) = headers {
request = request.headers(headers);
@@ -346,7 +350,7 @@ impl PingUploadManager {
Ok(request) => Some(request),
Err(e) => {
log::warn!("Error trying to build ping request: {}", e);
- self.directory_manager.delete_file(document_id);
+ self.directory_manager.delete_file(&document_id);
// Record the error.
// Currently the only possible error is PingBodyOverflow.
@@ -362,23 +366,21 @@ impl PingUploadManager {
}
/// Enqueue a ping for upload.
- pub fn enqueue_ping(
- &self,
- glean: &Glean,
- document_id: &str,
- path: &str,
- body: &str,
- headers: Option,
- ) {
+ pub fn enqueue_ping(&self, glean: &Glean, ping: PingPayload) {
let mut queue = self
.queue
.write()
.expect("Can't write to pending pings queue.");
+ let PingPayload {
+ ref document_id,
+ upload_path: ref path,
+ ..
+ } = ping;
// Checks if a ping with this `document_id` is already enqueued.
if queue
.iter()
- .any(|request| request.document_id == document_id)
+ .any(|request| request.document_id.as_str() == document_id)
{
log::warn!(
"Attempted to enqueue a duplicate ping {} at {}.",
@@ -404,7 +406,7 @@ impl PingUploadManager {
}
log::trace!("Enqueuing ping {} at {}", document_id, path);
- if let Some(request) = self.build_ping_request(glean, document_id, path, body, headers) {
+ if let Some(request) = self.build_ping_request(glean, ping) {
queue.push_back(request)
}
}
@@ -455,7 +457,7 @@ impl PingUploadManager {
// Thus, we reverse the order of the pending pings vector,
// so that we iterate in descending order (newest -> oldest).
cached_pings.pending_pings.reverse();
- cached_pings.pending_pings.retain(|(file_size, (document_id, _, _, _))| {
+ cached_pings.pending_pings.retain(|(file_size, PingPayload {document_id, ..})| {
pending_pings_count += 1;
pending_pings_directory_size += file_size;
@@ -493,14 +495,14 @@ impl PingUploadManager {
// Enqueue the remaining pending pings and
// enqueue all deletion-request pings.
- let deletion_request_pings = cached_pings.deletion_request_pings.drain(..);
- for (_, (document_id, path, body, headers)) in deletion_request_pings {
- self.enqueue_ping(glean, &document_id, &path, &body, headers);
- }
- let pending_pings = cached_pings.pending_pings.drain(..);
- for (_, (document_id, path, body, headers)) in pending_pings {
- self.enqueue_ping(glean, &document_id, &path, &body, headers);
- }
+ cached_pings
+ .deletion_request_pings
+ .drain(..)
+ .for_each(|(_, ping)| self.enqueue_ping(glean, ping));
+ cached_pings
+ .pending_pings
+ .drain(..)
+ .for_each(|(_, ping)| self.enqueue_ping(glean, ping));
}
}
@@ -532,10 +534,8 @@ impl PingUploadManager {
/// * `glean` - The Glean object holding the database.
/// * `document_id` - The UUID of the ping in question.
pub fn enqueue_ping_from_file(&self, glean: &Glean, document_id: &str) {
- if let Some((doc_id, path, body, headers)) =
- self.directory_manager.process_file(document_id)
- {
- self.enqueue_ping(glean, &doc_id, &path, &body, headers)
+ if let Some(ping) = self.directory_manager.process_file(document_id) {
+ self.enqueue_ping(glean, ping);
}
}
@@ -883,7 +883,17 @@ mod test {
let upload_manager = PingUploadManager::no_policy(dir.path());
// Enqueue a ping
- upload_manager.enqueue_ping(&glean, &Uuid::new_v4().to_string(), PATH, "", None);
+ upload_manager.enqueue_ping(
+ &glean,
+ PingPayload {
+ document_id: Uuid::new_v4().to_string(),
+ upload_path: PATH.into(),
+ json_body: "".into(),
+ headers: None,
+ body_has_info_sections: true,
+ ping_name: "ping-name".into(),
+ },
+ );
// Try and get the next request.
// Verify request was returned
@@ -900,7 +910,17 @@ mod test {
// Enqueue a ping multiple times
let n = 10;
for _ in 0..n {
- upload_manager.enqueue_ping(&glean, &Uuid::new_v4().to_string(), PATH, "", None);
+ upload_manager.enqueue_ping(
+ &glean,
+ PingPayload {
+ document_id: Uuid::new_v4().to_string(),
+ upload_path: PATH.into(),
+ json_body: "".into(),
+ headers: None,
+ body_has_info_sections: true,
+ ping_name: "ping-name".into(),
+ },
+ );
}
// Verify a request is returned for each submitted ping
@@ -928,7 +948,17 @@ mod test {
// Enqueue the max number of pings allowed per uploading window
for _ in 0..max_pings_per_interval {
- upload_manager.enqueue_ping(&glean, &Uuid::new_v4().to_string(), PATH, "", None);
+ upload_manager.enqueue_ping(
+ &glean,
+ PingPayload {
+ document_id: Uuid::new_v4().to_string(),
+ upload_path: PATH.into(),
+ json_body: "".into(),
+ headers: None,
+ body_has_info_sections: true,
+ ping_name: "ping-name".into(),
+ },
+ );
}
// Verify a request is returned for each submitted ping
@@ -938,7 +968,17 @@ mod test {
}
// Enqueue just one more ping
- upload_manager.enqueue_ping(&glean, &Uuid::new_v4().to_string(), PATH, "", None);
+ upload_manager.enqueue_ping(
+ &glean,
+ PingPayload {
+ document_id: Uuid::new_v4().to_string(),
+ upload_path: PATH.into(),
+ json_body: "".into(),
+ headers: None,
+ body_has_info_sections: true,
+ ping_name: "ping-name".into(),
+ },
+ );
// Verify that we are indeed told to wait because we are at capacity
match upload_manager.get_upload_task(&glean, false) {
@@ -961,7 +1001,17 @@ mod test {
// Enqueue a ping multiple times
for _ in 0..10 {
- upload_manager.enqueue_ping(&glean, &Uuid::new_v4().to_string(), PATH, "", None);
+ upload_manager.enqueue_ping(
+ &glean,
+ PingPayload {
+ document_id: Uuid::new_v4().to_string(),
+ upload_path: PATH.into(),
+ json_body: "".into(),
+ headers: None,
+ body_has_info_sections: true,
+ ping_name: "ping-name".into(),
+ },
+ );
}
// Clear the queue
@@ -979,7 +1029,14 @@ mod test {
let (mut glean, _t) = new_glean(None);
// Register a ping for testing
- let ping_type = PingType::new("test", true, /* send_if_empty */ true, true, vec![]);
+ let ping_type = PingType::new(
+ "test",
+ true,
+ /* send_if_empty */ true,
+ true,
+ true,
+ vec![],
+ );
glean.register_ping_type(&ping_type);
// Submit the ping multiple times
@@ -1011,7 +1068,14 @@ mod test {
let (mut glean, dir) = new_glean(None);
// Register a ping for testing
- let ping_type = PingType::new("test", true, /* send_if_empty */ true, true, vec![]);
+ let ping_type = PingType::new(
+ "test",
+ true,
+ /* send_if_empty */ true,
+ true,
+ true,
+ vec![],
+ );
glean.register_ping_type(&ping_type);
// Submit the ping multiple times
@@ -1041,7 +1105,14 @@ mod test {
let (mut glean, dir) = new_glean(None);
// Register a ping for testing
- let ping_type = PingType::new("test", true, /* send_if_empty */ true, true, vec![]);
+ let ping_type = PingType::new(
+ "test",
+ true,
+ /* send_if_empty */ true,
+ true,
+ true,
+ vec![],
+ );
glean.register_ping_type(&ping_type);
// Submit a ping
@@ -1071,7 +1142,14 @@ mod test {
let (mut glean, dir) = new_glean(None);
// Register a ping for testing
- let ping_type = PingType::new("test", true, /* send_if_empty */ true, true, vec![]);
+ let ping_type = PingType::new(
+ "test",
+ true,
+ /* send_if_empty */ true,
+ true,
+ true,
+ vec![],
+ );
glean.register_ping_type(&ping_type);
// Submit a ping
@@ -1101,7 +1179,14 @@ mod test {
let (mut glean, _t) = new_glean(None);
// Register a ping for testing
- let ping_type = PingType::new("test", true, /* send_if_empty */ true, true, vec![]);
+ let ping_type = PingType::new(
+ "test",
+ true,
+ /* send_if_empty */ true,
+ true,
+ true,
+ vec![],
+ );
glean.register_ping_type(&ping_type);
// Submit a ping
@@ -1133,7 +1218,14 @@ mod test {
let (mut glean, dir) = new_glean(None);
// Register a ping for testing
- let ping_type = PingType::new("test", true, /* send_if_empty */ true, true, vec![]);
+ let ping_type = PingType::new(
+ "test",
+ true,
+ /* send_if_empty */ true,
+ true,
+ true,
+ vec![],
+ );
glean.register_ping_type(&ping_type);
// Submit a ping
@@ -1174,7 +1266,17 @@ mod test {
let path2 = format!("/submit/app_id/test-ping/1/{}", doc2);
// Enqueue a ping
- upload_manager.enqueue_ping(&glean, &doc1, &path1, "", None);
+ upload_manager.enqueue_ping(
+ &glean,
+ PingPayload {
+ document_id: doc1.clone(),
+ upload_path: path1,
+ json_body: "".into(),
+ headers: None,
+ body_has_info_sections: true,
+ ping_name: "test-ping".into(),
+ },
+ );
// Try and get the first request.
let req = match upload_manager.get_upload_task(&glean, false) {
@@ -1184,7 +1286,17 @@ mod test {
assert_eq!(doc1, req.document_id);
// Schedule the next one while the first one is "in progress"
- upload_manager.enqueue_ping(&glean, &doc2, &path2, "", None);
+ upload_manager.enqueue_ping(
+ &glean,
+ PingPayload {
+ document_id: doc2.clone(),
+ upload_path: path2,
+ json_body: "".into(),
+ headers: None,
+ body_has_info_sections: true,
+ ping_name: "test-ping".into(),
+ },
+ );
// Mark as processed
upload_manager.process_ping_upload_response(
@@ -1221,7 +1333,14 @@ mod test {
glean.set_debug_view_tag("valid-tag");
// Register a ping for testing
- let ping_type = PingType::new("test", true, /* send_if_empty */ true, true, vec![]);
+ let ping_type = PingType::new(
+ "test",
+ true,
+ /* send_if_empty */ true,
+ true,
+ true,
+ vec![],
+ );
glean.register_ping_type(&ping_type);
// Submit a ping
@@ -1248,8 +1367,28 @@ mod test {
let path = format!("/submit/app_id/test-ping/1/{}", doc_id);
// Try to enqueue a ping with the same doc_id twice
- upload_manager.enqueue_ping(&glean, &doc_id, &path, "", None);
- upload_manager.enqueue_ping(&glean, &doc_id, &path, "", None);
+ upload_manager.enqueue_ping(
+ &glean,
+ PingPayload {
+ document_id: doc_id.clone(),
+ upload_path: path.clone(),
+ json_body: "".into(),
+ headers: None,
+ body_has_info_sections: true,
+ ping_name: "test-ping".into(),
+ },
+ );
+ upload_manager.enqueue_ping(
+ &glean,
+ PingPayload {
+ document_id: doc_id,
+ upload_path: path,
+ json_body: "".into(),
+ headers: None,
+ body_has_info_sections: true,
+ ping_name: "test-ping".into(),
+ },
+ );
// Get a task once
let task = upload_manager.get_upload_task(&glean, false);
@@ -1267,7 +1406,14 @@ mod test {
let (mut glean, dir) = new_glean(None);
// Register a ping for testing
- let ping_type = PingType::new("test", true, /* send_if_empty */ true, true, vec![]);
+ let ping_type = PingType::new(
+ "test",
+ true,
+ /* send_if_empty */ true,
+ true,
+ true,
+ vec![],
+ );
glean.register_ping_type(&ping_type);
// Submit the ping multiple times
@@ -1317,7 +1463,14 @@ mod test {
let (mut glean, dir) = new_glean(None);
// Register a ping for testing
- let ping_type = PingType::new("test", true, /* send_if_empty */ true, true, vec![]);
+ let ping_type = PingType::new(
+ "test",
+ true,
+ /* send_if_empty */ true,
+ true,
+ true,
+ vec![],
+ );
glean.register_ping_type(&ping_type);
// Submit the ping multiple times
@@ -1331,7 +1484,10 @@ mod test {
// The pending pings array is sorted by date in ascending order,
// the newest element is the last one.
let (_, newest_ping) = &pending_pings.last().unwrap();
- let (newest_ping_id, _, _, _) = &newest_ping;
+ let PingPayload {
+ document_id: newest_ping_id,
+ ..
+ } = &newest_ping;
// Create a new upload manager pointing to the same data_path as the glean instance.
let mut upload_manager = PingUploadManager::no_policy(dir.path());
@@ -1385,7 +1541,14 @@ mod test {
let (mut glean, dir) = new_glean(None);
// Register a ping for testing
- let ping_type = PingType::new("test", true, /* send_if_empty */ true, true, vec![]);
+ let ping_type = PingType::new(
+ "test",
+ true,
+ /* send_if_empty */ true,
+ true,
+ true,
+ vec![],
+ );
glean.register_ping_type(&ping_type);
// How many pings we allow at maximum
@@ -1406,7 +1569,7 @@ mod test {
.iter()
.rev()
.take(count_quota)
- .map(|(_, ping)| ping.0.clone())
+ .map(|(_, ping)| ping.document_id.clone())
.collect::>();
// Create a new upload manager pointing to the same data_path as the glean instance.
@@ -1457,7 +1620,14 @@ mod test {
let (mut glean, dir) = new_glean(None);
// Register a ping for testing
- let ping_type = PingType::new("test", true, /* send_if_empty */ true, true, vec![]);
+ let ping_type = PingType::new(
+ "test",
+ true,
+ /* send_if_empty */ true,
+ true,
+ true,
+ vec![],
+ );
glean.register_ping_type(&ping_type);
let expected_number_of_pings = 3;
@@ -1477,7 +1647,7 @@ mod test {
.iter()
.rev()
.take(expected_number_of_pings)
- .map(|(_, ping)| ping.0.clone())
+ .map(|(_, ping)| ping.document_id.clone())
.collect::>();
// Create a new upload manager pointing to the same data_path as the glean instance.
@@ -1531,7 +1701,14 @@ mod test {
let (mut glean, dir) = new_glean(None);
// Register a ping for testing
- let ping_type = PingType::new("test", true, /* send_if_empty */ true, true, vec![]);
+ let ping_type = PingType::new(
+ "test",
+ true,
+ /* send_if_empty */ true,
+ true,
+ true,
+ vec![],
+ );
glean.register_ping_type(&ping_type);
let expected_number_of_pings = 2;
@@ -1551,7 +1728,7 @@ mod test {
.iter()
.rev()
.take(expected_number_of_pings)
- .map(|(_, ping)| ping.0.clone())
+ .map(|(_, ping)| ping.document_id.clone())
.collect::>();
// Create a new upload manager pointing to the same data_path as the glean instance.
@@ -1622,8 +1799,28 @@ mod test {
upload_manager.set_rate_limiter(secs_per_interval, max_pings_per_interval);
// Enqueue two pings
- upload_manager.enqueue_ping(&glean, &Uuid::new_v4().to_string(), PATH, "", None);
- upload_manager.enqueue_ping(&glean, &Uuid::new_v4().to_string(), PATH, "", None);
+ upload_manager.enqueue_ping(
+ &glean,
+ PingPayload {
+ document_id: Uuid::new_v4().to_string(),
+ upload_path: PATH.into(),
+ json_body: "".into(),
+ headers: None,
+ body_has_info_sections: true,
+ ping_name: "ping-name".into(),
+ },
+ );
+ upload_manager.enqueue_ping(
+ &glean,
+ PingPayload {
+ document_id: Uuid::new_v4().to_string(),
+ upload_path: PATH.into(),
+ json_body: "".into(),
+ headers: None,
+ body_has_info_sections: true,
+ ping_name: "ping-name".into(),
+ },
+ );
// Get the first ping, it should be returned normally.
match upload_manager.get_upload_task(&glean, false) {
@@ -1679,12 +1876,28 @@ mod test {
let upload_manager = PingUploadManager::no_policy(dir.path());
// Enqueue a ping and start processing it
- let identifier = &Uuid::new_v4().to_string();
- upload_manager.enqueue_ping(&glean, identifier, PATH, "", None);
+ let identifier = &Uuid::new_v4();
+ let ping = PingPayload {
+ document_id: identifier.to_string(),
+ upload_path: PATH.into(),
+ json_body: "".into(),
+ headers: None,
+ body_has_info_sections: true,
+ ping_name: "ping-name".into(),
+ };
+ upload_manager.enqueue_ping(&glean, ping);
assert!(upload_manager.get_upload_task(&glean, false).is_upload());
// Attempt to re-enqueue the same ping
- upload_manager.enqueue_ping(&glean, identifier, PATH, "", None);
+ let ping = PingPayload {
+ document_id: identifier.to_string(),
+ upload_path: PATH.into(),
+ json_body: "".into(),
+ headers: None,
+ body_has_info_sections: true,
+ ping_name: "ping-name".into(),
+ };
+ upload_manager.enqueue_ping(&glean, ping);
// No new pings should have been enqueued so the upload task is Done.
assert_eq!(
@@ -1695,7 +1908,7 @@ mod test {
// Process the upload response
upload_manager.process_ping_upload_response(
&glean,
- identifier,
+ &identifier.to_string(),
UploadResult::http_status(200),
);
}
diff --git a/glean-core/src/upload/request.rs b/glean-core/src/upload/request.rs
index 0fd5ec5713..b4ac6eba97 100644
--- a/glean-core/src/upload/request.rs
+++ b/glean-core/src/upload/request.rs
@@ -62,6 +62,8 @@ pub struct Builder {
body: Option>,
headers: HeaderMap,
body_max_size: usize,
+ body_has_info_sections: Option,
+ ping_name: Option,
}
impl Builder {
@@ -87,6 +89,8 @@ impl Builder {
body: None,
headers,
body_max_size,
+ body_has_info_sections: None,
+ ping_name: None,
}
}
@@ -138,6 +142,18 @@ impl Builder {
self
}
+ /// Sets whether the request body has {client|ping}_info sections.
+ pub fn body_has_info_sections(mut self, body_has_info_sections: bool) -> Self {
+ self.body_has_info_sections = Some(body_has_info_sections);
+ self
+ }
+
+ /// Sets the ping's name aka doctype.
+ pub fn ping_name>(mut self, ping_name: S) -> Self {
+ self.ping_name = Some(ping_name.into());
+ self
+ }
+
/// Sets a header for this request.
pub fn header>(mut self, key: S, value: S) -> Self {
self.headers.insert(key.into(), value.into());
@@ -174,6 +190,12 @@ impl Builder {
.expect("path must be set before attempting to build PingRequest"),
body,
headers: self.headers,
+ body_has_info_sections: self.body_has_info_sections.expect(
+ "body_has_info_sections must be set before attempting to build PingRequest",
+ ),
+ ping_name: self
+ .ping_name
+ .expect("ping_name must be set before attempting to build PingRequest"),
})
}
}
@@ -192,6 +214,10 @@ pub struct PingRequest {
pub body: Vec,
/// A map with all the headers to be sent with the request.
pub headers: HeaderMap,
+ /// Whether the body has {client|ping}_info sections.
+ pub body_has_info_sections: bool,
+ /// The ping's name. Likely also somewhere in `path`.
+ pub ping_name: String,
}
impl PingRequest {
@@ -208,12 +234,7 @@ impl PingRequest {
/// Verifies if current request is for a deletion-request ping.
pub fn is_deletion_request(&self) -> bool {
- // The path format should be `/submit///`
- self.path
- .split('/')
- .nth(3)
- .map(|url| url == "deletion-request")
- .unwrap_or(false)
+ self.ping_name == "deletion-request"
}
/// Decompresses and pretty-format the ping payload
@@ -257,11 +278,15 @@ mod test {
.document_id("woop")
.path("/random/path/doesnt/matter")
.body("{}")
+ .body_has_info_sections(false)
+ .ping_name("whatevs")
.build()
.unwrap();
assert_eq!(request.document_id, "woop");
assert_eq!(request.path, "/random/path/doesnt/matter");
+ assert!(!request.body_has_info_sections);
+ assert_eq!(request.ping_name, "whatevs");
// Make sure all the expected headers were added.
assert!(request.headers.contains_key("X-Telemetry-Agent"));
diff --git a/glean-core/tests/event.rs b/glean-core/tests/event.rs
index ed8f7d807f..c83e225ca2 100644
--- a/glean-core/tests/event.rs
+++ b/glean-core/tests/event.rs
@@ -166,6 +166,7 @@ fn test_sending_of_event_ping_when_it_fills_up() {
true,
false,
true,
+ true,
vec!["max_capacity".to_string()],
));
}
@@ -450,6 +451,7 @@ fn event_storage_trimming() {
true,
false,
true,
+ true,
vec![],
));
diff --git a/glean-core/tests/object.rs b/glean-core/tests/object.rs
new file mode 100644
index 0000000000..1e734e99d2
--- /dev/null
+++ b/glean-core/tests/object.rs
@@ -0,0 +1,104 @@
+// This Source Code Form is subject to the terms of the Mozilla Public
+// License, v. 2.0. If a copy of the MPL was not distributed with this
+// file, You can obtain one at https://mozilla.org/MPL/2.0/.
+
+mod common;
+use crate::common::*;
+
+use serde_json::json;
+
+use glean_core::metrics::*;
+use glean_core::storage::StorageManager;
+use glean_core::{CommonMetricData, Lifetime};
+
+#[test]
+fn object_serializer_should_correctly_serialize_objects() {
+ let (mut tempdir, _) = tempdir();
+
+ {
+ // We give tempdir to the `new_glean` function...
+ let (glean, dir) = new_glean(Some(tempdir));
+ // And then we get it back once that function returns.
+ tempdir = dir;
+
+ let metric = ObjectMetric::new(CommonMetricData {
+ name: "object_metric".into(),
+ category: "telemetry".into(),
+ send_in_pings: vec!["store1".into()],
+ disabled: false,
+ lifetime: Lifetime::User,
+ ..Default::default()
+ });
+
+ let obj = serde_json::from_str("{ \"value\": 1 }").unwrap();
+ metric.set_sync(&glean, obj);
+
+ let snapshot = StorageManager
+ .snapshot_as_json(glean.storage(), "store1", true)
+ .unwrap();
+ assert_eq!(
+ json!({"object": {"telemetry.object_metric": { "value": 1 }}}),
+ snapshot
+ );
+ }
+
+ // Make a new Glean instance here, which should force reloading of the data from disk
+ // so we can ensure it persisted, because it has User lifetime
+ {
+ let (glean, _t) = new_glean(Some(tempdir));
+ let snapshot = StorageManager
+ .snapshot_as_json(glean.storage(), "store1", true)
+ .unwrap();
+ assert_eq!(
+ json!({"object": {"telemetry.object_metric": { "value": 1 }}}),
+ snapshot
+ );
+ }
+}
+
+#[test]
+fn set_value_properly_sets_the_value_in_all_stores() {
+ let (glean, _t) = new_glean(None);
+ let store_names: Vec = vec!["store1".into(), "store2".into()];
+
+ let metric = ObjectMetric::new(CommonMetricData {
+ name: "object_metric".into(),
+ category: "telemetry".into(),
+ send_in_pings: store_names.clone(),
+ disabled: false,
+ lifetime: Lifetime::Ping,
+ ..Default::default()
+ });
+
+ let obj = serde_json::from_str("{ \"value\": 1 }").unwrap();
+ metric.set_sync(&glean, obj);
+
+ for store_name in store_names {
+ let snapshot = StorageManager
+ .snapshot_as_json(glean.storage(), &store_name, true)
+ .unwrap();
+
+ assert_eq!(
+ json!({"object": {"telemetry.object_metric": { "value": 1 }}}),
+ snapshot
+ );
+ }
+}
+
+#[test]
+fn getting_data_json_encoded() {
+ let (glean, _t) = new_glean(None);
+
+ let object: ObjectMetric = ObjectMetric::new(CommonMetricData {
+ name: "transformation".into(),
+ category: "local".into(),
+ send_in_pings: vec!["store1".into()],
+ ..Default::default()
+ });
+
+ let obj_str = "{\"value\":1}";
+ let obj = serde_json::from_str(obj_str).unwrap();
+ object.set_sync(&glean, obj);
+
+ assert_eq!(obj_str, object.get_value(&glean, Some("store1")).unwrap());
+}
diff --git a/glean-core/tests/ping.rs b/glean-core/tests/ping.rs
index 0ee3736168..17944b4c24 100644
--- a/glean-core/tests/ping.rs
+++ b/glean-core/tests/ping.rs
@@ -15,7 +15,7 @@ use glean_core::Lifetime;
fn write_ping_to_disk() {
let (mut glean, _temp) = new_glean(None);
- let ping = PingType::new("metrics", true, false, true, vec![]);
+ let ping = PingType::new("metrics", true, false, true, true, vec![]);
glean.register_ping_type(&ping);
// We need to store a metric as an empty ping is not stored.
@@ -36,7 +36,7 @@ fn write_ping_to_disk() {
fn disabling_upload_clears_pending_pings() {
let (mut glean, _t) = new_glean(None);
- let ping = PingType::new("metrics", true, false, true, vec![]);
+ let ping = PingType::new("metrics", true, false, true, true, vec![]);
glean.register_ping_type(&ping);
// We need to store a metric as an empty ping is not stored.
@@ -105,9 +105,9 @@ fn deletion_request_only_when_toggled_from_on_to_off() {
fn empty_pings_with_flag_are_sent() {
let (mut glean, _t) = new_glean(None);
- let ping1 = PingType::new("custom-ping1", true, true, true, vec![]);
+ let ping1 = PingType::new("custom-ping1", true, true, true, true, vec![]);
glean.register_ping_type(&ping1);
- let ping2 = PingType::new("custom-ping2", true, false, true, vec![]);
+ let ping2 = PingType::new("custom-ping2", true, false, true, true, vec![]);
glean.register_ping_type(&ping2);
// No data is stored in either of the custom pings
@@ -139,10 +139,10 @@ fn test_pings_submitted_metric() {
None,
);
- let metrics_ping = PingType::new("metrics", true, false, true, vec![]);
+ let metrics_ping = PingType::new("metrics", true, false, true, true, vec![]);
glean.register_ping_type(&metrics_ping);
- let baseline_ping = PingType::new("baseline", true, false, true, vec![]);
+ let baseline_ping = PingType::new("baseline", true, false, true, true, vec![]);
glean.register_ping_type(&baseline_ping);
// We need to store a metric as an empty ping is not stored.
@@ -218,7 +218,7 @@ fn test_pings_submitted_metric() {
fn events_ping_with_metric_but_no_events_is_not_sent() {
let (mut glean, _t) = new_glean(None);
- let events_ping = PingType::new("events", true, true, true, vec![]);
+ let events_ping = PingType::new("events", true, true, true, true, vec![]);
glean.register_ping_type(&events_ping);
let counter = CounterMetric::new(CommonMetricData {
name: "counter".into(),
diff --git a/glean-core/tests/ping_maker.rs b/glean-core/tests/ping_maker.rs
index 29b6bccaca..a7eba0ea84 100644
--- a/glean-core/tests/ping_maker.rs
+++ b/glean-core/tests/ping_maker.rs
@@ -13,7 +13,7 @@ fn set_up_basic_ping() -> (Glean, PingMaker, PingType, tempfile::TempDir) {
let (tempdir, _) = tempdir();
let (mut glean, t) = new_glean(Some(tempdir));
let ping_maker = PingMaker::new();
- let ping_type = PingType::new("store1", true, false, true, vec![]);
+ let ping_type = PingType::new("store1", true, false, true, true, vec![]);
glean.register_ping_type(&ping_type);
// Record something, so the ping will have data
@@ -94,7 +94,7 @@ fn test_metrics_must_report_experimentation_id() {
})
.unwrap();
let ping_maker = PingMaker::new();
- let ping_type = PingType::new("store1", true, false, true, vec![]);
+ let ping_type = PingType::new("store1", true, false, true, true, vec![]);
glean.register_ping_type(&ping_type);
// Record something, so the ping will have data
@@ -147,7 +147,7 @@ fn experimentation_id_is_removed_if_send_if_empty_is_false() {
.unwrap();
let ping_maker = PingMaker::new();
- let unknown_ping_type = PingType::new("unknown", true, false, true, vec![]);
+ let unknown_ping_type = PingType::new("unknown", true, false, true, true, vec![]);
glean.register_ping_type(&unknown_ping_type);
assert!(ping_maker
@@ -163,7 +163,7 @@ fn collect_must_report_none_when_no_data_is_stored() {
let (mut glean, ping_maker, ping_type, _t) = set_up_basic_ping();
- let unknown_ping_type = PingType::new("unknown", true, false, true, vec![]);
+ let unknown_ping_type = PingType::new("unknown", true, false, true, true, vec![]);
glean.register_ping_type(&ping_type);
assert!(ping_maker
@@ -187,7 +187,7 @@ fn seq_number_must_be_sequential() {
for i in 0..=1 {
for ping_name in ["store1", "store2"].iter() {
- let ping_type = PingType::new(*ping_name, true, false, true, vec![]);
+ let ping_type = PingType::new(*ping_name, true, false, true, true, vec![]);
let ping = ping_maker
.collect(&glean, &ping_type, None, "", "")
.unwrap();
@@ -200,7 +200,7 @@ fn seq_number_must_be_sequential() {
// Test that ping sequence numbers increase independently.
{
- let ping_type = PingType::new("store1", true, false, true, vec![]);
+ let ping_type = PingType::new("store1", true, false, true, true, vec![]);
// 3rd ping of store1
let ping = ping_maker
@@ -218,7 +218,7 @@ fn seq_number_must_be_sequential() {
}
{
- let ping_type = PingType::new("store2", true, false, true, vec![]);
+ let ping_type = PingType::new("store2", true, false, true, true, vec![]);
// 3rd ping of store2
let ping = ping_maker
@@ -229,7 +229,7 @@ fn seq_number_must_be_sequential() {
}
{
- let ping_type = PingType::new("store1", true, false, true, vec![]);
+ let ping_type = PingType::new("store1", true, false, true, true, vec![]);
// 5th ping of store1
let ping = ping_maker
@@ -244,7 +244,7 @@ fn seq_number_must_be_sequential() {
fn clear_pending_pings() {
let (mut glean, _t) = new_glean(None);
let ping_maker = PingMaker::new();
- let ping_type = PingType::new("store1", true, false, true, vec![]);
+ let ping_type = PingType::new("store1", true, false, true, true, vec![]);
glean.register_ping_type(&ping_type);
// Record something, so the ping will have data
@@ -272,7 +272,7 @@ fn no_pings_submitted_if_upload_disabled() {
// Regression test, bug 1603571
let (mut glean, _t) = new_glean(None);
- let ping_type = PingType::new("store1", true, true, true, vec![]);
+ let ping_type = PingType::new("store1", true, true, true, true, vec![]);
glean.register_ping_type(&ping_type);
assert!(ping_type.submit_sync(&glean, None));
@@ -290,7 +290,7 @@ fn no_pings_submitted_if_upload_disabled() {
fn metadata_is_correctly_added_when_necessary() {
let (mut glean, _t) = new_glean(None);
glean.set_debug_view_tag("valid-tag");
- let ping_type = PingType::new("store1", true, true, true, vec![]);
+ let ping_type = PingType::new("store1", true, true, true, true, vec![]);
glean.register_ping_type(&ping_type);
assert!(ping_type.submit_sync(&glean, None));
diff --git a/gradle-plugin/src/main/groovy/mozilla/telemetry/glean-gradle-plugin/GleanGradlePlugin.groovy b/gradle-plugin/src/main/groovy/mozilla/telemetry/glean-gradle-plugin/GleanGradlePlugin.groovy
index 958dd46cb3..49d4ddac7f 100644
--- a/gradle-plugin/src/main/groovy/mozilla/telemetry/glean-gradle-plugin/GleanGradlePlugin.groovy
+++ b/gradle-plugin/src/main/groovy/mozilla/telemetry/glean-gradle-plugin/GleanGradlePlugin.groovy
@@ -50,7 +50,7 @@ abstract class GleanMetricsYamlTransform implements TransformAction {
// The version of glean_parser to install from PyPI.
- private String GLEAN_PARSER_VERSION = "11.0"
+ private String GLEAN_PARSER_VERSION = "13.0"
// The version of Miniconda is explicitly specified.
// Miniconda3-4.5.12 is known to not work on Windows.
private String MINICONDA_VERSION = "4.5.11"
@@ -552,7 +552,7 @@ except:
void apply(Project project) {
isOffline = project.gradle.startParameter.offline
- project.ext.glean_version = "57.0.0"
+ project.ext.glean_version = "58.0.0"
def parserVersion = gleanParserVersion(project)
// Print the required glean_parser version to the console. This is
diff --git a/gradle/libs.versions.toml b/gradle/libs.versions.toml
new file mode 100644
index 0000000000..beb75f0f8f
--- /dev/null
+++ b/gradle/libs.versions.toml
@@ -0,0 +1,90 @@
+# In general, these versions should be kept in sync with AC to avoid introducing
+# possible conflicts and compatibility issues. This primarily applies to dependencies
+# for shipping code, however. Libraries used only for the build system or testing
+# can be safely bumped when convenient.
+
+[versions]
+# AGP
+android-plugin = "8.0.2"
+
+# Kotlin
+kotlin-compiler = "1.8.22"
+kotlinx-coroutines = "1.7.2"
+
+# Mozilla
+rust-android-gradle = "0.9.3"
+
+# AndroidX
+androidx-annotation = "1.7.1"
+androidx-appcompat = "1.6.1"
+androidx-browser = "1.7.0"
+androidx-lifecycle = "2.7.0"
+androidx-work = "2.9.0"
+
+# JNA
+jna = "5.14.0"
+
+# Linting and Static Analysis
+detekt = "1.23.5"
+ktlint = "0.50.0"
+
+# AndroidX Testing
+androidx-test-espresso = "3.5.1"
+androidx-test-core = "1.5.0"
+androidx-test-junit = "1.1.5"
+androidx-test-runner = "1.5.2"
+androidx-test-uiautomator = "2.3.0"
+
+# Third Party Testing
+junit = "4.13.2"
+mockito = "5.10.0"
+mockwebserver = "4.12.0"
+robolectric = "4.11.1"
+
+# Miscellaneous Gradle plugins
+jacoco = "0.8.11"
+python-envs = "0.0.31"
+
+[libraries]
+# AGP
+tools-android-plugin = { group = "com.android.tools.build", name = "gradle", version.ref = "android-plugin" }
+
+# Kotlin
+kotlin-gradle-plugin = { group = "org.jetbrains.kotlin", name = "kotlin-gradle-plugin", version.ref = "kotlin-compiler" }
+kotlinx-coroutines = { group = "org.jetbrains.kotlinx", name = "kotlinx-coroutines-android", version.ref = "kotlinx-coroutines" }
+
+# Mozilla
+mozilla-rust-android-gradle = { group = "org.mozilla.rust-android-gradle", name = "plugin", version.ref = "rust-android-gradle" }
+
+# AndroidX
+androidx-annotation = { group = "androidx.annotation", name = "annotation", version.ref = "androidx-annotation" }
+androidx-appcompat = { group = "androidx.appcompat", name = "appcompat", version.ref = "androidx-appcompat" }
+androidx-browser = { group = "androidx.browser", name = "browser", version.ref = "androidx-browser" }
+androidx-lifecycle-common = { group = "androidx.lifecycle", name = "lifecycle-common", version.ref = "androidx-lifecycle" }
+androidx-lifecycle-process = { group = "androidx.lifecycle", name = "lifecycle-process", version.ref = "androidx-lifecycle" }
+androidx-work = { group = "androidx.work", name = "work-runtime-ktx", version.ref = "androidx-work" }
+
+# JNA
+jna = { group = "net.java.dev.jna", name = "jna", version.ref = "jna" }
+
+# Linting and Static Analysis
+ktlint = { module = "com.pinterest:ktlint", version.ref = "ktlint" }
+
+# AndroidX Testing
+test-core = { group = "androidx.test", name = "core-ktx", version.ref = "androidx-test-core" }
+test-espresso-core = { group = "androidx.test.espresso", name = "espresso-core", version.ref = "androidx-test-espresso" }
+test-junit-ext = { group = "androidx.test.ext", name = "junit-ktx", version.ref = "androidx-test-junit" }
+test-rules = { group = "androidx.test", name = "rules", version.ref = "androidx-test-core" }
+test-runner = { group = "androidx.test", name = "runner", version.ref = "androidx-test-runner" }
+test-uiautomator = { group = "androidx.test.uiautomator", name = "uiautomator", version.ref = "androidx-test-uiautomator" }
+test-work = { group = "androidx.work", name = "work-testing", version.ref = "androidx-work" }
+
+# Third Party Testing
+junit = { group = "junit", name = "junit", version.ref = "junit" }
+mockito = { group = "org.mockito", name = "mockito-core", version.ref = "mockito" }
+mockwebserver = { group = "com.squareup.okhttp3", name = "mockwebserver", version.ref = "mockwebserver" }
+robolectric = { group = "org.robolectric", name = "robolectric", version.ref = "robolectric" }
+
+[plugins]
+detekt = { id = "io.gitlab.arturbosch.detekt", version.ref = "detekt" }
+gradle-python-envs = { id = "com.jetbrains.python.envs", version.ref = "python-envs" }
diff --git a/pyproject.toml b/pyproject.toml
index f9793c9eab..66d3d18cad 100644
--- a/pyproject.toml
+++ b/pyproject.toml
@@ -4,7 +4,7 @@ build-backend = "maturin"
[project]
name = "glean-sdk"
-version = "57.0.0"
+version = "58.0.0"
requires-python = ">=3.8"
classifiers = [
"Intended Audience :: Developers",
@@ -22,7 +22,7 @@ maintainers = [
dependencies = [
"semver>=2.13.0",
- "glean_parser~=11.0",
+ "glean_parser~=13.0",
]
[project.urls]
diff --git a/samples/android/app/build.gradle b/samples/android/app/build.gradle
index 5d8bae83c2..3d6233d74b 100644
--- a/samples/android/app/build.gradle
+++ b/samples/android/app/build.gradle
@@ -3,7 +3,7 @@
* file, You can obtain one at http://mozilla.org/MPL/2.0/. */
plugins {
- id "com.jetbrains.python.envs" version "0.0.26"
+ alias libs.plugins.gradle.python.envs
}
apply plugin: 'com.android.application'
@@ -41,18 +41,17 @@ android {
dependencies {
implementation project(':glean')
- implementation "androidx.appcompat:appcompat:$rootProject.versions.androidx_appcompat"
- implementation "androidx.browser:browser:$rootProject.versions.androidx_browser"
+ implementation libs.androidx.appcompat
+ implementation libs.androidx.browser
-
- androidTestImplementation "androidx.test:core-ktx:$rootProject.versions.androidx_test"
- androidTestImplementation "androidx.test:runner:$rootProject.versions.androidx_test"
- androidTestImplementation "androidx.test:rules:$rootProject.versions.androidx_test"
- androidTestImplementation "androidx.test.ext:junit:$rootProject.versions.androidx_junit"
- androidTestImplementation "androidx.test.uiautomator:uiautomator:$rootProject.versions.androidx_uiautomator"
- androidTestImplementation "androidx.test.espresso:espresso-core:$rootProject.versions.androidx_espresso"
- androidTestImplementation "androidx.work:work-testing:$rootProject.versions.androidx_work"
- androidTestImplementation "com.squareup.okhttp3:mockwebserver:$rootProject.versions.mockwebserver"
+ androidTestImplementation libs.mockwebserver
+ androidTestImplementation libs.test.core
+ androidTestImplementation libs.test.espresso.core
+ androidTestImplementation libs.test.junit.ext
+ androidTestImplementation libs.test.rules
+ androidTestImplementation libs.test.runner
+ androidTestImplementation libs.test.uiautomator
+ androidTestImplementation libs.test.work
}
ext.gleanNamespace = "mozilla.telemetry.glean"
diff --git a/samples/rust/Cargo.toml b/samples/rust/Cargo.toml
index 6b453156a1..6c397056db 100644
--- a/samples/rust/Cargo.toml
+++ b/samples/rust/Cargo.toml
@@ -7,6 +7,7 @@ license = "MIT"
[dependencies]
env_logger = { version = "0.10.0", default-features = false, features = ["humantime"] }
+flate2 = "1.0.19"
glean = { path = "../../glean-core/rlb" }
tempfile = "3.3.0"
diff --git a/samples/rust/metrics.yaml b/samples/rust/metrics.yaml
index 9de283ac61..f1cbf928ea 100644
--- a/samples/rust/metrics.yaml
+++ b/samples/rust/metrics.yaml
@@ -24,3 +24,27 @@ test.metrics:
expires: never
send_in_pings:
- prototype
+
+party:
+ balloons:
+ type: object
+ description: |
+ Just testing objects
+ bugs:
+ - https://bugzilla.mozilla.org/1839640
+ data_reviews:
+ - N/A
+ notification_emails:
+ - CHANGE-ME@example.com
+ expires: never
+ send_in_pings:
+ - prototype
+ structure:
+ type: array
+ items:
+ type: object
+ properties:
+ colour:
+ type: string
+ diameter:
+ type: number
diff --git a/samples/rust/src/main.rs b/samples/rust/src/main.rs
index 437e302e78..a221cb31a9 100644
--- a/samples/rust/src/main.rs
+++ b/samples/rust/src/main.rs
@@ -3,18 +3,60 @@
// file, You can obtain one at https://mozilla.org/MPL/2.0/.
use std::env;
+use std::fs::File;
+use std::io::{Read, Write};
use std::path::PathBuf;
use std::thread;
use std::time::Duration;
use tempfile::Builder;
-use glean::{ClientInfoMetrics, Configuration};
+use flate2::read::GzDecoder;
+use glean::{net, ClientInfoMetrics, ConfigurationBuilder};
pub mod glean_metrics {
include!(concat!(env!("OUT_DIR"), "/glean_metrics.rs"));
}
+#[derive(Debug)]
+struct MovingUploader(String);
+
+impl net::PingUploader for MovingUploader {
+ fn upload(&self, upload_request: net::PingUploadRequest) -> net::UploadResult {
+ let net::PingUploadRequest {
+ body, url, headers, ..
+ } = upload_request;
+ let mut gzip_decoder = GzDecoder::new(&body[..]);
+ let mut s = String::with_capacity(body.len());
+
+ let data = gzip_decoder
+ .read_to_string(&mut s)
+ .ok()
+ .map(|_| &s[..])
+ .or_else(|| std::str::from_utf8(&body).ok())
+ .unwrap();
+
+ let mut out_path = PathBuf::from(&self.0);
+ out_path.push("sent_pings");
+ std::fs::create_dir_all(&out_path).unwrap();
+
+ let docid = url.rsplit('/').next().unwrap();
+ out_path.push(format!("{docid}.json"));
+ let mut fp = File::create(out_path).unwrap();
+
+ // pseudo-JSON, let's hope this works.
+ writeln!(fp, "{{").unwrap();
+ writeln!(fp, " \"url\": {url},").unwrap();
+ for (key, val) in headers {
+ writeln!(fp, " \"{key}\": \"{val}\",").unwrap();
+ }
+ writeln!(fp, "}}").unwrap();
+ writeln!(fp, "{data}").unwrap();
+
+ net::UploadResult::http_status(200)
+ }
+}
+
fn main() {
env_logger::init();
@@ -27,21 +69,12 @@ fn main() {
root.path().to_path_buf()
};
- let cfg = Configuration {
- data_path,
- application_id: "org.mozilla.glean_core.example".into(),
- upload_enabled: true,
- max_events: None,
- delay_ping_lifetime_io: false,
- server_endpoint: Some("invalid-test-host".into()),
- uploader: None,
- use_core_mps: true,
- trim_data_to_registered_pings: false,
- log_level: None,
- rate_limit: None,
- enable_event_timestamps: false,
- experimentation_id: None,
- };
+ let uploader = MovingUploader(data_path.display().to_string());
+ let cfg = ConfigurationBuilder::new(true, data_path, "org.mozilla.glean_core.example")
+ .with_server_endpoint("invalid-test-host")
+ .with_use_core_mps(true)
+ .with_uploader(uploader)
+ .build();
let client_info = ClientInfoMetrics {
app_build: env!("CARGO_PKG_VERSION").to_string(),
@@ -54,6 +87,19 @@ fn main() {
glean_metrics::test_metrics::sample_boolean.set(true);
+ use glean_metrics::party::{BalloonsObject, BalloonsObjectItem};
+ let balloons = BalloonsObject::from([
+ BalloonsObjectItem {
+ colour: Some("red".to_string()),
+ diameter: Some(5),
+ },
+ BalloonsObjectItem {
+ colour: Some("blue".to_string()),
+ diameter: None,
+ },
+ ]);
+ glean_metrics::party::balloons.set(balloons);
+
glean_metrics::prototype.submit(None);
// Need to wait a short time for Glean to actually act.
thread::sleep(Duration::from_millis(100));
diff --git a/settings.gradle b/settings.gradle
index 555f813b12..23a74d7ed9 100644
--- a/settings.gradle
+++ b/settings.gradle
@@ -5,7 +5,7 @@
import org.yaml.snakeyaml.Yaml
buildscript {
dependencies {
- classpath 'org.yaml:snakeyaml:2.0'
+ classpath 'org.yaml:snakeyaml:2.2'
}
repositories {
mavenCentral()
diff --git a/supply-chain/audits.toml b/supply-chain/audits.toml
index 02cb39beed..f989a88d11 100644
--- a/supply-chain/audits.toml
+++ b/supply-chain/audits.toml
@@ -262,6 +262,12 @@ who = "Jan-Erik Rediger "
criteria = "safe-to-run"
delta = "0.2.2 -> 0.2.4"
+[[audits.fd-lock]]
+who = "Jan-Erik Rediger "
+criteria = "safe-to-deploy"
+delta = "3.0.12 -> 3.0.13"
+notes = "Dependency updates only"
+
[[audits.flate2]]
who = "Jan-Erik Rediger "
criteria = "safe-to-deploy"
@@ -397,6 +403,12 @@ criteria = "safe-to-deploy"
delta = "1.3.0 -> 1.4.1"
notes = "Internal refactoring, new target support"
+[[audits.xshell-venv]]
+who = "Jan-Erik Rediger "
+criteria = "safe-to-deploy"
+delta = "1.1.0 -> 1.2.0"
+notes = "Added a file lock on the created directory"
+
[[trusted.is-terminal]]
criteria = "safe-to-deploy"
user-id = 6825 # Dan Gohman (sunfishcode)
diff --git a/supply-chain/imports.lock b/supply-chain/imports.lock
index cfff0d5eb1..4fc461232a 100644
--- a/supply-chain/imports.lock
+++ b/supply-chain/imports.lock
@@ -163,6 +163,24 @@ criteria = "safe-to-deploy"
version = "0.1.2"
notes = "This should be portable to any POSIX system and seems like it should be part of the libc crate, but at any rate it's safe as is."
+[[audits.bytecode-alliance.audits.fd-lock]]
+who = "Pat Hickey "
+criteria = "safe-to-deploy"
+version = "3.0.9"
+notes = "This crate uses unsafe to make Windows syscalls, to borrow an Fd with an appropriate lifetime, and to zero a windows API structure that appears to have a valid representation with zeroed memory."
+
+[[audits.bytecode-alliance.audits.fd-lock]]
+who = "Pat Hickey "
+criteria = "safe-to-deploy"
+delta = "3.0.9 -> 3.0.10"
+notes = "Just a dependency version bump"
+
+[[audits.bytecode-alliance.audits.fd-lock]]
+who = "Dan Gohman "
+criteria = "safe-to-deploy"
+delta = "3.0.10 -> 3.0.12"
+notes = "Just a dependency version bump"
+
[[audits.bytecode-alliance.audits.form_urlencoded]]
who = "Alex Crichton "
criteria = "safe-to-deploy"
diff --git a/taskcluster/docker/linux/Dockerfile b/taskcluster/docker/linux/Dockerfile
index e9b5b6f9ed..a0b7158644 100644
--- a/taskcluster/docker/linux/Dockerfile
+++ b/taskcluster/docker/linux/Dockerfile
@@ -22,10 +22,10 @@ WORKDIR /builds/worker/
# Configuration
-ENV ANDROID_BUILD_TOOLS "33.0.2"
-ENV ANDROID_TOOLS_VERSION "9477386"
-ENV ANDROID_PLATFORM_VERSION "33"
-ENV ANDROID_NDK_VERSION "25.2.9519653"
+ENV ANDROID_BUILD_TOOLS "34.0.0"
+ENV ANDROID_TOOLS_VERSION "11076708"
+ENV ANDROID_PLATFORM_VERSION "34"
+ENV ANDROID_NDK_VERSION "26.2.11394342"
# Set up the language variables to avoid problems (we run locale-gen later).
ENV LANG en_US.UTF-8