diff --git a/mozdata/monitoring/index.html b/mozdata/monitoring/index.html index 6e501cb8cf4..c05d5c34430 100644 --- a/mozdata/monitoring/index.html +++ b/mozdata/monitoring/index.html @@ -5081,6 +5081,16 @@

airflow_trigger

Integer Yes + + trigger_id + + Integer + Yes + + created_at + + Timestamp + Yes diff --git a/mozdata/mozilla_org/index.html b/mozdata/mozilla_org/index.html index 73c5d32a0b9..4bf5fef89fc 100644 --- a/mozdata/mozilla_org/index.html +++ b/mozdata/mozilla_org/index.html @@ -3451,6 +3451,173 @@

Mozilla.org

ga_sessions

mozilla_org.ga_sessions

None

+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
Schema
ColumnDescriptionTypeNullable
ga_client_idUniquely identifiers a GA client, using a cookie on moz.org.StringYes
ga_session_idUniquely identifiers a GA session.StringYes
session_dateThe date of the session. Some sessions span two days: if it does, we take the earlier date.DateYes
is_first_sessionWhether this is the first session for the client.BooleanYes
session_numberThe session number for this client. Starts at 1, consecutively increasing.IntegerYes
time_on_siteAmount of time the user was on the site for this session.IntegerYes
pageviewsTotal pageviews for this session.IntegerYes
countryFirst reported country for a GA user.StringYes
regionFirst reported region for a GA user.StringYes
cityFirst reported city for a GA user.StringYes
campaign_idFirst reported campaign ID. Usually associated with AdWords.StringYes
campaignFirst reported campaign value. Usually set by the utm_campaign URL parameter.StringYes
sourceFirst reported source of the traffic. Could be the name of the search engine, the referring hostname, or a value of the utm_source URL parameter. +StringYes
mediumFirst reported medium of the traffic source. Could be 'organic', 'cpc', 'referral', or the value of the utm_medium URL parameter.StringYes
termFirst reported term, or keyword, value. If this was a search results page, this is the keyword entered.StringYes
contentFirst reported ad content of the traffic source. Can be set by the utm_content URL parameter.StringYes
gclidA Google Click ID, which uniquely represent an ad click for Google ads.StringYes
device_categoryFirst reported device category value. The type of device (Mobile, Tablet, Desktop).StringYes
mobile_device_modelFirst reported device model value.StringYes
mobile_device_stringFirst reported mobile device string. The branding, model, and marketing name used to identify the mobile device.StringYes
osFirst reported operating system of the device (e.g., 'Macintosh' or 'Windows').StringYes
os_versionFirst reported os_version value.StringYes
languageFirst reported language the device is set to use. Expressed as the IETF language code.StringYes
browserFirst reported browser used (e.g., 'Chrome' or 'Firefox').StringYes
browser_versionFirst reported browser_version value.StringYes
had_download_eventWhether this session had a download event for Firefox.BooleanYes
last_reported_install_targetThe last reported install target for this session (e.g. 'desktop_release' or 'android_beta').StringYes
all_reported_install_targetsAll install targets reported for this session (e.g. 'desktop_release' or 'android_beta'.StringNo
last_reported_stub_session_idThe last reported Stub Session ID for this session. Can be used to join with `dl_ga_triplets` to get dl_tokens.StringYes
all_reported_stub_session_idsAll reported Stub Session IDs for this session. Can be used to join with `dl_ga_triplets` to get dl_tokens.StringNo
landing_screenThe first reported landing screen for this session. Most sessions only have one, so this is a safe value to use.StringYes
+ diff --git a/mozfun/bits28/index.html b/mozfun/bits28/index.html index fc49426a451..9cb2af122f2 100644 --- a/mozfun/bits28/index.html +++ b/mozfun/bits28/index.html @@ -2820,11 +2820,11 @@
Referenced Tables
-

Parameters

+

Parameters

INPUTS

os_version STRING
@@ -4132,17 +3657,6 @@

Parameters

STRUCT<name STRING, version_name STRING, version_number DECIMAL, build_number INT64>

Source | Edit

-

os (UDF)

-

Normalize an operating system string to one of the three major desktop -platforms, one of the two major mobile platforms, or "Other".

-

This is a reimplementation of -logic used in the data pipeline> - to populate normalized_os.

-

Parameters

-

INPUTS

-
os STRING
-
-

Source | Edit

fenix_build_to_datetime (UDF)

Convert the Fenix client_info.app_build-format string to a DATETIME. May return NULL on failure.

@@ -4164,7 +3678,7 @@

fenix_build_to_datetime (UDF)

This function tolerates both formats.

After using this you may wish to DATETIME_TRUNC(result, DAY) for grouping by build date.

-

Parameters

+

Parameters

INPUTS

app_build STRING
@@ -4172,123 +3686,94 @@

Parameters

INT64

Source | Edit

-

result_type_to_product_name (UDF)

-

Convert urlbar result types into product-friendly names

-

This UDF converts result types from urlbar events (engagement, impression, abandonment) into product-friendly names.

-

Parameters

+

extract_version (UDF)

+

Extracts numeric version data from a version string like +<major>.<minor>.<patch>.

+

Note: Non-zero minor and patch versions will be floating point Numeric.

+

Usage:

+
SELECT
+    mozfun.norm.extract_version(version_string, 'major') as major_version,
+    mozfun.norm.extract_version(version_string, 'minor') as minor_version,
+    mozfun.norm.extract_version(version_string, 'patch') as patch_version
+
+

Example using "96.05.01":

+
SELECT
+    mozfun.norm.extract_version('96.05.01', 'major') as major_version, -- 96
+    mozfun.norm.extract_version('96.05.01', 'minor') as minor_version, -- 5
+    mozfun.norm.extract_version('96.05.01', 'patch') as patch_version  -- 1
+
+

Parameters

INPUTS

-
res STRING
+
version_string STRING, extraction_level STRING
 

 
OUTPUTS

-
STRING
+
NUMERIC
 

-
Source  |  Edit

-
truncate_version (UDF)

-
Truncates a version string like <major>.<minor>.<patch> to
-either the major or minor version. The return value is NUMERIC,
-which means that you can sort the results without fear (e.g. 100
-will be categorized as greater than 80, which isn't the case when
-sorting lexigraphically).

-
For example, "5.1.0" would be translated to 5.1 if the parameter is
-"minor" or 5 if the parameter is major.

-
If the version is only a major and/or minor version, then it will be left
-unchanged (for example "10" would stay as 10 when run through this
-function, no matter what the arguments).

-
This is useful for grouping Linux and Mac operating system versions inside
-aggregate datasets or queries where there may be many different patch
-releases in the field.

-
Parameters

+
Source  |  Edit

+
os (UDF)

+
Normalize an operating system string to one of the three major desktop
+platforms, one of the two major mobile platforms, or "Other".

+
This is a reimplementation of
+logic used in the data pipeline>
+  to populate normalized_os.

+
Parameters

 
INPUTS

-
os_version STRING, truncation_level STRING
+
os STRING
+

+
Source  |  Edit

+
glean_baseline_client_info (UDF)

+
Accepts a glean client_info struct as input and returns a modified struct that includes a few parsed or normalized variants of the input fields.

+
Parameters

+
INPUTS

+
client_info ANY TYPE, metrics ANY TYPE
 

 
OUTPUTS

-
NUMERIC
+
string
 

-
Source  |  Edit

-
fenix_app_info (UDF)

-
Returns canonical, human-understandable identification info for Fenix sources.

-
The Glean telemetry library for Android by design routes pings based
-on the Play Store appId value of the published application.
-As of August 2020, there have been 5 separate Play Store appId
- values associated with different builds of Fenix,
- each corresponding to different datasets in BigQuery,
-and the mapping of appId to logical app names
-(Firefox vs. Firefox Preview) and channel names
- (nightly, beta, or release) has changed over time; see the
-spreadsheet of naming history for Mozilla's mobile browsers.>

-
This function is intended as the source of truth for how to map a
-specific ping in BigQuery to a logical app names and channel.
-It should be expected that the output of this function may evolve
-over time. If we rename a product or channel, we may choose to
-update the values here so that analyses consistently get the new name.

-
The first argument (app_id) can be fairly fuzzy; it is tolerant
-of actual Google Play Store appId values like 'org.mozilla.firefox_beta'
-(mix of periods and underscores) as well as BigQuery dataset names
-with suffixes like 'org_mozilla_firefox_beta_stable'.

-
The second argument (app_build_id) should be the value
-in client_info.app_build.

-
The function returns a STRUCT that contains the logical app_name
-and channel as well as the Play Store app_id in the canonical
-form which would appear in Play Store URLs.

-
Note that the naming of Fenix applications changed on 2020-07-03,
-so to get a continuous view of the pings associated with a logical
-app channel, you may need to union together tables from multiple
-BigQuery datasets. To see data for all Fenix channels together, it
-is necessary to union together tables from all 5 datasets.
-For basic usage information, consider using
-telemetry.fenix_clients_last_seen which already handles the
-union. Otherwise, see the example below as a template for how
-construct a custom union.

-
Mapping of channels to datasets:

-
    
-
  • release: org_mozilla_firefox
    • 
-
  • beta: org_mozilla_firefox_beta (current) and org_mozilla_fenix
    • 
-
  • nightly: org_mozilla_fenix (current), org_mozilla_fennec_aurora,
-and org_mozilla_fenix_nightly
    • 
-

-
-- Example of a query over all Fenix builds advertised as "Firefox Beta"
-CREATE TEMP FUNCTION extract_fields(app_id STRING, m ANY TYPE) AS (
-  (
-    SELECT AS STRUCT
-      m.submission_timestamp,
-      m.metrics.string.geckoview_version,
-      mozfun.norm.fenix_app_info(app_id, m.client_info.app_build).*
-  )
-);
-
-WITH base AS (
-  SELECT
-    extract_fields('org_mozilla_firefox_beta', m).*
-  FROM
-    org_mozilla_firefox_beta.metrics AS m
-  UNION ALL
-  SELECT
-    extract_fields('org_mozilla_fenix', m).*
-  FROM
-    org_mozilla_fenix.metrics AS m
-)
-SELECT
-  DATE(submission_timestamp) AS submission_date,
-  geckoview_version,
-  COUNT(*)
-FROM
-  base
-WHERE
-  app_name = 'Fenix'  -- excludes 'Firefox Preview'
-  AND channel = 'beta'
-  AND DATE(submission_timestamp) = '2020-08-01'
-GROUP BY
-  submission_date,
-  geckoview_version
+
Source  |  Edit

+
get_earliest_value (UDF)

+
This UDF returns the earliest not-null value pair and datetime
+from a list of values and their corresponding timestamp.

+
The function will return the first value pair in the input array,
+that is not null and has the earliest timestamp.

+
Because there may be more than one value on the same date e.g. more
+than one value reported by different pings on the same date, the dates
+must be given as TIMESTAMPS and the values as STRING.

+
Usage:

+
SELECT
+   mozfun.norm.get_earliest_value(ARRAY<STRUCT<value STRING, value_source STRING, value_date DATETIME>>) AS <alias>
 

-
Parameters

+
Parameters

 
INPUTS

-
app_id STRING, app_build_id STRING
+
value_set ARRAY<STRUCT<value STRING, value_source STRING, value_date DATETIME>>
 

 
OUTPUTS

-
STRUCT<app_name STRING, channel STRING, app_id STRING>
+
STRUCT<earliest_value STRING, earliest_value_source STRING, earliest_date DATETIME>
 

-
Source  |  Edit

+
Source  |  Edit

+
browser_version_info (UDF)

+
Adds metadata related to the browser version in a struct.

+
This is a temporary solution that allows browser version analysis.
+It should eventually be replaced with one or more browser version tables
+that serves as a source of truth for version releases.

+
Parameters

+
INPUTS

+
version_string STRING
+

+
OUTPUTS

+
STRUCT<version STRING, major_version NUMERIC, minor_version NUMERIC, patch_revision NUMERIC, is_major_release BOOLEAN>
+

+
Source  |  Edit

+
vpn_attribution (UDF)

+
Accepts vpn attribution fields as input and returns a struct of normalized fields.

+
Parameters

+
INPUTS

+
utm_campaign STRING, utm_content STRING, utm_medium STRING, utm_source STRING
+

+
OUTPUTS

+
STRUCT<normalized_acquisition_channel STRING, normalized_campaign STRING, normalized_content STRING, normalized_medium STRING, normalized_source STRING, website_channel_group STRING>
+

+
Source  |  Edit

 
product_info (UDF)

 
Returns a normalized app_name and canonical_app_name
 for a product based on legacy_app_name and normalized_os values.
@@ -4504,7 +3989,7 @@ 
product_info (UDF)

 
 
 
-
Parameters

+
Parameters

 
INPUTS

 
legacy_app_name STRING, normalized_os STRING
 

@@ -4512,6 +3997,161 @@ 
Parameters

 
STRUCT<app_name STRING, product STRING, canonical_app_name STRING, canonical_name STRING, contributes_to_2019_kpi BOOLEAN, contributes_to_2020_kpi BOOLEAN, contributes_to_2021_kpi BOOLEAN>
 

 
Source  |  Edit

+
glean_ping_info (UDF)

+
Accepts a glean ping_info struct as input and returns a modified struct that includes a few parsed or normalized variants of the input fields.

+
Parameters

+
INPUTS

+
ping_info ANY TYPE
+

+
Source  |  Edit

+
metadata (UDF)

+
Accepts a pipeline metadata struct as input and returns a modified struct that includes a few parsed or normalized variants of the input metadata fields.

+
Parameters

+
INPUTS

+
metadata ANY TYPE
+

+
OUTPUTS

+
`date`, CAST(NULL
+

+
Source  |  Edit

+
fenix_app_info (UDF)

+
Returns canonical, human-understandable identification info for Fenix sources.

+
The Glean telemetry library for Android by design routes pings based
+on the Play Store appId value of the published application.
+As of August 2020, there have been 5 separate Play Store appId
+ values associated with different builds of Fenix,
+ each corresponding to different datasets in BigQuery,
+and the mapping of appId to logical app names
+(Firefox vs. Firefox Preview) and channel names
+ (nightly, beta, or release) has changed over time; see the
+spreadsheet of naming history for Mozilla's mobile browsers.>

+
This function is intended as the source of truth for how to map a
+specific ping in BigQuery to a logical app names and channel.
+It should be expected that the output of this function may evolve
+over time. If we rename a product or channel, we may choose to
+update the values here so that analyses consistently get the new name.

+
The first argument (app_id) can be fairly fuzzy; it is tolerant
+of actual Google Play Store appId values like 'org.mozilla.firefox_beta'
+(mix of periods and underscores) as well as BigQuery dataset names
+with suffixes like 'org_mozilla_firefox_beta_stable'.

+
The second argument (app_build_id) should be the value
+in client_info.app_build.

+
The function returns a STRUCT that contains the logical app_name
+and channel as well as the Play Store app_id in the canonical
+form which would appear in Play Store URLs.

+
Note that the naming of Fenix applications changed on 2020-07-03,
+so to get a continuous view of the pings associated with a logical
+app channel, you may need to union together tables from multiple
+BigQuery datasets. To see data for all Fenix channels together, it
+is necessary to union together tables from all 5 datasets.
+For basic usage information, consider using
+telemetry.fenix_clients_last_seen which already handles the
+union. Otherwise, see the example below as a template for how
+construct a custom union.

+
Mapping of channels to datasets:

+
    
+
  • release: org_mozilla_firefox
    • 
+
  • beta: org_mozilla_firefox_beta (current) and org_mozilla_fenix
    • 
+
  • nightly: org_mozilla_fenix (current), org_mozilla_fennec_aurora,
+and org_mozilla_fenix_nightly
    • 
+

+
-- Example of a query over all Fenix builds advertised as "Firefox Beta"
+CREATE TEMP FUNCTION extract_fields(app_id STRING, m ANY TYPE) AS (
+  (
+    SELECT AS STRUCT
+      m.submission_timestamp,
+      m.metrics.string.geckoview_version,
+      mozfun.norm.fenix_app_info(app_id, m.client_info.app_build).*
+  )
+);
+
+WITH base AS (
+  SELECT
+    extract_fields('org_mozilla_firefox_beta', m).*
+  FROM
+    org_mozilla_firefox_beta.metrics AS m
+  UNION ALL
+  SELECT
+    extract_fields('org_mozilla_fenix', m).*
+  FROM
+    org_mozilla_fenix.metrics AS m
+)
+SELECT
+  DATE(submission_timestamp) AS submission_date,
+  geckoview_version,
+  COUNT(*)
+FROM
+  base
+WHERE
+  app_name = 'Fenix'  -- excludes 'Firefox Preview'
+  AND channel = 'beta'
+  AND DATE(submission_timestamp) = '2020-08-01'
+GROUP BY
+  submission_date,
+  geckoview_version
+

+
Parameters

+
INPUTS

+
app_id STRING, app_build_id STRING
+

+
OUTPUTS

+
STRUCT<app_name STRING, channel STRING, app_id STRING>
+

+
Source  |  Edit

+
diff_months (UDF)

+
Determine the number of whole months after grace period between start and end. Month is dependent on timezone, so start and end must both be datetimes, or both be dates, in the correct timezone. Grace period can be used to account for billing delay, usually 1 day, and is counted after months. When inclusive is FALSE, start and end are not included in whole months.
+For example, diff_months(start => '2021-01-01', end => '2021-03-01', grace_period => INTERVAL 0 day, inclusive => FALSE) returns 1, because start plus two months plus grace period is not less than end. Changing inclusive to TRUE returns 2, because start plus two months plus grace period is less than or equal to end. diff_months(start => '2021-01-01', end => '2021-03-02 00:00:00.000001', grace_period => INTERVAL 1 DAY, inclusive => FALSE) returns 2, because start plus two months plus grace period is less than end.

+
Parameters

+
INPUTS

+
start DATETIME, `end` DATETIME, grace_period INTERVAL, inclusive BOOLEAN
+

+
Source  |  Edit

+
windows_version_info (UDF)

+
Given an unnormalized set off Windows identifiers, return a friendly version
+of the operating system name.

+
Requires os, os_version and windows_build_number.

+
E.G. from windows_build_number >= 22000 return Windows 11

+
Parameters

+
INPUTS

+
os STRING, os_version STRING, windows_build_number INT64
+

+
OUTPUTS

+
STRING
+

+
Source  |  Edit

+
result_type_to_product_name (UDF)

+
Convert urlbar result types into product-friendly names

+
This UDF converts result types from urlbar events (engagement, impression, abandonment) into product-friendly names.

+
Parameters

+
INPUTS

+
res STRING
+

+
OUTPUTS

+
STRING
+

+
Source  |  Edit

+
truncate_version (UDF)

+
Truncates a version string like <major>.<minor>.<patch> to
+either the major or minor version. The return value is NUMERIC,
+which means that you can sort the results without fear (e.g. 100
+will be categorized as greater than 80, which isn't the case when
+sorting lexigraphically).

+
For example, "5.1.0" would be translated to 5.1 if the parameter is
+"minor" or 5 if the parameter is major.

+
If the version is only a major and/or minor version, then it will be left
+unchanged (for example "10" would stay as 10 when run through this
+function, no matter what the arguments).

+
This is useful for grouping Linux and Mac operating system versions inside
+aggregate datasets or queries where there may be many different patch
+releases in the field.

+
Parameters

+
INPUTS

+
os_version STRING, truncation_level STRING
+

+
OUTPUTS

+
NUMERIC
+

+
Source  |  Edit

 
 
 
diff --git a/mozfun/vpn/index.html b/mozfun/vpn/index.html
index c7e738bcf08..898a7540934 100644
--- a/mozfun/vpn/index.html
+++ b/mozfun/vpn/index.html
@@ -3080,11 +3080,11 @@
     
    
       
         
  • 
-  
-    acquisition_channel (UDF)
+  
+    pricing_plan (UDF)
   
   
-