diff --git a/changelog/28858.txt b/changelog/28858.txt new file mode 100644 index 000000000000..422b9d650609 --- /dev/null +++ b/changelog/28858.txt @@ -0,0 +1,3 @@ +```release-note:feature +**Product Usage Reporting**: Added product usage reporting, which collects anonymous, numerical, non-sensitive data about Vault feature usage, and adds it to the existing utilization reports. +``` \ No newline at end of file diff --git a/website/content/docs/configuration/index.mdx b/website/content/docs/configuration/index.mdx index 0f8d06b6c407..070cd53608eb 100644 --- a/website/content/docs/configuration/index.mdx +++ b/website/content/docs/configuration/index.mdx @@ -13,6 +13,14 @@ The format of this file is [HCL](https://github.com/hashicorp/hcl) or JSON. An example configuration is shown below: + + +For multi-node clusters, replace the loopback address with a valid, routable IP address for each Vault node in your network. + +Refer to the [Vault HA clustering with integrated storage tutorial](/vault/tutorials/raft/raft-storage) for a complete scenario. + + + ```hcl ui = true cluster_addr = "https://127.0.0.1:8201" @@ -66,15 +74,19 @@ to specify where the configuration is. - `user_lockout` `([UserLockout][user-lockout]: nil)` – Configures the user-lockout behaviour for failed logins. For more information, please see the - [user lockout configuration documentation](/vault/docs/configuration/user-lockout). + [user lockout configuration documentation](/vault/docs/configuration/user-lockout). - `seal` `([Seal][seal]: nil)` – Configures the seal type to use for auto-unsealing, as well as for [seal wrapping][sealwrap] as an additional layer of data protection. -- `cluster_name` `(string: )` – Specifies the identifier for the - Vault cluster. If omitted, Vault will generate a value. When connecting to - Vault Enterprise, this value will be used in the interface. +- `reporting` `([Reporting][reporting]: nil)` - + Configures options relating to license reporting in Vault. + +- `cluster_name` `(string: )` – Specifies a human-readable + identifier for the Vault cluster. If omitted, Vault will generate a value. + The cluster name is included as a label in some [telemetry metrics](/vault/docs/internals/telemetry/metrics/). + The cluster name is safe to update on an existing Vault cluster. - `cache_size` `(string: "131072")` – Specifies the size of the read cache used by the physical storage subsystem. The value is in number of entries, so the @@ -115,10 +127,14 @@ to specify where the configuration is. sudo setcap cap_ipc_lock=+ep $(readlink -f $(which vault)) ``` - ~> Note: Since each plugin runs as a separate process, you need to do the same + + + Since each plugin runs as a separate process, you need to do the same for each plugin in your [plugins directory](/vault/docs/plugins/plugin-architecture#plugin-directory). + + If you use a Linux distribution with a modern version of systemd, you can add the following directive to the "[Service]" configuration section: @@ -169,10 +185,10 @@ to specify where the configuration is. maximum request duration allowed before Vault cancels the request. This can be overridden per listener via the `max_request_duration` value. -- `detect_deadlocks` `(string: "")` - A comma separated string that specifies the internal -mutex locks that should be monitored for potential deadlocks. Currently supported values +- `detect_deadlocks` `(string: "")` - A comma separated string that specifies the internal +mutex locks that should be monitored for potential deadlocks. Currently supported values include `statelock`, `quotas` and `expiration` which will cause "POTENTIAL DEADLOCK:" -to be logged when an attempt at a core state lock appears to be deadlocked. Enabling this +to be logged when an attempt at a core state lock appears to be deadlocked. Enabling this can have a negative effect on performance due to the tracking of each lock attempt. - `raw_storage_endpoint` `(bool: false)` – Enables the `sys/raw` endpoint which @@ -210,12 +226,20 @@ can have a negative effect on performance due to the tracking of each lock attem Supported values (in order of descending detail) are `trace`, `debug`, `info`, `warn`, and `error`. This can also be specified via the `VAULT_LOG_LEVEL` environment variable. - ~> Note: On SIGHUP (`sudo kill -s HUP` _pid of vault_), if a valid value is specified, Vault will update the existing log level, + + + On SIGHUP (`sudo kill -s HUP` _pid of vault_), if a valid value is specified, Vault will update the existing log level, overriding (even if specified) both the CLI flag and environment variable. - ~> Note: Not all parts of Vault's logging can have its log level be changed dynamically this way; in particular, + + + + + Not all parts of Vault's logging can have its log level be changed dynamically this way; in particular, secrets/auth plugins are currently not updated dynamically. + + - `log_format` - Equivalent to the [`-log-format` command-line flag](/vault/docs/commands/server#_log_format). - `log_file` - Equivalent to the [`-log-file` command-line flag](/vault/docs/commands/server#_log_file). @@ -237,7 +261,7 @@ can have a negative effect on performance due to the tracking of each lock attem `imprecise_lease_role_tracking` affects role-based lease count quotas, but reduces latencies when not using role based quotas. - `request_limiter` `([Request Limiter][request-limiter]: )` – Allows - operators to enable Vault's Request Limiter functionality. +operators to enable Vault's Request Limiter functionality. ### High availability parameters diff --git a/website/content/docs/configuration/reporting.mdx b/website/content/docs/configuration/reporting.mdx new file mode 100644 index 000000000000..4978d6dd3654 --- /dev/null +++ b/website/content/docs/configuration/reporting.mdx @@ -0,0 +1,42 @@ +--- +layout: docs +page_title: Reporting - Configuration +description: |- + The reporting stanza specifies various parameters for tuning reporting and licensing related values. +--- + +# `replication` stanza + +@include 'alerts/enterprise-only.mdx' + +The `reporting` stanza specifies various parameters for tuning replication related values. + +Please see the pages relating to +[license utilization reporting](/vault/docs/enterprise/license/utilization-reporting) +and [product usage reporting](/vault/docs/enterprise/license/product-usage-reporting) +for more information regarding the license reporting. + +```hcl +reporting { + snapshot_retention_time = 9600 + disable_product_usage_reporting = false + license { + enabled = true + } +} +``` + +## `reporting` parameters + +- `snapshot_retention_time` `(duration: 9600h)` - The retention time for manual reporting snapshots in hours. Defaults to 9600 (400 days). +Uses [duration format strings](/vault/docs/concepts/duration-format). +- `disable_product_usage_reporting` `(boolean: false)` - Determines whether [product usage reporting](/vault/docs/enterprise/license/product-usage-reporting) +is enabled. + +## `license` parameters + +- `enabled` `(boolean: true)` - Toggles automatic reporting of license utilization. See the +[license utilization reporting](/vault/docs/enterprise/license/utilization-reporting) +page for more details. +- `billing_start_timestamp` `(timestamp)` - The start timestamp for billing for license reporting (manual and automated). +Defaults to the license start timestamp. diff --git a/website/content/docs/enterprise/license/product-usage-reporting.mdx b/website/content/docs/enterprise/license/product-usage-reporting.mdx new file mode 100644 index 000000000000..860d2bc28c35 --- /dev/null +++ b/website/content/docs/enterprise/license/product-usage-reporting.mdx @@ -0,0 +1,131 @@ +--- +layout: docs +page_title: Product usage reporting +description: >- + Learn what anonymous usage data HashiCorp collects as part of Enterprise utilization reporting. Enable or disable collection. +--- + +# Product usage reporting + +@include 'alerts/enterprise-only.mdx' + +HashiCorp collects usage data about how Vault clusters are being used. This data is not +used for billing or and is numerical only, and no sensitive information of +any nature is being collected. The data is GDPR compliant and is collected as part of +the [license utilization reporting](/vault/docs/enterprise/license/utilization-reporting) +process. If automated reporting is enabled, this data will be collected automatically. +If automated reporting is disabled, then this will be collected as part of the manual reports. + +## Opt out + +While none of the collected usage metrics are sensitive in any way, if you are still concerned +about these usage metrics being reported, then you can opt-out of them being collected. + +If you are considering opting out because you’re worried about the data, we +strongly recommend that you review the [usage metrics list](#usage-metrics-list) +before opting out. If you have concerns with any of the automatically-reported +data please bring them to your account manager. + +You have two options to opt out of product usage collection: + +- HCL configuration (recommended) +- Environment variable (requires restart) + + +#### HCL configuration + +Opting out in your product's configuration file doesn't require a system +restart, and is the method we recommend. Add the following block to your server +configuration file (e.g. `vault-config.hcl`). + +```hcl +reporting { + disable_product_usage_reporting = true +} +``` + + + + When you have a cluster, each node must have the reporting stanza in its + configuration to be consistent. In the event of leadership change, nodes will + use its server configuration to determine whether or not to opt-out the + product usage collection. Inconsistent configuration between nodes will change the + reporting status upon active unseal. + + + + +You will find the following entries in the server log. + + + + ``` + [DEBUG] activity: there is no reporting agent configured, skipping counts reporting + ``` + + + +#### Environment variable + +If you need to, you can also opt out using an environment variable, which will +provide a startup message confirming that you have product usage data collection. +This option requires a system restart. + + + + If the reporting stanza exists in the configuration file, the + `OPTOUT_PRODUCT_USAGE_REPORTING` value overrides the configuration. + + + +Set the following environment variable. + +```shell-session +$ export OPTOUT_PRODUCT_USAGE_REPORTING=true +``` + +Now, restart your [Vault servers](/vault/docs/commands/server) from the shell +where you set the environment variable. + +You will find the following entries in the server log. + + + + ``` + [DEBUG] core: product usage reporting disabled + ``` + + + +If your configuration file and environment variable differ, the environment +variable setting will take precedence. + +## Usage metrics list + +HashiCorp collects the following product usage metrics as part of the `metrics` part of the +[JSON payload that it collects for licence utilization](/vault/docs/enterprise/license/utilization-reporting#example-payloads). +All of these metrics are numerical, and contain no sensitive values or additional metadata: + +| Metric Name | Description | +|--------------------------------------------|--------------------------------------------------------------------------| +| `vault.namespaces.count` | Total number of namespaces. | +| `vault.leases.count` | Total number of leases within Vault. | +| `vault.quotas.ratelimit.count` | Total number of rate limit quotas within Vault. | +| `vault.quotas.leasecount.count` | Total number of lease count quotas within Vault. | +| `vault.kv.version1.secrets.count` | Total number of KVv1 secrets within Vault. | +| `vault.kv.version2.secrets.count` | Total number of KVv2 secrets within Vault. | +| `vault.kv.version1.secrets.namespace.max` | The highest number of KVv1 secrets in a namespace in Vault, e.g. `1000`. | +| `vault.kv.version2.secrets.namespace.max` | The highest number of KVv2 secrets in a namespace in Vault, e.g. `1000`. | +| `vault.kv.version1.secrets.namespace.min` | The lowest number of KVv1 secrets in a namespace in Vault, e.g. `2`. | +| `vault.kv.version2.secrets.namespace.min` | The highest number of KVv2 secrets in a namespace in Vault, e.g. `1000`. | +| `vault.kv.version1.secrets.namespace.mean` | The mean number of KVv1 secrets in namespaces in Vault, e.g. `52.8`. | +| `vault.kv.version1.secrets.namespace.mean` | The mean number of KVv2 secrets in namespaces in Vault, e.g. `52.8`. | + +## Usage metadata list + +HashiCorp collects the following product usage metadata as part of the `metadata` part of the +[JSON payload that it collects for licence utilization](/vault/docs/enterprise/license/utilization-reporting#example-payloads): + +| Metadata Name | Description | +|----------------------|----------------------------------------------------------------------| +| `replication_status` | Replication status of this cluster, e.g. `perf-disabled,dr-disabled` | diff --git a/website/content/docs/release-notes/1.16.1.mdx b/website/content/docs/release-notes/1.16.1.mdx index b3600afb0c32..9e3c3e9309e8 100644 --- a/website/content/docs/release-notes/1.16.1.mdx +++ b/website/content/docs/release-notes/1.16.1.mdx @@ -13,16 +13,24 @@ description: |- ## Important changes -| Version | Change | -|-----------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| -| 1.16.0+ | [Existing clusters do not show the current Vault version in UI by default](/vault/docs/upgrading/upgrade-to-1.16.x#default-policy-changes) | -| 1.16.0+ | [Default LCQ enabled when upgrading pre-1.9](/vault/docs/upgrading/upgrade-to-1.16.x#default-lcq-pre-1.9-upgrade) | -| 1.16.0+ | [External plugin environment variables take precedence over server variables](/vault/docs/upgrading/upgrade-to-1.16.x#external-plugin-variables) -| 1.16.0+ | [LDAP auth entity alias names no longer include upndomain](/vault/docs/upgrading/upgrade-to-1.16.x#ldap-auth-entity-alias-names-no-longer-include-upndomain) -| 1.16.0+ | [Secrets Sync now requires a one-time flag to operate](/vault/docs/upgrading/upgrade-to-1.16.x#secrets-sync-now-requires-setting-a-one-time-flag-before-use) -| 1.16.0+ | [Azure secrets engine role creation failing](/vault/docs/upgrading/upgrade-to-1.16.x#azure-secrets-engine-role-creation-failing) -| 1.16.1 - 1.16.3 | [New nodes added by autopilot upgrades provisioned with the wrong version](/vault/docs/upgrading/upgrade-to-1.15.x#new-nodes-added-by-autopilot-upgrades-provisioned-with-the-wrong-version) -| 1.15.8+ | [Autopilot upgrade for Vault Enterprise fails](/vault/docs/upgrading/upgrade-to-1.15.x#autopilot) +| Version | Change | +|-----------------------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| +| 1.16.0+ | [Existing clusters do not show the current Vault version in UI by default](/vault/docs/upgrading/upgrade-to-1.16.x#default-policy-changes) | +| 1.16.0+ | [Default LCQ enabled when upgrading pre-1.9](/vault/docs/upgrading/upgrade-to-1.16.x#default-lcq-pre-1.9-upgrade) | +| 1.16.0+ | [External plugin environment variables take precedence over server variables](/vault/docs/upgrading/upgrade-to-1.16.x#external-plugin-variables) | +| 1.16.0+ | [LDAP auth entity alias names no longer include upndomain](/vault/docs/upgrading/upgrade-to-1.16.x#ldap-auth-entity-alias-names-no-longer-include-upndomain) | +| 1.16.0+ | [Secrets Sync now requires a one-time flag to operate](/vault/docs/upgrading/upgrade-to-1.16.x#secrets-sync-now-requires-setting-a-one-time-flag-before-use) | +| 1.16.0+ | [Azure secrets engine role creation failing](/vault/docs/upgrading/upgrade-to-1.16.x#azure-secrets-engine-role-creation-failing) | +| 1.16.1 - 1.16.3 | [New nodes added by autopilot upgrades provisioned with the wrong version](/vault/docs/upgrading/upgrade-to-1.15.x#new-nodes-added-by-autopilot-upgrades-provisioned-with-the-wrong-version) | +| 1.15.8+ | [Autopilot upgrade for Vault Enterprise fails](/vault/docs/upgrading/upgrade-to-1.15.x#autopilot) | +| 1.16.5 | [Listener stops listening on untrusted upstream connection with particular config settings](/vault/docs/upgrading/upgrade-to-1.16.x#listener-proxy-protocol-config) | +| 1.16.3 - 1.16.6 | [Vault standby nodes not deleting removed entity-aliases from in-memory database](/vault/docs/upgrading/upgrade-to-1.16.x#dangling-entity-alias-in-memory) | +| 0.7.0+ | [Duplicate identity groups created](/vault/docs/upgrading/upgrade-to-1.16.x#duplicate-identity-groups-created-when-concurrent-requests-sent-to-the-primary-and-pr-secondary-cluster) | | +| Known Issue (0.7.0+) | [Manual entity merges fail](/vault/docs/upgrading/upgrade-to-1.16.x#manual-entity-merges-sent-to-a-pr-secondary-cluster-are-not-persisted-to-storage) | +| Known Issue (1.16.7-1.16.8) | [Some values in the audit logs not hmac'd properly](/vault/docs/upgrading/upgrade-to-1.16.x#client-tokens-and-token-accessors-audited-in-plaintext) | +| New default (1.16.13) | [Vault product usage metrics reporting](/vault/docs/upgrading/upgrade-to-1.6.x#product-usage-reporting) | +| Deprecation (1.16.13) | [`default_report_months` is deprecated for the `sys/internal/counters` API](/vault/docs/upgrading/upgrade-to-1.16.x#activity-log-changes) | + ## Vault companion updates @@ -233,8 +241,8 @@ Follow the learn more links for more information, or browse the list of GA - - Use automatic identity tokes for workload identity federation + + Use automatic identity tokes for workload identity federation authentication flows with the AWS secret engine without explicitly configuring sensitive security credentials.

diff --git a/website/content/docs/upgrading/upgrade-to-1.16.x.mdx b/website/content/docs/upgrading/upgrade-to-1.16.x.mdx index 4ea30571d8c6..afb330f54321 100644 --- a/website/content/docs/upgrading/upgrade-to-1.16.x.mdx +++ b/website/content/docs/upgrading/upgrade-to-1.16.x.mdx @@ -88,14 +88,108 @@ operation called an activation-flag. The feature is gated until a Vault operator decides to trigger the flag. More information can be found in the [secrets sync documentation](/vault/docs/sync#activating-the-feature). +### Activity Log Changes + +#### Default Activity Log Querying Period + +As of 1.16.13 and later, the field `default_report_months` can no longer be configured or read. Any previously set values +will be ignored by the system. + + +Attempts to modify `default_report_months` through the +[/sys/internal/counters/config](/vault/api-docs/system/internal-counters#update-the-client-count-configuration) +endpoint, will result in the following warning from Vault: + + + + ```shell-session + + WARNING! The following warnings were returned from Vault: + + * default_report_months is deprecated: defaulting to billing start time + + + ``` + + + + +The `current_billing_period` toggle for `/sys/internal/counters/activity` is also deprecated, as this will be set +true by default. + +Attempts to set `current_billing_period` will result in the following warning from Vault: + + + + ```shell-session + + WARNING! The following warnings were returned from Vault: + + * current_billing_period is deprecated; unless otherwise specified, all requests will default to the current billing period + + + ``` + + + ### Auto-rolled billing start date -As of 1.16.7 and later, the billing start date (license start date if not configured) automatically rolls over to the latest billing year at the end of the last cycle. +As of 1.16.7 and later, the billing start date (license start date if not configured) automatically rolls over to the latest billing year at the end of the last cycle. @include 'auto-roll-billing-start.mdx' @include 'auto-roll-billing-start-example.mdx' +======= +### Docker image no longer contains `curl` + +As of 1.16.7 and later, the `curl` binary is no longer included in the published Docker container +images for Vault and Vault Enterprise. If your workflow depends on `curl` being available in the +container, consider one of the following strategies: + +#### Create a wrapper container image + +Use the HashiCorp image as a base image to create a new container image with `curl` installed. + +```Dockerfile +FROM hashicorp/vault-enterprise +RUN apk add curl +``` + +**NOTE:** While this is the preferred option it will require managing your own registry and rebuilding new images. + +#### Install it at runtime dynamically + +When running the image as root (not recommended), you can install it at runtime dynamically by using the `apk` package manager: + +```shell-session +docker exec apk add curl +``` +```shell-session +kubectl exec -ti -- apk add curl +``` + +When running the image as non-root without privilege escalation (recommended) you can use existing +tools to install a static binary of `curl` into the `vault` users home directory: + +```shell-session +docker exec wget https://github.com/moparisthebest/static-curl/releases/latest/download/curl-amd64 -O /home/vault/curl && chmod +x /home/vault/curl +``` +```shell-session +kubectl exec -ti -- wget https://github.com/moparisthebest/static-curl/releases/latest/download/curl-amd64 -O /home/vault/curl && chmod +x /home/vault/curl +``` + +**NOTE:** When using this option you'll want to verify that the static binary comes from a trusted source. + +### Product usage reporting + +As of 1.16.13, Vault will collect anonymous product usage metrics for HashiCorp. This information will be collected +alongside client activity data, and will be sent automatically if automated reporting is configured, or added to manual +reports if manual reporting is preferred. + +See the main page for [Vault product usage metrics reporting](/vault/docs/enterprise/license/product-usage-reporting) for +more details, and information about opt-out. + ## Known issues and workarounds @include 'known-issues/1_16-jwt_auth_bound_audiences.mdx' diff --git a/website/data/docs-nav-data.json b/website/data/docs-nav-data.json index a5d713ba3a2c..65e1d7e4bcf6 100644 --- a/website/data/docs-nav-data.json +++ b/website/data/docs-nav-data.json @@ -387,6 +387,15 @@ }, "path": "configuration/replication" }, + { + "title": "reporting", + "badge": { + "text": "ENTERPRISE", + "type": "outlined", + "color": "neutral" + }, + "path": "configuration/reporting" + }, { "title": "seal", "routes": [ @@ -440,6 +449,11 @@ }, { "title": "sentinel", + "badge": { + "text": "ENTERPRISE", + "type": "outlined", + "color": "neutral" + }, "path": "configuration/sentinel" }, { @@ -2700,6 +2714,10 @@ "title": "Manual license utilization reporting", "path": "enterprise/license/manual-reporting" }, + { + "title": "Product usage reporting", + "path": "enterprise/license/product-usage-reporting" + }, { "title": "FAQ", "path": "enterprise/license/faq"