Skip to content

Commit

Permalink
Merge branch 'main' into evgb-StreamBufAllocs
Browse files Browse the repository at this point in the history
  • Loading branch information
jeniawhite authored Jul 12, 2023
2 parents 5552e9b + 2dc3f67 commit 2cd8ca0
Show file tree
Hide file tree
Showing 253 changed files with 2,645 additions and 1,123 deletions.
144 changes: 144 additions & 0 deletions .ci/bump-golang-7.17.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,144 @@
---
name: Bump golang-version to latest version in 7.17
pipelineid: "bump-golang-version-7.17"

scms:
githubConfig:
kind: github
spec:
user: '{{ requiredEnv "GIT_USER" }}'
email: '{{ requiredEnv "GIT_EMAIL" }}'
owner: elastic
repository: beats
token: '{{ requiredEnv "GITHUB_TOKEN" }}'
username: '{{ requiredEnv "GIT_USER" }}'
branch: 7.17

actions:
beats:
kind: github/pullrequest
scmid: githubConfig
sourceid: latestGoVersion
title: '[Automation][7.17] Bump Golang version to {{ source "latestGoVersion" }}'
spec:
automerge: false
labels:
- dependencies
- backport-skip
description: |
Generated automatically with {{ requiredEnv "JOB_URL" }}
sources:
minor:
name: Get minor version in .go-version
kind: shell
transformers:
- findsubmatch:
pattern: '^\d+.(\d+).\d+$'
captureindex: 1
spec:
command: cat .go-version

latestGoVersion:
name: Get Latest Go Release
kind: githubrelease
dependson:
- minor
transformers:
- trimprefix: v
spec:
owner: elastic
repository: golang-crossbuild
token: '{{ requiredEnv "GITHUB_TOKEN" }}'
username: '{{ requiredEnv "GIT_USER" }}'
versionfilter:
kind: regex
pattern: v1\.{{ source "minor" }}\.(\d*)$

gomod:
dependson:
- latestGoVersion
name: Get version in go.mod format
kind: shell
transformers:
- findsubmatch:
pattern: '^(\d+.\d+).\d+'
captureindex: 1
spec:
command: echo {{ source "latestGoVersion" }}

conditions:
dockerTag:
name: Is docker image golang:{{ source "latestGoVersion" }} published
kind: dockerimage
spec:
image: golang
tag: '{{ source "latestGoVersion" }}'
sourceid: latestGoVersion

goDefaultVersion-check:
name: Check if defined golang version differs
kind: shell
sourceid: latestGoVersion
spec:
command: 'grep -v -q {{ source "latestGoVersion" }} .go-version #'

targets:
update-go-version:
name: "Update .go-version"
sourceid: latestGoVersion
scmid: githubConfig
kind: file
spec:
content: '{{ source "latestGoVersion" }}'
file: .go-version
matchpattern: '\d+.\d+.\d+'
update-golang.ci:
name: "Update .golangci.yml"
sourceid: latestGoVersion
scmid: githubConfig
kind: file
spec:
content: '{{ source "latestGoVersion" }}'
file: .golangci.yml
matchpattern: '\d+.\d+.\d+'
update-version.asciidoc:
name: "Update version.asciidoc"
sourceid: latestGoVersion
scmid: githubConfig
kind: file
spec:
content: ':go-version: {{ source "latestGoVersion" }}'
file: libbeat/docs/version.asciidoc
matchpattern: ':go-version: \d+.\d+.\d+'
update-dockerfiles:
name: "Update from dockerfiles"
sourceid: latestGoVersion
scmid: githubConfig
kind: file
spec:
content: 'FROM golang:{{ source "latestGoVersion" }}'
# This list differs from the main branch, this is the main reason we have a separate job
files:
- ./metricbeat/Dockerfile
- ./metricbeat/module/vsphere/_meta/Dockerfile
- ./metricbeat/module/nats/_meta/Dockerfile
- ./metricbeat/module/http/_meta/Dockerfile
- ./filebeat/Dockerfile
- ./auditbeat/Dockerfile
- ./heartbeat/Dockerfile
- ./packetbeat/Dockerfile
- ./libbeat/Dockerfile
- ./x-pack/metricbeat/module/stan/_meta/Dockerfile
- ./x-pack/functionbeat/Dockerfile
- ./x-pack/libbeat/Dockerfile
matchpattern: 'FROM golang:\d+.\d+.\d+'
update-gomod:
name: "Update go.mod"
sourceid: gomod
scmid: githubConfig
kind: file
spec:
content: 'go {{ source "gomod" }}'
file: go.mod
matchpattern: 'go \d+.\d+'
7 changes: 7 additions & 0 deletions .ci/bump-golang.yml
Original file line number Diff line number Diff line change
Expand Up @@ -124,6 +124,13 @@ targets:
- ./metricbeat/Dockerfile
- ./packetbeat/Dockerfile
- ./x-pack/functionbeat/Dockerfile
- ./metricbeat/module/nats/_meta/Dockerfile
- ./metricbeat/module/http/_meta/Dockerfile
- ./metricbeat/module/vsphere/_meta/Dockerfile
- ./dev-tools/kubernetes/metricbeat/Dockerfile.debug
- ./dev-tools/kubernetes/filebeat/Dockerfile.debug
- ./dev-tools/kubernetes/heartbeat/Dockerfile.debug
- ./x-pack/metricbeat/module/stan/_meta/Dockerfile
matchpattern: 'FROM golang:\d+.\d+.\d+'
update-gomod:
name: "Update go.mod"
Expand Down
10 changes: 8 additions & 2 deletions .github/workflows/bump-golang.yml
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ name: bump-golang

on:
schedule:
- cron: '0 20 * * 6'
- cron: "0 20 * * 6"

permissions:
contents: read
Expand All @@ -15,7 +15,6 @@ jobs:
bump:
runs-on: ubuntu-latest
steps:

- uses: actions/checkout@v3

- uses: elastic/apm-pipeline-library/.github/actions/updatecli@current
Expand All @@ -24,3 +23,10 @@ jobs:
vaultRoleId: ${{ secrets.VAULT_ROLE_ID }}
vaultSecretId: ${{ secrets.VAULT_SECRET_ID }}
pipeline: ./.ci/bump-golang.yml

- uses: elastic/apm-pipeline-library/.github/actions/updatecli@current
with:
vaultUrl: ${{ secrets.VAULT_ADDR }}
vaultRoleId: ${{ secrets.VAULT_ROLE_ID }}
vaultSecretId: ${{ secrets.VAULT_SECRET_ID }}
pipeline: ./.ci/bump-golang-7.17.yml
16 changes: 16 additions & 0 deletions .golangci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -62,6 +62,7 @@ linters:
- unconvert # Remove unnecessary type conversions
- wastedassign # wastedassign finds wasted assignment statements.
- gomodguard # check for blocked dependencies
- prealloc # Finds slice declarations that could potentially be pre-allocated

# all available settings of specific linters
linters-settings:
Expand Down Expand Up @@ -147,3 +148,18 @@ linters-settings:
- G401 # Detect the usage of DES, RC4, MD5 or SHA1: Used in non-crypto contexts.
- G501 # Import blocklist: crypto/md5: Used in non-crypto contexts.
- G505 # Import blocklist: crypto/sha1: Used in non-crypto contexts.

prealloc:
# IMPORTANT: we don't recommend using this linter before doing performance profiling.
# For most programs usage of prealloc will be a premature optimization.

# Report pre-allocation suggestions only on simple loops that have no returns/breaks/continues/gotos in them.
# Default: true
simple: false
# Report pre-allocation suggestions on range loops.
# Default: true
range-loops: true
# Report pre-allocation suggestions on for loops.
# Default: false
for-loops: true

48 changes: 48 additions & 0 deletions CHANGELOG.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,54 @@
:issue: https://github.com/elastic/beats/issues/
:pull: https://github.com/elastic/beats/pull/

[[release-notes-8.8.2]]
=== Beats version 8.8.2
https://github.com/elastic/beats/compare/v8.8.1\...v8.8.2[View commits]

==== Bugfixes

*Affecting all Beats*

- Make sure k8s watchers are closed when closing k8s meta processor. {pull}35630[35630]
- Upgraded apache arrow library used in `x-pack/libbeat/reader/parquet` from v11 to v12.0.1 in order to fix cross-compilation issues. {pull}35640[35640]
- Fix panic when the disk queue's MaxRetryInterval configuration is specified, but RetryInterval is not. {pull}35820[35820]

*Filebeat*

- Fix syslog message parsing for fortinet.firewall to take into account quoted values. {pull}35522[35522]
- [Filebeat GCS input] Fixed an issue where `bucket_timeout` was being applied to the entire bucket poll interval and not individual bucket object read operations. Fixed a map write concurrency issue arising from data races when using a high number of workers. Fixed the flaky tests that were present in the GCS test suit. {pull}35605[35605]
- Fix handling of IPv6 unspecified addresses in TCP input. {issue}35064[35064] {pull}35637[35637]
- Improve error reporting and fix IPv6 handling of TCP and UDP metric collection. {pull}35772[35772]
- Fix input reload on autodiscover. {issue}34388[34388] {pull}35645[35645]

*Heartbeat*

- Fix serialization of processors when running diagnostics. {pull}35698[35698]

*Metricbeat*

- Fix calculation of the `host.cpu.usage` metric for EC2. {pull}35717[35717]

==== Added

*Affecting all Beats*

- Update Go version to 1.19.10. {pull}35751[35751]

*Filebeat*

- [GCS] Added scheduler debug logs and improved the context passing mechanism by removing them from struct params and passing them as function arguments. {pull}35674[35674]

*Packetbeat*

- Add AF_PACKET metrics. {issue}35428[35428] {pull}35489[35489]

==== Deprecated

*Heartbeat*

- Removed support for `zip_url` and `local` browser sources. {pull}35429[35429]

[[release-notes-8.8.1]]
=== Beats version 8.8.1
https://github.com/elastic/beats/compare/v8.8.0\...v8.8.1[View commits]
Expand Down
31 changes: 29 additions & 2 deletions CHANGELOG.next.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -12,11 +12,11 @@ https://github.com/elastic/beats/compare/v8.8.1\...main[Check the HEAD diff]
- Update Go version to 1.19.10 {pull}35751[35751]
- Fix status reporting to Elastic-Agent when output configuration is invalid running under Elastic-Agent {pull}35719[35719]


*Auditbeat*


*Filebeat*
- Fix input reload on autodiscover {issue}34388[34388] {pull}35645[35645]


*Heartbeat*
Expand Down Expand Up @@ -70,6 +70,8 @@ https://github.com/elastic/beats/compare/v8.8.1\...main[Check the HEAD diff]
- Make sure k8s watchers are closed when closing k8s meta processor. {pull}35630[35630]
- Upgraded apache arrow library used in x-pack/libbeat/reader/parquet from v11 to v12.0.1 in order to fix cross-compilation issues {pull}35640[35640]
- Fix panic when MaxRetryInterval is specified, but RetryInterval is not {pull}35820[35820]
- Do not print context cancelled error message when running under agent {pull}36006[36006]
- Fix recovering from invalid output configuration when running under Elastic-Agent {pull}36016[36016]
- Improve StreamBuf append {pull}35928[35928]

*Auditbeat*
Expand Down Expand Up @@ -141,6 +143,10 @@ https://github.com/elastic/beats/compare/v8.8.1\...main[Check the HEAD diff]
- Fix CEL input JSON marshalling of nested objects. {issue}35763[35763] {pull}35774[35774]
- Fix metric collection in GCPPubSub input. {pull}35773[35773]
- Fix end point deregistration in http_endpoint input. {issue}35899[35899] {pull}35903[35903]
- Fix duplicate ID panic in filestream metrics. {issue}35964[35964] {pull}35972[35972]
- Improve error reporting and fix IPv6 handling of TCP and UDP metric collection. {pull}35996[35996]
- Fix handling of NUL-terminated log lines in Fortinet Firewall module. {issue}36026[36026] {pull}36027[36027]
- Make redact field configuration recommended in CEL input and log warning if missing. {pull}36008[36008]

*Heartbeat*

Expand All @@ -166,6 +172,10 @@ automatic splitting at root level, if root level element is an array. {pull}3415
- Fix serialization of processors when running diagnostics. {pull}35698[35698]
- Filter dev flags for ui monitors inside synthetics_args. {pull}35788[35788]
- Fix temp dir running out of space with project monitors. {issue}35843[35843]
- Fixing the grok expression outputs of log files {pull}35221[35221]

*Heartbeat*


*Heartbeat*

Expand Down Expand Up @@ -239,6 +249,10 @@ automatic splitting at root level, if root level element is an array. {pull}3415
*Functionbeat*


*Functionbeat*



*Elastic Logging Plugin*


Expand Down Expand Up @@ -325,9 +339,12 @@ automatic splitting at root level, if root level element is an array. {pull}3415
- Add metrics for gcp-pubsub input. {pull}35614[35614]
- [GCS] Added scheduler debug logs and improved the context passing mechanism by removing them from struct params and passing them as function arguments. {pull}35674[35674]
- Allow non-AWS endpoints for awss3 input. {issue}35496[35496] {pull}35520[35520]
- Under elastic-agent the input metrics will now be included in agent diagnostics dumps. {pull}35798[35798]
- Add Okta input package for entity analytics. {pull}35611[35611]
- Expose harvester metrics from filestream input {pull}35835[35835] {issue}33771[33771]
- Add device support for Azure AD entity analytics. {pull}35807[35807]
- Improve CEL input performance. {pull}35915[35915]
- Adding filename details from zip to response for httpjson {issue}33952[33952] {pull}34044[34044]

*Auditbeat*
- Migration of system/package module storage from gob encoding to flatbuffer encoding in bolt db. {pull}34817[34817]
Expand Down Expand Up @@ -359,6 +376,8 @@ automatic splitting at root level, if root level element is an array. {pull}3415
- Add new parameter `include_linked_accounts` to enable/disable metrics collection from multiple linked AWS Accounts {pull}35648[35648]
- Migrate Azure Billing, Monitor, and Storage metricsets to the newer SDK. {pull}33585[33585]
- Add support for float64 values parsing for statsd metrics of counter type. {pull}35099[35099]
- Add kubernetes.deployment.status.* fields for Kubernetes module {pull}35999[35999]


*Osquerybeat*

Expand All @@ -367,6 +386,11 @@ automatic splitting at root level, if root level element is an array. {pull}3415

- Added `packetbeat.interfaces.fanout_group` to allow a Packetbeat sniffer to join an AF_PACKET fanout group. {issue}35451[35451] {pull}35453[35453]
- Add AF_PACKET metrics. {issue}35428[35428] {pull}35489[35489]
- Under elastic-agent the input metrics will now be included in agent diagnostics dumps. {pull}35798[35798]
- Add support for multiple regions in GCP {pull}32964[32964]

*Packetbeat*


*Winlogbeat*

Expand All @@ -378,6 +402,7 @@ automatic splitting at root level, if root level element is an array. {pull}3415

- Set `host.os.type` and `host.os.family` to "windows" if not already set. {pull}35435[35435]
- Handle empty DNS answer data in QueryResults for the Sysmon Pipeline {pull}35207[35207]
- Under elastic-agent the input metrics will now be included in agent diagnostics dumps. {pull}35798[35798]


*Elastic Log Driver*
Expand All @@ -394,7 +419,6 @@ automatic splitting at root level, if root level element is an array. {pull}3415

*Heartbeat*

- Removed zip_url and local browser sources. {pull}35429[35429]

*Metricbeat*

Expand All @@ -415,3 +439,6 @@ automatic splitting at root level, if root level element is an array. {pull}3415


==== Known Issues



Loading

0 comments on commit 2cd8ca0

Please sign in to comment.