Global Variables, Decl Tags, and package pin
#1632
ti-mo
announced in
Announcements
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
As we close the door on the year 2024, we're also wrapping up the ebpf-go v0.17 development cycle. This release represents a large body of work and is our largest feature release since BTF.
Users are, as always, highly encouraged to upgrade, and we're looking forward to hearing your feedback on the new Global Variables API. We've been putting this interface trough its paces internally, along with a few eager community members, and we're ready to let the wider community make use of it.
This release also marks the end of our Go 1.21 support; Go 1.22 is required going forward.
Breaking Changes
Before we get to the fun stuff, we need to call out a few breaking changes:
Global variables with a
staticqualifier can no longer be rewritten usingCollectionSpec.RewriteConstants()! See the section on global variables under major features below.program: remove LogSize and VerifierError.Truncated by @lmb in program: remove LogSize and VerifierError.Truncated #1526
prog: add ProgramOptions.LogSizeStart to obtain full log after verifier bug by @ti-mo in prog: add ProgramOptions.LogSizeStart to obtain full log after verifier bug #1630
The
ProgramOptions.LogSizefield was deprecated and a no-op since 0.16 and has now been removed completely. In its place, a new field was added to obtain full verifier logs when the verifier hits an internal bug. TheProgramOptions.LogSizeStartfield controls the starting size of the log buffer. Use this if you have a good understanding of the log size you're expecting, or if you're trying to pull out the full log when the verifier hits an internal bug. The error string now also contains some hints about what happened and how to handle this situation.map: remove MapSpec.Freeze field by @ti-mo in map: remove MapSpec.Freeze field #1558
elf_reader: don't use BPF_F_RDONLY_PROG flag to trigger map freezing by @ti-mo in elf_reader: don't use BPF_F_RDONLY_PROG flag to trigger map freezing #1568
The Freeze field was ambiguous from the start, and has been a source of subtle bugs over the years. Automatic map freezing is now done based solely on map name (.(ro)data prefix). If you were manually setting this flag, consider using a conventional map name instead.
info: expose ksym info and func info by ProgramInfo by @Asphaltt in info: expose ksym info and func info by ProgramInfo #1576
info: expose more prog jited info by @Asphaltt in info: expose more prog jited info #1598
Some ext_info types in package btf were redefined to enable pulling raw func and line infos out of
Program.Info(). These were all types without methods and all unexported fields, but calling them out regardless in case someone's doingunintendedadvanced things with BTF.kallsyms: change Modules caching strategy, cache address lookups by @ti-mo in kallsyms: change Modules caching strategy, cache address lookups #1590
Users attaching bpf programs to kernel symbols should see a marked decrease in allocations made in the library during program creation in the general case. Only used entries are now cached, making kallsyms lookups on subsequent program loads free. In the pathological case (new kernel symbols on every load), scanning is repeated, resulting in more CPU time being used instead of holding on to all kallsyms symbols indefinitely.
ProgramSpec.KernelModule()was unexported until further notice.Also, bpf2go users should now be able to generate Go bindings as unprivileged users once again. Oops!
Major Features
Easy global variables: introducing
ebpf.VariableSpecandebpf.VariableThis has been a frequent ask in the community for years, so we finally bit the bullet and committed to an API to interact with global bpf variables through
CollectionSpec.VariablesandCollectionSpec.Variables. We've published a small guide over on ebpf-go.dev/concepts/global-variables that details how to use them, so we're not going to reiterate here.See the documentation on the use of the
staticqualifier! These are no longer considered global variables, and can't be modified at runtime.bpf2go now also generates objects to be used with
CollectionSpec.Assignand.LoadAndAssign(), and also emits Go type declarations for C types used as bpf C variables. Our test coverage was somewhat lacking for the latter, please drop something on the issue tracker if you're noticing surprising bpf2go output.Note that it's currently not possible to manually create a
VariableSpecand wire it up to aCollectionSpec, so if you were previously relying on e.g. manually assembling aCollectionSpecand usingRewriteConstants()in tests, this will no longer work. This may land as the API crystallizes and we settle on a good mechanism to enable this. Please reach out on the Cilium Slack if you'd like to see this happen.Tags!
As you'll notice, the
btf.Struct,btf.Union,btf.Member,btf.Typedef,btf.Funcandbtf.Varhave gained a new field:Tags! Set one or more tags on these objects from bpf C using__attribute__((btf_decl_tag("mytag")))andyou'll find
Tagsbeing populated in Go.There's also the
btf.Func.ParamTagsfield that holds tags declared on individual function parameters. These are part of theFuncsince they appear only in the function prototype, not in the parameter type itself, since those can appear in many different types. (it does get confusing!)Also, an honorable mention to
btf.TypeTag, which has now been exported. This allows tagging pointers with an arbitrary string value and is mainly used by the Linux verifier to tag kernel pointers, see the__kptrand__kptr_refmacros in Linux'bpf_helpers.h.Pins!
We've added a new root-level package
pin, which currently features theLoad()andWalkDir()functions. Since allLoadPinned*functions in the library now check for the underlying object's type, we've repurposed the machinery to allow loading an opaque pin and returning a Go object of the right type.WalkDirdoes what it says on the tin, it walks bpffs directories and invokes a callback that received the loaded object. Super convenient!Minor Features
We've also added a handful of minor features during this release:
bpf_prog_infofields inProgramInfoby @tyrone-wu in info: expose additionalbpf_prog_infofields inProgramInfo#1512Other Changes
Last but not least, there's bugfixes, CI changes and some internal refactoring that happened to eventually make ebpf-go work with ebpf-for-windows. See individual PRs for more context. Stay tuned!
objNameAllowsDot, return *btf.Var fromVariable{Spec}.Type()by @ti-mo in UnprivilegedobjNameAllowsDot, return *btf.Var fromVariable{Spec}.Type()#1612epoll.Waitby using sentinel errors by @def in Optimizeepoll.Waitby using sentinel errors #1619New Contributors
bpf_prog_infofields inProgramInfo#1512epoll.Waitby using sentinel errors #1619Full Changelog: v0.16.0...v0.17.0
This discussion was created from the release Global Variables, Decl Tags, and package `pin`.
Beta Was this translation helpful? Give feedback.
All reactions