Cuegen is a tool to build kubernetes resources with CUE. It solves the problems of composing and versioning of "charts", as well as easily importing external data (scripts, certificates, keys, etc.) into CUE charts. For this purpose, it extends the rich possibilities that CUE already provides with the ability to "compose" charts from various sources and load resources, controlled by attributes.
If CUE and creating k8s resources with CUE is new to you, the CUE homepage, the k8s tutorial, as well as examples in this repository (in that order) are good starting points.
- Compose manifests from various, versioned sources (local directories, git-repositories)
- Load file contents into CUE values, e.g. a script into a ConfigMap key
- Load whole directories as key/values data at once, e.g. into a ConfigMap
- Load structured data (JSON/YAML/env) into a CUE struct
- Automatically decrypt SOPS-encrypted data
- Load remote charts
Download the latest release or build with go1.20rc3 or later:
go install github.com/noris-network/cuegen@latest
cuegen path/to/cuegen.yaml
# or
cuegen path/to/directory-containing-cuegen-dot-yaml
# or
cuegen https://git.example.com/deployments/myapp.git
Have a look at the examples for some ready-to-run examples.
Cuegen can be used as config management plugin in ArgoCD. A container
with the plugin.yaml configuration
is available at docker hub. The cuegen-cmp plugin container
expects the cuegen config file to be named cuegen.yaml.
A configuration file (preferred name: cuegen.cue, schema) is
required to run cuegen. For backwards compatibility the yaml format will still
be supported in the future.
cuegen: {
objectsPath: "objects" // this will be dumped to YAML
secretDataPath: "secret.*.data" // values matching this path will be loaded as []byte and
// automatically be base64 encoded when dumped to YAML
checkPath: // this path is checked to contain only concrete values
checkPaths: [ // when more than one path needs to be checked, the
"values", // 'objectsPath' with always be checked.
"global",
]
components: [ // merge cue files from these soures into the main entrypoint
"https://$GITLAB_TOKEN@gitlab.noris.net/mcs/components/cuegen/mongodb.git?ref=v6.0.4-mcs.1",
"https://$GITLAB_TOKEN@gitlab.noris.net/mcs/components/cuegen/wekan.git?ref=v6.71-mcs.0",
]
debug: false // print some info useful for debugging
}
Some environment variables can help working with cuegen:
CUEGEN_DEBUG turn on debug output with "true"
CUEGEN_HTTP_PASSWORD password for git authentication
CUEGEN_HTTP_USERNAME username for git authentication
DUMP_OVERLAYS_TO directory to dump overlays to (debug only)
SOPS_AGE_KEY age key for decryption
SOPS_AGE_KEY_FILE age key file for decryption
YQ_PRETTYPRINT !="": run yaml output thru `yq -P`
starting with `/`: use as path to yq
Components can be
- directories outside the main chart
- zip files
- git repositories
Local paths are not restricted. As this could be a security problem, this will change in a future release.
Environment variables in components are expanded, that can be used to
e.g. add authentication to git urls or prefix local paths. For cloning private
repositories via Http url, CUEGEN_HTTP_USERNAME and CUEGEN_HTTP_PASSWORD can
also be set in the environment.
Components Example:
cuegen: components: [
"../database-chart",
"../common-static-resources.zip",
"https://github.com/nxcc/cuegen-test-chart-one?ref=v0.1.0",
"https://github.com/noris-network/cuegen?ref=v0.2.1#examples/configmap",
]
Because of the way CUE and cuegen work, all *.cue files need to be in the root
of the components directory.
No special files need to be present for a chart, and although cuegen does not
require or evaluate them, it is recommended to add metadata (see examples).
For git repositories, a branch or tag name can be specified with the ref
parameter.
If a zip file has only one directory in it's root, this level is skipped and
cuegen uses all files in that directory.
The directory containing cuegen.yaml is the base directory, all paths are relative
to this.
Attribute @readfile loads contents of given file(s) into a CUE value. An optional
suffix controlls whitespace (=nl: ensure \n at end, =trim: trim whitespace
from begin and end).
Load version.txt into Version, trim any whitespace from begin and end:
Version: string @readfile(version.txt=trim)
Load my.crt and chain.crt concatenated into Certificate. A \n is ensured
to be placed after my.crt:
Certificate: string @readfile(my.crt=nl, chain.crt)
Attribute @readmap loads contents of given file(s) into a CUE value as key/value
data. When the cue path matches secretDataPath, values are inserted as []byte.
This will automatically base64 encode them when dumped to YAML. This is useful
for Secrets. This can also be forced by an =bytes suffix.
Load values from data.json as []bytes:
secret: foo: {
data: {} @readmap(data.json=bytes)
}
Attribute @read(filename) tries to load structured data (JSON/YAML/env) into a
CUE struct.
Load key/value data from env.env and data.yaml into configMap.myconfig.data:
configMap: "myconfig": {
data: {} @read(env.env, data.yaml)
}
Attribute @read(directory) tries to load regular files from the given directory
as key/values into a CUE struct.
Load all files from directory scripts as key/values into configMap.scripts.data:
configMap: "scripts": {
data: {} @read(scripts)
}
Until issue 2555 is resolved in CUE, there is a temporary workaround.
Cuegen 0.15.0 is based on cue v0.10.0. When apiVersion is set to "v1alpha4", cue attribute handling
is removed in favour for cue native embedding. Right now sops encrypted files need to be
named like <filename>.sops.<ext> for formats supported by sops, otherwise like <filename>.<ext>.sops.
They will be temporarily decrypted to files named like <filename>.<ext>, and removed again after Yaml
output was generated (usage example).
All future releases at least until release 1.0.0 will be backwards compatible to current
non-experimental cuegen behaviour.
v0.1.0- Initial releasev0.1.1- Improved attribute lookupv0.2.0- Added checks when readingcuegen.yamlv0.2.1- Improved error messagesv0.3.0- Added ability to read subpaths from git reposv0.3.1- No code changes, trigger cmp buildv0.3.2- No code changes, bump go version to 1.20v0.4.0- switch default config tocuegen.cue, use cue v0.5.0-beta5v0.4.1- Make components & checkPaths optionalv0.4.2- downgrade cue to v0.5.0-beta.2 (performance regression)v0.4.3- fix running as kustomize pluginv0.4.4- improve handling of git urlsv0.5.0- add cuegen default configv0.6.0- add dumpOverlays optionv0.7.0- upgrade cue to v0.5.0 (many fixes, rare performance regression still present)v0.7.1- fix secret handling of @readfilev0.7.2- internal cleanupv0.8.0- allow remote cuegen directories, rm kustomize plugin supportv0.9.0- upgrade cue to v0.6.0v0.10.0- addYQ_PRETTYPRINTto filter output thruyq -Pv0.11.0- addDUMP_OVERLAYS_TOto dump overlays to directory (debug)v0.11.1- upgrade sopsv0.12.0- add workaround for file order bugv0.13.0- No code changes, bump cue version to 0.7.0v0.13.1- bump depsv0.14.x- "silent release" of v1alpha2 schema...v0.14.4- No code changes, bump cue version to 0.8.0v0.14.5- No code changes, bump cue version to 0.8.1v0.14.6- No code changes, bump cue version to 0.8.2v0.15.0- Introduce "embed experiment'