Bug fixes and various new features

This release fixes some bugs, improves some errors and adds API. This version also drops support for some old constructs that have not been supported in the CUE tool for some time now. The main reason to start being more “aggressive” about this is to avoid surprises as we’re inching towards a full backwards compatibility guarantee. Note that “aggressive” is probably a strong word, these constructs have not been supported for over a year.

Language additions

Value aliases

CUE now allows value aliases, as suggested in various proposals and remarks. This allows, for instance, users to rewrite:

foo: {
    a: b + c
    b: int
    c: int
}
bar: foo & {b: 1, c: 2}

as

foo: X={
    a: X.b + X.c
}
bar: foo & {b: 1, c: 2}

In this case the benefit may be small, but there are various cases where not having this ability results in significant inconvenience. Also the query proposal, the UX design around the usage of the proposed must builtin, as well as various other patterns, rely on this ability.

Note the subtle but important difference between this and using field aliases. Using a field alias here, say X=foo: a: X.b + X.c would be equivalent to foo: a: foo.b + foo.c. As a result, in bar: foo, X.b would still be bound to the original location and not resolve to any b specified in bar.

Which one to use depends on the application. As a general rule, field aliases should be used when defining “types”, whereas value aliases should be used when defining “values”. An example of where one wants to use field aliases:

List="my-list-type": {
    value: _
    next:  List
}

Here, using a value alias would result in a list where every value must be the same. This is probably the most subtle aspect of CUE. The guideline of using values aliases when defining values, and field aliases when defining types, however, should give users a good steer on the matter.

Tooling

Tag variables to inject contextual system values

The injection mechanism now allows injecting specific variables, such as current time, username, or hostname, or a random number, into fields marked with a @tag attribute. For instance:

import "path"

_os: string @tag(os,var=os)

path.Base(dir, _os)

allows for an OS-specific Base operation. Except for cue cmd, the var will only be injected when the -T is explicitly specified on the command line.

This approach 1) allows packages to remain hermetic, 2) allows using such functionality without having to resort to cue cmd, and 3) avoids introducing another injection point.

See cue help injection for more details.

Binary file type

The cue tool now supports binary files. These are very much like the supported “text” files, but translate the file contents to a CUE binary literal, rather than a text literal.

There are no default file extensions to recognize a binary file. The binary: file qualifier can be used to read a file as binary. See cue help filetypes for more info.

To make loading binary files in bulk more useful when importing, the import command now supports the --ext flag, allowing certain file types to be interpreted as a certain format. This also works for other types. See cue help import for more info.

--force/-f

This flag is now allowed for any command that outputs files.

API additions

Context.NewList

Uses can now construct a new list cue.Value with Context.NewList. This is defined on Context to allow the creation of empty lists, which still need to be associated with a Context.

Value.FillPath

FillPath now accepts list indices. The Go-to-CUE are now documented in more detail.

Fillpath now also accepts the selectors AnyString and AnyIndex, which allows setting pattern constraints. For instance, v.FillPath(cue.MakePath(cue.Str("a"), cue.AnyString, x), adds a: [string]: x to v.

It also allows setting optional fields. For instance, v.FillPath(cue.MakePath(cue.Str("b").Optional()), 1) will add b?: 1 to v, using the usual rules for unifying such fields.

Context.Encode

The documentation of this method has been significantly improved. This also fixes some bugs that were uncovered in the process.

These are now also two additional options: InferBuiltins and ImportPath. These were exposed to support the cue command in using the new API, but thus also allows other tool writers to use the same kind of functionality.

CUE package additions

crypto/hmac

The new package hmac implements the Keyed-Hash Message Authentication Code (HMAC) as defined in U.S. Federal Information Processing Standards Publication 198.

uuid

The new package uuid defines functionality for creating UUIDs as defined in RFC 4122.

Currently only Version 5 (SHA1) and Version 3 (MD5) are supported. The new tag variable injection mechanism that was introduced in this release, however, also opens up the possibility to include other versions, if needed.

Backwards incompatible changes

Dropped support for old-style aliases

Aliases of the style X = 4 (now let X = 4) have been deprecated since v0.1.4. However, by setting the version or in API use, these aliases would still work. Support for them has been fully removed. Down the line, this syntax can be reused as value aliases as well, relaxing the places where value aliases can be used.

For the time being, cue fix can still be used to rewrite old-style aliases to their let equivalent. There was a bug that missed the detection of some forms of the deprecated alias type. This has now been fixed.

Dropped support for old-style comprehensions

Support for the old-style comprehensions (e.g. [ x+1 for x in foo]) has now been fully removed. Use an older version of CUE to rewrite such old-style comprehensions using cue fmt or cue fix.

Removed ast.TemplateLabel type from API

This has not been parsed, supported, or used for ages. The type has now been removed from the API.

Value.Expr

Value.Expr now translates the or and and builtins as if they were native | or & constructs. This fixes a bug in encoding/openapi and is likely to prevent many issues. If one was relying on detecting the use of these builtins, however, this now will no longer work.

Value.Decode

This method was reimplemented to not use Value.MarshalJSON. This fixes many bugs, but may cause issues if users were relying on old behavior. The new implementation still strictly adheres to the JSON encoding spec when this makes sense, but deviates in some important aspects. For instance, numbers like 1 are now integers in CUE, rather than float. Also binary values are now handled correctly.

Bug fixes

• hidden references used on the command line now resolve to hidden fields in the package under evaluation
• equality of null value

Changelog

5b14995 ci: debug tip trigger failures
24ce27e ci: fix broken tip and new version triggers
a8369ae ci: fix tip and new version curl and env problems
c143a3a ci: fix tip triggers with proper quoting
746e02e ci: more debugging of tip triggers
6c6b4e7 ci: trigger unity and cuelang.org builds/equivalent on tip/new versions
97cac92 cmd/cue/cmd: do not silently ignore unknown filetypes
89abc18 cmd/cue/cmd: fix resolution of hidden values across tools
4f7caea cmd/cue/cmd: move tag flags handling to central location
b9e7d90 cmd/cue/cmd: support binary file type
b449c0f cmd/cue: support --force for all commands that allow output flags
aa70399 cue/ast: fully remove support for old-style comprehensions
b73ab0b cue/ast: remove support for TemplateLabel
1f1c3f6 cue/load: provide reason for exclusion of files
975ba50 cue/load: remove unused code
bcdf277 cue/load: support injection of system variables
c2a68a9 cue/parser: allow pattern constraints unconditionally
3e10918 cue/parser: better error messages for use of deprecated constructs
3ef90b3 cue/testdata, doc, pkg: fix some it's/its typos
f60c88a cue: Value.Expr: process or and and builtins semantically.
d9af603 cue: add InferBuiltins EncodeOption
b49cbbd cue: add NewList
1f618f0 cue: align import with top-level package for -e
66efc67 cue: allow setting list elements in FillPath
67c6b6f cue: better document Context.Encode and friends
9d044f0 cue: hoist attribute-related functionality to separate file
ed5cdf0 cue: more path cleanup
3347302 cue: move Decode to separate file
c365513 cue: prepare API implementation for structure sharing
0457356 cue: properly decode "any" labels
819cf95 cue: reimplement Decode to not use MarshalJSON
5481b41 cue: remove use of isDef
2e1ac0f cue: support optional fields for FillPath
ac7e992 cue: undeprecate Iterator.Label
3316905 doc/ref/spec.md: introduction of value aliases
a31dd01 doc/ref/spec.md: remove "dead code"
2198ac3 docs: add link to Code of Conduct to CONTRIBUTING guide
f5213be github: add bug report template link to repro advice
1ad66ab internal/core/adt: fix null values equality
3cdf845 internal/core/adt: use Closed for checking recursive closedness
5cd76bb internal/core/comple: drop support for old-style aliases
8823e2a internal/core: support field value aliases
6d67b48 pkg/crypto/hmac: add crypto/hmac support
2118921 pkg/crypto/hmac: fix package documentation
a7d3987 pkg/time: fix documentation bug in Unix
fd05bf4 pkg/uuid: implementation of hermetic UUID functions

v0.4.0-alpha.2

This release focuses on setting direction for the CUE API. This brings the overall API much closer to where we would like it to go. It also prepares the API for the query extension.

Note that this is deliberately marked as an alpha release to allow people to give feedback in the case we need to make some tweaks to the API.

A central piece in that are the cue.Path and cue.Selector types. Another key part is getting rid of the Instance type. This type proves to be unnecessary and not using it results in a considerably nicer UX.

The old functionality is marked as deprecated and will stay around until we provide a way to automatically rewrite them. We will just no longer support them and we use a trick mentioned in this tweet to hide most of the deprecated methods from the Go docs in the meantime (pending a "proper" fix in pkg.go.dev).

Other improvements include:

• another big round of tuning error messages, most notably adding more line information.
• a performance bug fix that caused significant slowdown when doing repeated calls of Fill or Unify on the same values.

API additions

The cue.Path model

The old API was designed before the existence of definitions. The fact that definitions live in a different namespace, broke the API at various levels, and patching this up resulted in a brittle and complicated API.

The cue.Path-centred API addresses these issues, resulting in a much smaller API surface while at the same time being less prone to error.

This new API also paves the way for the query extension.

Some deprecated types and methods:

The following types and methods all get replaced by LookupPath:

• FieldInfo
• Instance.Lookup
• Instance.LookupDef
• Instance.LookupField
• Value.Elem
• Value.FieldByName
• Value.Lookup
• Value.LookupDef
• Value.LookupField
• Value.Struct
• Value.Template
• Struct

Similarly, FillPath and ReferencePath replace (Value|Instance).Fill and Reference. The latter also is instrumental in getting rid of Instance. In the long run we will want to repurpose Value.Lookup and Value.Fill, but that will be a long way out.

The new Iterator.Selector method replaces:

• Iterator.Label
• Iterator.IsLabel
• Iterator.IsOptional
• Iterator.IsDefinition

Using Selector also makes it more explicit what kind of labels one has and is thus less error prone. The selectors can be used to construct paths for lookup methods without loss of information, making the API more precise.

Phasing out Instance

The Instance type was initially intended to allow enforcing certain constraints. The data model of the new evaluator allows doing so without the need for Instance.

Getting rid of Instance has some big benefits. Most notably, the need to link Value to Instance makes it very hard to avoid memory retention, even when a user believes a Value is no longer needed. This causes long-running CUE services to grow in memory usage over time.

Another issue is usability: aside for duplication of methods, it may be hard for a user to know when to use a Value or Instance.

Context to replace Runtime

There are a few reasons for this change. As part of removing Instance, we needed a new way to build Values. The Context type now defines a new set of Value constructors.

Another reason was to have a cleaner way to break the dependency cycle that existed for linking in the core builtins. Previously, the user had to add a import _ "cuelang.org/go/pkg" somewhere in the code, or use a package that already did so, to make the builtins available. Now the user is expected to create a Context using cuecontext.New(), which takes care of this.

Finally, the name Runtime was confusing to some. The Context maintains indices and tables that are shared between creations of values, but there is no inherent “running” state.

The Context now also makes it easier to resolve identifiers in another Value’s scope or to directly encode Go values and types.

Value.Allows

This method now enables querying whether a Value would support a field of a specific type if it were to be added. This also uses the Selector type to specify the kind of field.

This replaces IsClosed.

Backwards incompatible changes

The new APIs are just additions. In many cases, the old API has been implemented in terms of the new API, but should still function as usual. This did result in some bug fixes, however, so one may observe changes.

Value.Format

The one change that may cause backwards incompatibility is the standard fmt.Formatter implementation of Value, which now has a more principled implementation. The standard %v formatter now prints it as a value, but allows incomplete values. The %+v formatter, as before, prints the fully evaluated value and is similar to cue export. The %#v formatter, which previously printed an esoteric debug format, now prints an equivalent of cue def.

Many of the standard Go formatting verbs will now be interpreted as such if the Value is of a compatible Go type. See the documentation of Value.Format for more details.

There have been various bug fixes in the exporter code as part of this change.

cue/encoding

This package has been removed. It really didn’t do anything except from being a distraction. In the off chance that anybody was using this package, just deleting that code would probably solve it.

Changelog

9e34a41 cue/ast/astutil: export ImportPathName
50c137a cue/encoding: removed unused package.
dcb2a1f cue/errors: add Wrap
f044ad1 cue/format: expose indent option
6822433 cue: add Iterator.Selector
f14c9a4 cue: add Selector.PkgPath
c5c9125 cue: add test for filling empty path
17d4e16 cue: clean up Format
618c10c cue: deprecate Instance.(Doc|Fill)
790bed3 cue: get rid of NewRuntime
4937cb9 cue: hide deprecated methods from docs
421ead3 cue: introduce Context to replace Runtime
d76e2cc cue: remove MakeValue
908614e internal/core/adt: dedup errors
b8ce660 internal/core/adt: improve performance for repeated calls to Unify/Fill
276ce26 internal/core/adt: record more error positions
14ec6e2 internal/core/export: add real Final option
b5b0429 internal/core/export: bug fixes for exporting API-generated values
c62b750 internal/core/export: bug fixes in definitions
64ede63 internal/core/export: extract docs from root
b937727 internal/core/export: fix definition wrapping
c290772 internal/value: implement interface type that is both value and instance
1f78a8d internal: replace untyped functions with typed ones
d5ff672 pkg: clean up builtin errors

Prototext and JSON PB support

This release introduces a more comprehensive implementation of protobuf, supporting textproto and JSON protobuf mappings, and paving the way for binary protobuf support as well.

This release required some significant changes to the cue command logic.

A hallmark property of protobuf data formats is that they cannot be parsed without a schema (unlike JSON, for instance). This is true to various degrees for the different formats, but it holds true in some shape or form for all of these. This required some changes in the CUE tooling to make this possible. It also required some backwards incompatible changes.

Protobuf changes

Filetype *.textproto

CUE now supports interpreting and writing text proto files. Text proto files cannot be interpreted, or even parsed, without a schema. This means that .textproto files can only be read with an explicit schema using the --schema/-d flag. Moreover, the schema must have @protobuf for certain types, such as maps, to be interpreted properly.

Note that the available .textproto parsing libraries are incredibly buggy. So there will be some rough edges that are kind out of CUE’s hands.

JSON conversion: package jsonpb and json+pb

The Protobuf documentation has a recommendation on how Proto messages should map to JSON. Package jsonpb now supports this mapping in both directions. It implements this by rewriting a CUE AST based on a given schema, a cue.Value. This allows this mapping to also be combined in conjunction with Yaml, for instance.

On the command line this feature can be used through the pb “interpretation”, for instance as json+pb or yaml+pb. Both input and output are supported.

Note that interpretations to CUE require a schema for the interpretation. This can be an explicitly specified schema using the --schema/-d flag or an implicit only by unifying it with a schema value or playing it within a schema using the placement flags (see cue help flags).

Backwards incompatible changes

@protobuf tag

The protobuf previously had only one required argument: the numeric value of the enum. The type was optional and only included if different in name from the CUE type. As it turned out, though, the CUE type was not always sufficient information to be able to represent proto values. Most notably, integer values are encoded differently in JSON depending on the type of integer in the proto specification (this is not a typo!!).

The new format includes the type unconditionally as the second argument. CUE does its best to recognize old formats for backwards compatibility purposes, but this may cause issues.

@protobuf(<tag num>,<type>, ...options)

Protobuf options are still represented as string literals, allowing CUE options, such as alternative names (name=x) to be represented as the usual <key>=<value> format.

Another change is the representation for map types. Previously, the Protobuf map type was included verbatim (map<T,U>). This was somewhat inconvenient for parsing attributes, though: angular brackets are, unlike (), [], and {} not matched. So the comma in the map type commanded some escaping. To avoid this, maps are now represented as map[T]U. We contemplated using [T]:U, but opted for the map prefix for clarity and future extendibility.

JSON mappings

An initial design decision for the Proto to CUE mapping was to have CUE follow the Proto to JSON mapping. By hindsight, though, this did not make much sense. The inconsistency of having integers represented as strings does not make sense for a language that supports expressions on such integers (unless we want to give up typing, perhaps).

The stance now is to take the representation that makes sense, and have a protobuf-specific CUE to/from JSON converters. This is akin to the JSON schema and OpenAPI converters, which map CUE to some data format, using JSON or YAML, for instance, as a transport layer.

Luckily, the Protobuf to CUE mapping already deviated from the recommended mapping, always defaulting to int for integer types. So there are no changes there, other than that there is now support for following the recommended mappings for import and export.

Enum mappings

The most noteworthy backwards incompatible change is how enum types are mapped. Previously, CUE followed the recommended JSON mapping of using strings. This turned out to be a bad idea for various reasons. Firstly, the source of truth are integer values, really, not the names. There may be multiple names per number, making comparing names somewhat meaningless. Finally, most other languages used integers as representations, making the CUE representation not interoperate well with such languages.

Although the old mapping is still supported, the integer-based mapping is now recommended.

The default enum representation when using cue import proto is now to represent enums as integers in CUE. The --proto_enum=json flag can be used to invoke the old conversion behavior.

The API will keep converting using the JSON format, but now has a Config.EnumMode option for selecting the alternative behavior.

The Protobuf to JSON interpretation (filetype json+pb or package jsonpb) supports converting back and forth in either format.

Changelog

aaf6e84 cmd/cue/cmd: compute schema before values
0b5084a cmd/cue/cmd: hook up protobuf encodings types
8e5eeab cmd/cue/cmd: parseArgs: split values and schemas early
f228236 cmd/cue/cmd: preparse command line path expressions
75d0180 cmd/cue/cmd: simplify parseArgs in preparation of proto support
dea3c5d cue/build: organized Instance fields
660b090 cue/build: remove support for file lists
38ad7c3 cue: add ReferencePath
af509c6 cue: eliminate context type
52db572 cue: get rid of internal index type
c0fe9ce cue: refactor index
362e3a5 encoding/protobuf/jsonpb: add encoder
4a288d5 encoding/protobuf/textproto: add decoder implementation
a0035de encoding/protobuf/textproto: add encoder
3bdfa5d encoding/protobuf: always include type as second argument
dcfff00 encoding/protobuf: support integer enums
8ba98ee internal/encoding: pass schema to config
80a0a6e internal: move DebugStr to new astinternal package
22abdad internal: replace internal.CoreValue with more type-safe variant

A couple of API bug fixes

Not a big release, just a couple of API bug fixes.

Changelog

c8b7fe0 internal/core/export: export floats as float literals
f1d3d81 internal/core/export: fix decl export printing and extraction

Bug fixes and API work

This release contains several bug fixes, API additions, some API deprecations, and a language addition. One of the bug fixes may lead to unexpected behavior (see below).

Backwards Incompatible Bug Fix

One common use case of cmd/cue involves combining CUE and non-CUE files. Consider the following example:

// check.cue
package check

map: {for n in nodes {"\(n.name)": n}}

# data.yaml
nodes:
- name: bar
  parent: foo

- name: baz
  parent: foo

With v0.3.0 and earlier, cue eval would happily combine these two files:

$ cue eval data.yaml check.cue
map: {
    bar: {
        name:   "bar"
        parent: "foo"
    }
    baz: {
        name:   "baz"
        parent: "foo"
    }
}
nodes: [{
    name:   "bar"
    parent: "foo"
}, {
    name:   "baz"
    parent: "foo"
}]

But cue vet would complain:

$ cue vet data.yaml check.cue
map: reference "nodes" not found:
    ./check.cue:3:16

cue vet is actually correct here. Identifier resolution should only happen across files that belong to the same package: non-CUE files are equivalent to a CUE file without a package clause or an anonymous package clause (package _), hence cue eval should really fail in this instance.

v0.3.1 makes cue eval and other commands consistent with cue vet. This was deemed a sufficiently breaking change to warrant its own release.

The fix in this case is to define nodes in the scope of package check:

// check.cue

package check

nodes: _
map: {for n in nodes {"\(n.name)": n}}

at which point cue eval succeeds as before.

An upcoming change will also ensure this condition of identifier resolution is satisfied for all non-package CUE file arguments.

API changes

The cue.Selector model has been extended to allow querying optional fields and arbitrary elements.

Deprecations

The following methods of Value have now been deprecated:
FieldByName
Elem
Template

Really, you should only need LookupPath for your lookup needs. For the most part, these old methods have been reimplemented using the new API.

Also, updated the doc that one really, really should not use Merge.

If you are a user of the API we strongly recommend running staticcheck: it will raise errors where you are using deprecated parts of the API.

Language Additions

CUE now allows parenthesized expressions as labels. This is not (yet) published in the spec, but part of the query proposal. So "\(x)": foo, where x is a string, may now be written as (x): foo.

Changelog

a8ae7d1 cmd/cue/cmd: allow "exec" in commands for TestX
460357b cue/ast: allow parentheses as labels
e70a1db cue/build: remove unused Scope field
f0adb4e cue: deprecate Value.Elem and Value.Template
f063a61 cue: deprecate Value.FieldByName
6a1ae9c cue: move LookupPath to query
e440183 cue: remove error type aliases
4459434 cue: resolve identifiers in an ast.Expr for FillPath
c505c19 cue: separate closedness for lists and structs
957003c cue: support optional field lookup in LookupPath
ad4d1a1 cue: update Merge comment to REALLY NOT USE IT
5c2b281 internal/core: don't resolve values into anonymous packages
72e8fb4 internal/diff: fix printing and top-level diffs

First large step towards backwards compatibility guarantee

After a long road, we have now an official v0.3.0 release. Aside from implementing some important changes to the language (see release notes), it prepares CUE for exciting new features such as the query and policy extensions and other language extensions as well as much better performance.

The focus will now be to get to language stability and a v1.0.0 release.

With this release will also come a different approach to versioning leading up to the v1.0.0 release. The goal is to get to the stability guarantee for the language first, then the API and tooling. Up to v1.0.0, version increments will have the following meanings:

• minor (_.x._)
  • backwards incompatible changes
  • bug fixes with notable backwards incompatible changes
  • large new features
• patch (_._.x)
  • bug and spec conformance fixes
  • new features

We plan to get the few small remaining backwards incompatible languages chances out as soon as possible. To make the transition as simple as possible, we aim to put possible impactful bug fixes in small releases, so that users get a chance to get used to them. And, of course, we plan to make any necessary transitions as smooth as possible using cue fix aids when possible.

Note that to smooth out any transition, users can register their repos at https://github.com/cue-sh/unity. This is used to ensure that their CUE evaluations will not unexpectedly stop working.

API

encoding/protobuf/jsonpb

Protobuf defines its own, somewhat peculiar, mapping to JSON. This package provides a way to interpret JSON for such mappings, given a CUE schema. This is a first step towards broader Protobuf value support.

cue.Selector

Several new features have been added to Selector, including the possibility to query whether a selector is a regular string field and the ability to create a Selector form an ast.Label.

Value.Attributes

Value has been extended with a more extensive API to retrieve attributes.

Value.Subsume

Now allows raw (CUE-native) subsumption, instead of just the "Schema" or "Data" interpretations.

Value.FillPath

For completeness, note that beta.8 already introduced FillPath.

Changelog

4476060 cmd/cue/cmd: load import dependencies of tool files
276e164 cue/literal: expose some internal data
c24a281 cue/load: drop out-of-date comment about type of value in Overlay map
792da39 cue: add function for Value.Attributes()
1ea47e0 cue: allow raw subsumption, taking defaults into account
20a4878 cue: expose some path utilities
48f2a22 doc/cmd: refer people to cue help for now
aa99414 doc: fix various typos
2115955 encoding/openapi: correctly extract type in the presence of null
a1903ca encoding/openapi: dedup conjuncts and disjuncts
3701bef encoding/protobuf/jsonpb: add Rewrite* for interpreting JSON in PB terms
96e84eb tools/flow/testdata: add test with package dependency

v0.3.0-beta.8 the real beta.7

This release really is what beta.7 should have been as we actually forgot to include what was indicated as the most important change in beta.7 🤦‍♂️.

In the meantime there are a few more fixes. If nothing serious comes in, we’ll cut v0.3.0 in about a week!

Changelog

2e934c0 ci: do not unset Code-Review label
b00c91f cmd/cue/cmd: disallow commands in non-tool files
27d305c cmd: fix -h handling
f9164c6 cue/ast: sort CommentGroups by position when adding
7ca3968 cue: add FillPath
4ca1a5d cue: allow hidden fields in Path
ccca558 cue: allow spaces around attribute args
efd22f6 cue: allow string value as first element in ParsePath
a1551b0 doc/ref: fix HTML anchor link in spec
6195171 doc: fix running Kubernetes test with CUE_UPDATE=1
cbda0d3 internal/core/adt: fix omission of closedness check
9b263eb internal/core/adt: track more incomplete errors
a6e1627 internal/core/adt: update comments for Subsume
e3e11e3 internal: store Body in Attr type
b1730b6 tools/trim: optional fields should not remove non-optional

Bug fixes, de facto release candidate

This release contains several bug fixes.

The most important bug fix, perhaps, is that it fixes a long-standing regression where commands for cue cmd were allowed to be specified in non-tool files. The current implementation will require a command to be define in at least one *_tool.cue file.

Performance fixes have been put on hold for the moment to allow some stabilization before the final v0.3.0 release.

Changelog

35e19b2 cmd/cue: fix bug in -o, which would not write files for some commands
7c5d28e cmd/cue: fix race in testscript tests that use go/packages
81aa8cd cmd/export: fix help
d4cb2c8 cue: add locations for disallowed field
eb7b60d doc/ref/spec.md: fix ListLit production
fec7a9c encoding/openapi: fix title handling regression
bbe68e1 internal/core/adt: allow pattern constraints alongside fields
5049ab7 internal/core/adt: fix nil interface panic
1f9300d internal/core/adt: handle error properly for failed pattern constraint
2a937a9 internal/core/adt: improve error message
dd5083f tools/flow: add clarifications in the API comments

More bug fixes

This release has some important bug fixes, fixing some regressions in disjunctions and trim, mostly, among some other bug fixes.

Some of the regressions in this release slipped in because a swat of tests was accidentally disabled in the move to the new evaluator. All former tests are now reenabled and a few more corresponding issues have been resolved.

There are some performance issues remaining. We are considering leaving those in and do a v0.3.0 release, addressing these in a subsequent release. In this case, we are getting fairly close to a v0.3.0.

There will still be a beta.7 release, which addresses a regression, allowing commands in non-tool files, that we would like to address in a separate release.

Changelog

91abe0d ci: fix goreleaser config
a113048 ci: move to using go1.16
f014c14 ci: set a "trybot" tag for trybot repository dispatch flow
5d3afa9 cmd/cue/cmd: correct documentation for -d flag
de1f80c cmd/cue/cmd: remove skip tests
304a3e0 cmd/cue/cmd: use unify semantics merging across packages
ae43812 cmd/cue: make get go operate on a best-efforts basis
cf5c48e cmd/cue: only ignore type and parse errors with get go
7326cfa cue: Value.Equal should not only consider values
830cfbc cue: remove build.Context reference
17ea1d5 doc/ref/spec.md: a few fixes
f5a19ef doc/ref/spec.md: fix escaping
3ca8680 doc/ref/spec.md: fix int_lit 0
e4d0d13 doc/ref/spec.md: remove faulty use of \x in double-quoted string
7df9fa4 doc/tutorial/basics: fix example
b51368e doc/tutorial/kubernetes: add required go mod init step
430c3dd encoding/jsonschema: fix dropping of struct for additionalProperties: true
1347dd4 general: add .unity-bin to gitignore
a0e1970 general: add unity configuration to codereview.cfg
351a77e internal/core/adt: add more error positions
79644c3 internal/core/adt: allow hidden fields alongside embedded lists
70e2f2e internal/core/adt: discard errors in field verification
452c7bd internal/core/adt: don't count optional fields as proof of struct
3d926af internal/core/adt: don't eliminate incomplete disjuncts prematurely
1589a07 internal/core/adt: get default when comparing ot bottom
e12e26e internal/core/compile: fix let resolution bug
77e6b51 internal/core/compile: reenable disabled tests
2c86835 internal/core/eval: minor performance improvements for disjunctions
e1e7031 internal/core/runtime: decouple cue API from Runtime
c6b7ab2 internal/cuetest: add support for issue-based conditions/checkers
38f0f63 pkg/internal: fix decimal and string list errors
af3c9dc pkg/list: fix list.Unique dedupping
dcf932f tools/trim: fix too aggressive elimination for disjunctions

v0.3.0-beta.5

This release has some important bug fixes. It also is a bit stricter in considering some errors that were previously evaluation-time errors to be compile time errors.

There is also a slight change in default handling, which should not affect most configurations.

New compile-time errors

The expression >10 <20 is syntactically valid CUE, but can never meaningfully evaluate to something. It parses as >(10 < 20) and thus evaluates to >true, which is undefined. The error messages associated with this were often confusing to the user. Also, leaving this to be detected during runtime would sometimes hide the error away.

Consistent with the “fail early” mantra, these cases are now detected and reported by the compiler.

Backwards compatibility

Defaults

Previously, for

#def: {
   *{} | {a: int} | {b: int}
   *{} | {c: int} | {d: int}
}
x: #def & {a: int}

would officially resolve to

x: {a: int} | {a: int, c: int} | {a: int, d: int}

Thanks to a hack in the current v0.3 implementation, the first disjunct was made the default and all resolved fine. We have now slightly changed the definition of defaults to handle this case.
Basically, now if during a unification all terms of a disjunction that are marked as default are eliminated, the remaining terms will behave as if they originated from an unmarked disjunction.
In practice this means defaults behave as users typically expect them to behave. This may also cover cases that were not previously handed

Note that v0.2 solved things differently: if in the result of a disjunction one term subsumes another, then the latter could be dropped. So in v0.2, the answer would be x: {a: int}, even in the absence of defaults. The problem with this approach is that it could lead to confusing answers and that the default could not always be accurately determined. A goal of v0.3 was to have a simpler story for disambiguation and let simplifications be merely a matter of optimization.

There is still a slight discrepancy in the current implementation and the spec that may cause unexpected results. We expect this to be sufficiently rare and we intend to address this in a rework of disjunction computation, aimed at further performance optimizations.

Changelog

1b03c9f ci: move to non-Docker version of goreleaser
ae51469 ci: skip known flakey test on go1.15 windows
500e431 doc/ref/spec.md: add missing binary_digits
89cada6 doc/tutorial: address cuelang/cuelang.org#126
77b475f doc: fix install instructions in main README and contributing
25ba1d9 internal/core/adt: comment out leak test
b25147c internal/core/adt: control fixes for if clauses
267379b internal/core/adt: fix trim bug
48d255a internal/core/adt: some optimizations for pattern constraints
87d43b7 internal/core/adt: strip "notDefault" status for disjunctions with unused defaults
760aa11 internal/core/compile: make >10 <20 a compile error
b20ac5a internal/core/convert: fix conversion from float
bcd752a internal/core/eval: handle embedded top correctly
a5daa16 internal/core/export: fix quoting of definitions
cb4fae9 internal/cuetest: consolidate testscript/cuetxtar helpers
bf6c3fc internal/cuetest: move Kubernetes related helper to where they are used
15934d9 tools/trim: add tracer
1d9aa17 tools/trim: don't trim shared nodes
af5ea97 tools/trim: fix defaults bug