umoci 0.4.7

NOTE This release has a minor bug -- umoci --version will tell you that the version is "unknown". This was fixed in #369.

A security flaw was found in umoci, and has been fixed in this release.
If umoci was used to unpack a malicious image (using either
umoci unpack or umoci raw unpack) that contained a symlink entry for /.,
umoci would apply subsequent layers to the target of the symlink
(resolved on the host filesystem). This means that if you ran umoci as
root, a malicious image could overwrite any file on the system (assuming
you didn't have any other access control restrictions). Thanks to Robin
Peraglie from Cure53 for discovering this bug. CVE-2021-29136

Other changes in this release:

• umoci now compiles on FreeBSD and appears to work, with the notable
  limitation that it currently refuses to extract non-Linux images on any
  platform (this will be fixed in a future release -- see #364). #357
• Initial fuzzer implementations for oss-fuzz. #365

• umoci will now read all trailing data from image layers, to combat the
  existence of some image generators that appear to append NUL bytes to the end
  of the gzip stream (which would previously cause checksum failures because we
  didn't read nor checksum the trailing junk bytes). However, umoci will still
  not read past the descriptor length. #360
• umoci now ignores all overlayfs xattrs during unpack and repack operations,
  to avoid causing issues when packing a raw overlayfs directory. #354
• Changes to the (still-internal) APIs to allow for users to use umoci more
  effectively as a library.
  • The garbage collection API now supports custom GC policies. #338
  • The mutate API now returns information about what layers were added by the
    operation. #344
  • The mutate API now supports custom compression, and has in-tree support for
    zstd. #348 #350
  • Support overlayfs-style whiteouts during unpack and repack. #342

Thanks to all of the people who made this release possible:

• Adam Korcz adam@adalogics.com
• Aleksa Sarai cyphar@cyphar.com
• Mateusz Kwiatkowski mateusz@serveraptor.com
• Nisha K nishak@vmware.com
• Ramkumar Chinchani rchincha@cisco.com
• Shengjing Zhu zhsj@debian.org
• Tycho Andersen tycho@tycho.pizza

LGTMs: @cyphar
Signed-off-by: Aleksa Sarai cyphar@cyphar.com

umoci 0.4.6

umoci has been adopted by the Open Container Initative as a reference
implementation of the OCI Image Specification. This will have little impact
on the roadmap or scope of umoci, but it does further solidify umoci as a
useful piece of "boring container infrastructure" that can be used to build
larger systems.

NOTICE: As part of the adoption procedure, the import path and module
name of umoci has changed from github.com/openSUSE/umoci to
github.com/opencontainers/umoci. This means that users of our (still
unstable) Go API will have to change their import paths in order to update
to newer versions of umoci.

The old GitHub project will contain a snapshot of v0.4.5 with a few
minor changes to the readme that explain the situation. Go projects which
import import the archived project will receive build warnings that
explain the need to update their import paths.

• umoci now builds on MacOS, and we currently run the unit tests on MacOS to
  hopefully catch core regressions (in the future we will get the
  integration tests running to catch more possible regressions).
  #318

• Suppress repeated xattr warnings on destination filesystems that do not
  support xattrs. #311
• Work around a long-standing issue in our command-line parsing library (see
  urfave/cli#1152) by disabling argument re-ordering for umoci config,
  which often takes --prefixed flag arguments. #328

Thanks to all of the people who made this release possible:

• Aleksa Sarai asarai@suse.de
• David Trudgian dave@trudgian.net
• Ramkumar Chinchani rchincha@cisco.com
• Tycho Andersen tycho@tycho.ws

LGTMs: @cyphar @tych0
Signed-off-by: Aleksa Sarai asarai@suse.de

umoci 0.4.5

• Expose umoci subcommands as part of the API, so they can be used by other Go
  projects. openSUSE/umoci#289
• Add extensible hooking to the core libraries in umoci, to allow for
  third-party media-types to be treated just like first-party ones (the key
  difference is the introspection and parsing logic). openSUSE/umoci#299
  openSUSE/umoci#307

• Use type: bind for generated config.json bind-mounts. While this doesn't
  make too much sense (see opencontainers/runc#2035), it does mean that
  rootless containers work properly with newer runc releases (which appear to
  have regressed when handling file-based bind-mounts with a "bad" type).
  openSUSE/umoci#294 openSUSE/umoci#295
• Don't insert a new layer if there is no diff. openSUSE/umoci#293
• Only output a warning if forbidden extended attributes are present inside the
  tar archive -- otherwise we fail on certain (completely broken) Docker
  images. openSUSE/umoci#304

Thanks to all of the people who made this release possible:

• Aleksa Sarai asarai@suse.de
• David Trudgian dave@trudgian.net
• Tycho Andersen tycho@tycho.ws

Signed-off-by: Aleksa Sarai asarai@suse.de

umoci 0.4.4

• Full-stack verification of blob hashes and descriptor sizes is now done on
  all operations, improving our hardening against bad blobs (we already did
  some verification of layer DiffIDs but this is far more thorough).
  openSUSE/umoci#278 openSUSE/umoci#280 openSUSE/umoci#282

Thanks to all of the people that made this release possible:

• Aleksa Sarai asarai@suse.de
• Cameron Nemo camerontnorman@gmail.com
• Tobias Klauser tklauser@distanz.ch

Signed-off-by: Aleksa Sarai asarai@suse.de

umoci 0.4.3

• All umoci commands that had --history.* options can now decide to omit a
  history entry with --no-history. Note that while this is supported for
  commands that create layers (umoci repack, umoci insert, and umoci raw add-layer) it is not recommended to use it for those commands since it can
  cause other tools to become confused when inspecting the image history. The
  primary usecase is to allow umoci config --no-history to leave no traces in
  the history. See OSInside/kiwi#871. openSUSE/umoci#270
• umoci insert now has a --tag option that allows you to non-destructively
  insert files into an image. The semantics match umoci config --tag.
  openSUSE/umoci#273

Thanks to all of the people that made this release possible:

• Aleksa Sarai asarai@suse.de
• Jintao Zhang zhangjintao9020@gmail.com
• Tycho Andersen tycho@tycho.ws

Signed-off-by: Aleksa Sarai asarai@suse.de

umoci 0.4.2

• umoci now has an exposed Go API. At the moment it's unclear whether it will
  be changed significantly, but at the least now users can use
  umoci-as-a-library in a fairly sane way. openSUSE/umoci#245
• Added umoci unpack --keep-dirlinks (in the same vein as rsync's flag with
  the same name) which allows layers that contain entries which have a symlink
  as a path component. openSUSE/umoci#246
• umoci insert now supports whiteouts in two significant ways. You can use
  --whiteout to "insert" a deletion of a given path, while you can use
  --opaque to replace a directory by adding an opaque whiteout (the default
  behaviour causes the old and new directories to be merged).
  openSUSE/umoci#257

• Docker has changed how they handle whiteouts for non-existent files. The
  specification is loose on this (and in umoci we've always been liberal with
  whiteout generation -- to avoid cases where someone was confused we didn't
  have a whiteout for every entry). But now that they have deviated from the
  spec, in the interest of playing nice, we can just follow their new
  restriction (even though it is not supported by the spec). This also makes
  our layers slightly smaller. openSUSE/umoci#254
• umoci unpack now no longer erases system.nfs4_acl and also has some more
  sophisticated handling of forbidden xattrs. openSUSE/umoci#252
  openSUSE/umoci#248
• umoci unpack now appears to work correctly on SELinux-enabled systems
  (previously we had various issues where umoci wouldn't like it when it was
  trying to ensure the filesystem was reproducibly generated and SELinux xattrs
  would act strangely). To fix this, now umoci unpack will only cause errors
  if it has been asked to change a forbidden xattr to a value different than
  it's current on-disk value. openSUSE/umoci#235 openSUSE/umoci#259

Thanks to all of the people that made this release possible:

• Aleksa Sarai asarai@suse.de
• Tycho Andersen tycho@tycho.ws

Signed-off-by: Aleksa Sarai asarai@suse.de

umoci 0.4.1

• The number of possible tags that are now valid with umoci subcommands has
  increased significantly due to an expansion in the specification of the
  format of the ref.name annotation. To quote the specification, the
  following is the EBNF of valid refname values. openSUSE/umoci#234

  refname   ::= component ("/" component)*
  component ::= alphanum (separator alphanum)*
  alphanum  ::= [A-Za-z0-9]+
  separator ::= [-._:@+] | "--"
  

• A new umoci insert subcommand which adds a given file to a path inside the
  container. openSUSE/umoci#237
• A new umoci raw unpack subcommand in order to allow users to unpack images
  without needing a configuration or any of the manifest generation.
  openSUSE/umoci#239
• umoci how has a logo. Thanks to Max Bailey for contributing
  this to the project. openSUSE/umoci#165 openSUSE/umoci#249

• umoci unpack now handles out-of-order regular whiteouts correctly (though
  this ordering is not recommended by the spec -- nor is it required). This is
  an extension of openSUSE/umoci#229 that was missed during review.
  openSUSE/umoci#232
• umoci unpack and umoci repack now make use of a far more optimised gzip
  compression library. In some benchmarks this has resulted in umoci repack
  speedups of up to 3x (though of course, you should do your own benchmarks).
  umoci unpack unfortunately doesn't have as significant of a performance
  improvement, due to the nature of gzip decompression (in future we may
  switch to zlib wrappers). openSUSE/umoci#225 openSUSE/umoci#233

Thanks to all of the contributors that made this release possible:

• Abhilash Raj maxking@users.noreply.github.com
• Akihiro Suda suda.akihiro@lab.ntt.co.jp
• Aleksa Sarai asarai@suse.de
• Felix Abecassis fabecassis@nvidia.com
• Max Bailey maxbailey@flywheel.io
• Tycho Andersen tycho@tycho.ws

Signed-off-by: Aleksa Sarai asarai@suse.de

umoci 0.4.0

• umoci repack now supports --refresh-bundle which will update the
  OCI bundle's metadata (mtree and umoci-specific manifests) after packing the
  image tag. This means that the bundle can be used as a base layer for
  future diffs without needing to unpack the image again. openSUSE/umoci#196
• Added a website, and reworked the documentation to be better structured. You
  can visit the website at umo.ci. openSUSE/umoci#188
• Added support for the user.rootlesscontainers specification, which allows
  for persistent on-disk emulation of chown(2) inside rootless containers.
  This implementation is interoperable with @AkihiroSuda's PRoot
  fork (though we do not test its interoperability at the
  moment) as both tools use the same protobuf
  specification. openSUSE/umoci#227
• umoci unpack now has support for opaque whiteouts (whiteouts which remove
  all children of a directory in the lower layer), though umoci repack does
  not currently have support for generating them. While this is technically a
  spec requirement, through testing we've never encountered an actual user of
  these whiteouts. openSUSE/umoci#224 openSUSE/umoci#229
• umoci unpack will now use some rootless tricks inside user namespaces for
  operations that are known to fail (such as mknod(2)) while other operations
  will be carried out as normal (such as lchown(2)). It should be noted that
  the /proc/self/uid_map checking we do can be tricked into not detecting
  user namespaces, but you would need to be trying to break it on purpose.
  openSUSE/umoci#171 openSUSE/umoci#230

• Fix a bug in our "parent directory restore" code, which is responsible for
  ensuring that the mtime and other similar properties of a directory are not
  modified by extraction inside said directory. The bug would manifest as
  xattrs not being restored properly in certain edge-cases (which we
  incidentally hit in a test-case). openSUSE/umoci#161 openSUSE/umoci#162
• umoci unpack will now "clean up" the bundle generated if an error occurs
  during unpacking. Previously this didn't happen, which made cleaning up the
  responsibility of the caller (which was quite difficult if you were
  unprivileged). This is a breaking change, but is in the error path so it's
  not critical. openSUSE/umoci#174 openSUSE/umoci#187
• umoci gc now will no longer remove unknown files and directories that
  aren't flock(2)ed, thus ensuring that any possible OCI image-spec
  extensions or other users of an image being operated on will no longer
  break. openSUSE/umoci#198
• umoci unpack --rootless will now correctly handle regular file unpacking
  when overwriting a file that umoci doesn't have write access to. In
  addition, the semantics of pre-existing hardlinks to a clobbered file are
  clarified (the hard-links will not refer to the new layer's inode).
  openSUSE/umoci#222 openSUSE/umoci#223

---

Thanks to all of the contributors that made this release possible:

• Aleksa Sarai asarai@suse.de
• Jonathan Boulle jonathanboulle@gmail.com
• Serge Hallyn serge@hallyn.com
• Tobias Klauser tklauser@distanz.ch
• Tycho Andersen tycho@tycho.ws
• W. Trevor King wking@tremily.us

Signed-off-by: Aleksa Sarai asarai@suse.de

umoci 0.3.1

• Fix several minor bugs in hack/release.sh that caused the release artefacts
  to not match the intended style, as well as making it more generic so other
  projects can use it. openSUSE/umoci#155 openSUSE/umoci#163

• A recent configuration issue caused go vet and go lint to not run as part
  of our CI jobs. This means that some of the information submitted as part of
  CII best practices badging was not accurate. This has been corrected,
  and after review we concluded that only stylistic issues were discovered by
  static analysis. openSUSE/umoci#158

• 32-bit unit test builds were broken in a refactor in [0.3.0]. This has been
  fixed, and we've added tests to our CI to ensure that something like this
  won't go unnoticed in the future. openSUSE/umoci#157

• umoci unpack would not correctly preserve set{uid,gid} bits. While this
  would not cause issues when building an image (as we only create a manifest
  of the final extracted rootfs), it would cause issues for other users of
  umoci. openSUSE/umoci#166 openSUSE/umoci#169

• Updated to v0.4.1 of go-mtree, which fixes several minor
  bugs with manifest generation. openSUSE/umoci#176

• umoci unpack would not handle "weird" tar archive layers previously (it
  would error out with DiffID errors). While this wouldn't cause issues for
  layers generated using Go's archive/tar implementation, it would cause
  issues for GNU gzip and other such tools. openSUSE/umoci#178
  openSUSE/umoci#179

• umoci unpack's mapping options (--uid-map and --gid-map) have had an
  interface change, to better match the user_namespaces(7)
  interfaces. Note that this is a breaking change, but the workaround is to
  switch to the trivially different (but now more consistent) format.
  openSUSE/umoci#167

• umoci unpack used to create the bundle and rootfs with world
  read-and-execute permissions by default. This could potentially result in an
  unsafe rootfs (containing dangerous setuid binaries for instance) being
  accessible by an unprivileged user. This has been fixed by always setting the
  mode of the bundle to 0700, which requires a user to explicitly work around
  this basic protection. This scenario was documented in our security
  documentation previously, but has now been fixed. openSUSE/umoci#181
  openSUSE/umoci#182

---

Thanks to all of the contributors that made this release possible:

• Aleksa Sarai asarai@suse.de
• Jonathan Boulle jonathanboulle@gmail.com
• Serge Hallyn serge@hallyn.com

Signed-off-by: Aleksa Sarai asarai@suse.de

umoci 0.3.0

• umoci now passes all of the requirements for the CII best practices
  bading program. openSUSE/umoci#134
• umoci also now has more extensive architecture, quick-start and
  roadmap documentation. openSUSE/umoci#134
• umoci now supports 1.0.0 of the OCI image
  specification and 1.0.0 of the OCI runtime
  specification, which are the first milestone release.
  Note that there are still some remaining UX issues with --image and
  other parts of umoci which may be subject to change in future
  versions. In particular, this update of the specification now means
  that images may have ambiguous tags. umoci will warn you if an
  operation may have an ambiguous result, but we plan to improve this
  functionality far more in the future. openSUSE/umoci#133
  openSUSE/umoci#142
• umoci also now supports more complicated descriptor walk structures,
  and also handles mutation of such structures more sanely. At the
  moment, this functionality has not been used "in the wild" and umoci
  doesn't have the UX to create such structures (yet) but these will be
  implemented in future versions. openSUSE/umoci#145
• umoci repack now supports --mask-path to ignore changes in the
  rootfs that are in a child of at least one of the provided masks when
  generating new layers. openSUSE/umoci#127

• Error messages from github.com/openSUSE/umoci/oci/cas/drivers/dir
  actually make sense now. openSUSE/umoci#121
• umoci unpack now generates config.json blobs according to the
  still proposed OCI image specification conversion
  document. openSUSE/umoci#120
• umoci repack also now automatically adding Config.Volumes from the
  image configuration to the set of masked paths. This matches recently
  added recommendations by the spec, but is a
  backwards-incompatible change because the new default is that
  Config.Volumes will be masked. If you wish to retain the old
  semantics, use --no-mask-volumes (though make sure to be aware of
  the reasoning behind Config.Volume masking). openSUSE/umoci#127
• umoci now uses SecureJoin rather than a patched
  version of FollowSymlinkInScope. The two implementations are roughly
  equivalent, but SecureJoin has a nicer API and is maintained as a
  separate project. openSUSE/umoci#148
• Switched to using golang.org/x/sys/unix over syscall where
  possible, which makes the codebase significantly cleaner.
  openSUSE/umoci#141

---

Thanks to all of the contributors that made this release possible:

• Aleksa Sarai asarai@suse.de
• Maximilian Meister mmeister@suse.de
• Valentin Rothberg vrothberg@suse.com

Signed-off-by: Aleksa Sarai asarai@suse.de