README.md

Wyng

Fast incremental backups for logical volumes and disk images.

Introduction

Wyng is able to deliver faster incremental backups for logical volumes and disk images. It accesses copy-on-write metadata (instead of comparing all data for each backup) to instantly find changes since the last backup. Combined with its efficient archive format, Wyng can also very quickly reclaim space from older backup sessions.

Having nearly instantaneous access to volume changes and a nimble archival format enables backing up even terabyte-sized volumes multiple times per hour with little impact on system resources.

Wyng pushes data to archives in a stream-like fashion, which avoids temporary data caches and re-processing data. And Wyng's ingenious snapshot rotation avoids common aging snapshot space consumption pitfalls.

Wyng also doesn't require the source admin system to ever mount processed volumes or to handle them as anything other than blocks, so it safely handles untrusted data in guest filesystems to bolster container-based security.

Status

Public beta with a range of features including:

• Incremental backups of Linux logical volumes from Btrfs, XFS and Thin-provisioned LVM

• Supported destinations: Local filesystem, Virtual machine or SSH host

• Fast pruning of old backup sessions

• Basic archive management such as add/delete volume and auto-pruning

• Automatic creation & management of local snapshots

• Data deduplication

• Marking and selecting archived snapshots with user-defined tags

Version 0.8 major enhancements:

• Btrfs and XFS reflink support

• Authenticated encryption with auth caching

• Full data & metadata integrity checking

• Fast differential receive based on available snapshots

• Overall faster operation

• Change autoprune behavior with --apdays

• Configure defaults in /etc/wyng/wyng.ini

• Mountpoints no longer required at destination

• Simple selection of archives and local paths: Choose any local or dest each time you run Wyng

• Multiple volumes can now be specified for most Wyng commands; send and receive support multiple storage pools

Wyng is released under a GPL license and comes with no warranties expressed or implied.

Wyng v0.8 Requirements & Setup

Before starting:

• Python 3.8 or greater is required for basic operation.

• For encryption and top performance, the python3-pycryptodome and python3-zstd packages should be installed, respectively.

• Volumes to be backed-up should reside locally in one of the following snapshot-capable storage types: LVM thin-provisioned pool, Btrfs subvolume, or XFS/reflink capable filesystem. Otherwise, volumes may be imported from or saved to other filesystems at standard (slower) speeds.

• For backing up from LVM, thin-provisioning-tools & lvm2 must be present on the source system. For Btrfs, the btrfs command must be present.

• The destination system where the Wyng archive is stored (if different from source) should also have python3, plus a basic Unix command set and filesystem (i.e. a typical Linux or BSD system). Otherwise, samba, FUSE, etc. may be used to access remote storage using smb, sftp, s3 or other protocols without concern for python or Unix commands.

• See the 'Testing' section below for tips and caveats about using the alpha and beta versions.

Getting Started

Wyng is distributed as a single Python executable with no complex supporting modules or other program files; it can be placed in '/usr/local/bin' or another place of your choosing.

Archives can be created with wyng arch-init:

wyng arch-init --dest=ssh://me@exmaple.com:/home/me/mylaptop.backup

...or...

wyng arch-init --dest=file:/mnt/drive2/mylaptop.backup

The examples above create a 'mylaptop.backup' directory on the destination. The --dest argument includes the destination type, remote system (where applicable) and directory path.

Next you can start making backups with wyng send:

wyng send --dest=file:/mnt/drive1/mylaptop.backup --local=volgrp1/pool1 root-volume home-volume

This command sends two volumes 'root-volume' and 'home-volume' from the LVM thin pool 'volgrp1/pool1' to the destination archive.

Operation

Run Wyng using the following commands and arguments in the form of:

wyng [--options] command [volume_names] [--options]

Command summary

Command	Description
list [volume_name]	List volumes or volume sessions.
send [volume_name]	Perform a backup of enabled volumes.
receive volume_name [*]	Restore volume(s) from the archive.
verify volume_name [*]	Verify volumes' data integrity.
prune [volume_name] [*]	Remove older backup sessions to recover archive space.
delete volume_name	Remove entire volume from config and archive.
rename vol_name new_name	Renames a volume in the archive.
arch-init	Create a new Wyng archive.
arch-deduplicate	Deduplicate existing data in an archive.
version	Print the Wyng version and exit.

Advanced commands

Command	Description
monitor	Collect volume change metadata & rotate snapshots.
diff volume_name [*]	Compare local volume with archived volume.
add volume_name [*]	Adds a volume name without session data to the archive.
arch-check [volume_name] [*]	Thorough check of archive data & metadata

send

Performs a backup by storing volume data to a new session in the archive. If the volume already exists in the archive, incremental mode is automatically used.

wyng send my_big_volume --local=vg/pool --dest=file:/mnt/drive1/mylaptop.backup

A send operation may refuse to backup a volume if there is not enough space on the destination. One way to avoid this situation is to specify --autoprune=on which will cause Wyng to remove older backup sessions from the archive when space is needed.

Volume names for non-LVM storage may include subdirectories, making them relative paths in the same manner as file paths in tar. For example, wyng --local=/mnt/pool1 send appvms/personal.img will send the volume located at '/mnt/pool1/appvms/personal.img'.

receive

Retrieves a volume instance (using the latest session ID if --session isn't specified) from the archive and saves it to either the --local storage or the path specified with --save-to. If --session is used, only one date-time is accepted. The volume name is required.

wyng receive vm-work-private --local=vg/pool --dest=file:/mnt/drive1/mylaptop.backup

...restores a volume called 'vm-work-private' to 'myfile.img' in the LVM thin pool 'vg/pool'. Note that --dest always refers to the archive location, so the volume is being restored from '/mnt/drive1/mylaptop.backup'.

Its possible to receive to any valid file path or block device using the --save-to option, which can be used in place of --local. For any save path, Wyng will try to discard old data before receiving unless --sparse, --sparse-write or --use-snapshot options are used.

verify

The verify command is similar to receive without saving the data. For both receive and verify modes, an error will be reported with a non-zero exit code if the received data does not pass integrity checks.

prune

Quickly reclaims space on a backup drive by removing any prior backup session you specify; it does this without re-writing data blocks or compromising volume integrity.

To use, supply a single exact date-time in YYYYMMDD-HHMMSS format to remove a specific session, or two date-times representing a range:

wyng prune --all --session=20180605-000000,20180701-140000 --dest=file:/mnt/drive1/mylaptop.backup

...removes backup sessions from midnight on June 5 through 2pm on July 1 for all volumes. Alternately, --all-before may be used with a single --session date-time to prune all sessions prior to that time.

The --keep option can accept a single date-time or a tag in the form ^tagID. Matching sessions will be excluded from pruning and autopruning.

delete

Removes a volume's Wyng-managed snapshots, config and metadata from the source system and all of its data from the destination archive (everything deleted except the source volume). Use with caution!

An alternate form of delete will remove all Wyng archive-related metadata (incl. snapshots) from the local system without affecting the archive on the destination:

wyng delete --clean

Alternately, using delete --clean --all will remove all known Wyng metadata from the local system, including any snapshots from the --local path.

rename

wyng rename oldname newname

Renames a volume 'oldname' in the archive to 'newname'. Note: This will rename only the archive volume, not your source volume.

arch-deduplicate

De-duplicates the entire archive by removing repeating patterns. This can save space on the destination's drive while keeping the archived volumes intact.

De-duplication can also be performed incrementally by using --dedup with send.

wyng arch-deduplicate

arch-init

Create a new archive on a mounted drive...

wyng arch-init --dest=file:/mnt/backups/archive1

Create a new archive with stronger compression on a remote system...

wyng arch-init --dest=ssh://user@example.com --compression=zstd:7

Optional parameters for arch-init are encrypt, compression, hashtype and chunk-factor. These cannot be changed for an archive after it is initialized.

arch-check

Intensive check of archive integrity, reading each session's deltas completely starting with the newest and working back to the oldest. This differs from verify which first builds a complete index and checks a complete volume.

Using --session=newest provides a 'verify the last session' function (useful after an incremental backup). Otherwise, supplying a date-time will make arch-check start the check from that point and then continue working toward the oldest session. Session ranges are not yet supported.

Depending on how arch-check is used, the verification process can be shorter or much longer than using verify as the latter is always the size of a volume snapshot. The longest, most complete form of arch-check is to supply no parameters, which checks all sessions in all volumes.

monitor

Frees disk space that is cumulatively occupied by aging snapshots, thereby addressing a common resource usage issue with snapshot-based backups. After harvesting their change metadata, the older snapshots are replaced with new ones occupying zero space. Running monitor isn't necessary, but it only takes a few seconds and is good to run on a frequent, regular basis if you have some volumes that are write-intensive. Volume names may also be specified if its desired to monitor only certain volumes.

This rule in /etc/cron.d runs monitor every 20 minutes:

*/20 * * * * root su -l -c '/usr/local/bin/wyng monitor --all'

diff

Compare a local volume snapshot with the archive and report any differences. This is useful for diagnostics and can also be useful after a verification error has occurred.

add

Adds new, empty volume name(s) to the archive. On subsequent send -a, Wyng will backup the volume data if present.

---

Parameters / Options summary

Option	Description
--dest=URL	Location of backup archive.
--local=vg/pool ...or...	Storage pool containing local volumes.
--local=/absolute/path
--authmin=N	Remember authentication for N minutes (default: 2)
--all, -a	Select all volumes (most cmds); Or clean all snapshots (delete).
--volex=volname	Exclude volumes (send, monitor, list, prune).
--dedup, -d	Use deduplication for send (see notes).
--session=date-time[,date-time]	Select a session or session range by date-time or tag (receive, verify, prune).
--all-before	Select all sessions before the specified --session date-time (prune).
--autoprune=off	Automatic pruning by calendar date.
--apdays=A:B:C:D	Number of days to keep or to thin-out older sessions
--keep=date-time	Specify date-time or tag of sessions to keep (prune).
--tag=tagname[,desc]	Use session tags (send, list).
--sparse	Receive volume data sparsely (implies --sparse-write)
--sparse-write	Overwrite local data only where it differs (receive)
--use-snapshot	Use snapshots when available for faster receive.
--send-unchanged	Record unchanged volumes, don't skip them (send)
--unattended, -u	Don't prompt for interactive input.
--clean	Perform garbage collection (arch-check) or metadata removal (delete).
--verbose	Increase details.
--quiet	Shhh...

Advanced Options

Option	Description
--save-to=path	Save volume to path (receive).
--local_from=json file	Specify local:[volumes] sets instead of --local.
--import-other-from	Import volume data from a non-snapshot capable path during send
--session-strict=_on	off_
--encrypt=cipher	Set encryption mode or 'off' (default: 'xchacha20-dgr')
--compression	(arch-init) Set compression type:level.
--hashtype	(arch-init) Set data hash algorithm: hmac-sha256 or blake2b.
--chunk-factor	(arch-init) Set archive chunk size.
--vid	Select volume by ID (delete)
--tar-bypass	Use direct access for file:/ archives (send)
--passcmd='command'	Read passphrase from output of a wallet/auth app
--upgrade-format	Upgrade older Wyng archive to current format. (arch-check)
--remap	Remap volume to current archive during send or diff.
--json	Output volume: session info in json format (list).
--force	Not used with most commands.
--meta-reduce=mode:N	Reduce or extend local metadata caching.
--meta-dir=path	Use a different metadata dir than the default.
--debug	Debug mode

Option Details

--dest=URL

This option tells Wyng where to access the archive and has the same meaning for all read or write commands. It accepts one of the following forms:

URL Form	Destination Type
file:/path	Local filesystem
ssh://user@example.com[:port][/path]	SSH server
qubes://vm-name[/path]	Qubes virtual machine
qubes-ssh://vm-name:me@example.com[:port][/path]	SSH server via a Qubes VM

--local

The location of local storage where logical volumes, disk images, etc. reside. This serves as the source for send commands, and as the place where receive restores/saves volumes.

This parameter takes one of two forms: Either the source volume group and pool as 'vgname/poolname' or a directory path on a reflink-capable filesystem such as Btrfs or XFS (for Btrfs the path should end at a subvolume). Required for commands send, monitor and diff (and receive when not using --saveto).

--session=<date-time>[,<date-time>] OR --session=^<tag>[,^<tag>]

Session allows you to specify a single date-time or tag spec for thereceive, verify, diff, and arch-check commands. Using a tag selects the last session having that tag. When specifying a tag, it must be prefixed by a ^ carat.

For prune, specifying a tag will have different effects: a single spec using a tag will remove only each individual session with that tag, whereas a tag in a dual (range) spec will define an inclusive range anchored at the first instance of the tag (when the tag is the first spec) or the last instance (when the tag is the second range spec). Also, date-times and tags may be used together in a range spec.

--volex=<volume1> [--volex=<volume2> *]

Exclude one or more volumes from processing. May be used with commands that operate on multiple volumes in a single invocation, such as send. volex is useful in cases where a volume is in the archive, but frequent automatic backups aren't needed. Or when certain volumes should be excluded from prune, monitor, etc.

Please note: volex syntax had to be changed from the v0.3 option syntax which used a comma to specify multiple volumes.

--sparse-write

Used with receive, the sparse-write mode tells Wyng not to create a brand-new local volume and results in the data being sparsely written into the existing volume instead. This is useful if the existing local volume is a clone/snapshot of another volume and you wish to save local disk space. It is also best used when the backup/archive storage is local (i.e. fast USB drive or similar) and you don't want the added CPU usage of full --sparse mode.

--sparse

The sparse mode can be used with the receive command to intelligently retrieve and overwrite an existing local volume so that only the differences between local and archived volumes will be fetched from the archive and written to the local volume. This results in reduced network usage at the expense of some extra CPU usage on the local machine, and also uses less local disk space when snapshots are a factor. The best situation for sparse mode is when you want to restore/revert a large volume with a containing a limited number of changes over a low-bandwidth connection.

--use-snapshot (experimental)

A faster-than-sparse option that uses a snapshot as the baseline for the receive, if one is available. Use with --sparse if you want Wyng to fall back to sparse mode when snapshots are not already present.

--tar-bypass (experimental)

Use direct access for file:/ archives during send. This can reduce sending times by up to 20%.

--dedup, -d

When used with the send command, data chunks from the new backup will be sent only if they don't already exist somewhere in the archive. Otherwise, a link will be used saving disk space and possibly time and bandwith.

The trade-off for deduplicating is longer startup time for Wyng, in addition to using more memory and CPU resources during backups. Using --dedup works best if you are backing-up multiple volumes that have a lot of the same content and/or you are backing-up over a slow Internet link.

--autoprune=(off | on | full)

Autoprune may be used with either the prune or send commands and will cause Wyng to automatically remove older backup sessions according to date criteria. When used with send specifically, the autopruning process will be triggered in advance of sending new sessions when using full mode, or in on mode only or if the destination filesytem is low on free space. (See --apdays to specify additional autoprune parameters.)

Selectable modes are:

off is the current default.

on removes some sessions, as space is needed on the destination.

full removes all sessions that are due to expire according to above criteria.

--apdays=A:B:C:D

Adjust autoprune with the following four parameters:

• A: The oldest day before which all sessions are removed. Default is 0 (disabled).
• B: Thinning days; the number of days before which some sessions will be removed according to the ratio D/C. Default is 62 days.
• C: Number of days for the D/C ratio. Default is 1.
• D: Number of sessions for the D/C ratio. Default is 2.

An example: --apdays=365:31:1:2 will cause autoprune to remove all sessions that are older than 365 days, and sessions older than 31 days will be thinned-out while preserving (roughly on average) two sessions per day.

--tag=<tagname[,description]>

With send, attach a tag name of your choosing to the new backup session/snapshot; this may be repeated on the command line to add multiple tags. Specifying an empty '' or '@' tag will cause Wyng to ask for one or more tags to be manually input; this also causes list to display tag information when listing sessions.

--authmin=<minutes>
--passcmd=<command>

These two options help automate Wyng authentication, and may be used together or separately.

--authmin takes a numeric value from -1 to 60 for the number of minutes to remember the current authentication for subsequent Wyng invocations. The default authmin time is 2 minutes. Specifying a -1 will cancel a prior authentication and 0 will skip storing the authentication.

The --passcmd option takes a string representing a shell command that outputs a passphrase to stdout which Wyng then reads instead of prompting for passphrase input. If a prior auth from --authmin is active, this option is ignored and the command will not be executed.

--import-other-from=volname:|:path

Enables sending a volume from a path that is not a supported snapshot storage type. This may be any regular file or a block device which is seek-able.

When it is specified this option causes slow delta comparisons to be used for the specified volume(s) instead of the default fast snapshot-based delta comparisons. It is not recommended for regular use with large volumes.

The special delimeter used to separate the volname (archive volume name) and the path is ':|:' which means this option cannot be used to send directly to volume names in the archive which contain that character sequence.

--session-strict=on|off

For receive, verify, diff: If set to 'on' (the default) Wyng won't retrieve volumes from next-oldest session if the specified volumes don't have an exact match for the specified session. When set to 'off' Wyng will try to retrieve the next-oldest version of the volume if one exists.

--local_from=_json file_

Specify both local storage and volume names for send or receive as sets, instead of using --local and volume names on the command line. The json file must take the form of {local-a: [[volname1, alias1], [volnameN, aliasN], ...], ...]}. This allows multiple local storage sources to be sent/received in a single session.

Alias can be 'null' for no alias or any valid name. However, the volume names (or aliases) must all be unique across different sources as they are stored in the same archive. Aliases define which local volume name into which an archive volume will be received, or when sending they indicate a request to actually rename the target volume to the alias.

--meta-reduce=mode:minutes

Control the degree to which locally cached session metadata is retained or removed when Wyng exits. This can effect a noticeable reduction in the space that Wyng uses in /var while trading off a little speed.

Mode is one of off, on, or extra: off results in no reduction (all metadata is retained); on removes uncompressed metadata; extra removes both compressed and uncompressed metadata.

Minutes is an integer defining the metadata's maximum age in minutes, where '0' will cause it to be removed immediately when Wyng exits.

The default setting is 'on:3000'.

--compression

Accepts the forms type or type:level. The three types available are zstd (zstandard), plus zlib and bz2 (bzip2). Note that Wyng will only default to zstd when the 'python3-zstd' package is installed; otherwise it will fall back to the less capable zlib. (default=zstd:3)

--hashtype

Accepts a value of either 'blake2b' or 'hmac-sha256' (default). The digest size is 256 bits.

--chunk-factor

Sets the pre-compression data chunk size used within the destination archive. Accepted range is an integer exponent from '1' to '6', resulting in a chunk size of 64kB for factor '1', 128kB for factor '2', 256kB for factor '3' and so on. To maintain a good space efficiency and performance balance, a factor of '2' or greater is suggested for archives that will store volumes larger than about 100GB. (default=2)

--encrypt

Selects the encryption cipher/mode. The available modes are:

• xchacha20-dgr — Using HMAC-SHA256(rnd||hash) function. This is the default.
• xchacha20-msr — Using HMAC-SHA256(rnd||msg) function.
• xchacha20-ct — Counter based; fast with certain safety trade-offs (see issue 158).
• off — Turns off Wyng's authentication and encryption.

Configuration files

Wyng will look in '/etc/wyng/wyng.ini' for option defaults. For options that are flags with no value like --dedup, use a 1 or 0 to indicate enable or disable (yes or no). For options allowing multiple entries per command line, in the .ini use multiple lines with the 2nd item onward indented by at least one space.

An example wyng.ini file:

[var-global-default]
dedup = 1
authmin = 10
autoprune = full
dest = ssh://user@192.168.0.8/home/user/wyng.backup
local = /mnt/btrfs01/vms
volex = misc/caches.img
  misc/deprecated_apps.img
  windows10_recovery.vmdk

Verifying Code

• Wyng code can be cryptographically verified using either gpg directly or via git:

# Import Key
~$ cd wyng-backup
~/wyng-backup$ gpg --import pubkey
gpg: key 1DC4D106F07F1886: public key "Christopher Laprise <tasket@posteo.net>" imported
gpg: Total number processed: 1
gpg:               imported: 1

# GPG Method
~/wyng-backup$ gpg --verify src/wyng.gpg src/wyng

# Git Method
~/wyng-backup$ git verify-commit HEAD

# Output:
gpg: Signature made Sat 26 Aug 2023 04:20:46 PM EDT
gpg:                using RSA key 0573D1F63412AF043C47B8C8448568C8B281C952
gpg: Good signature from "Christopher Laprise <tasket@posteo.net>" [unknown]
gpg:                 aka "Christopher Laprise <tasket@protonmail.com>" [unknown]

Security notes

Automated authentication:

Wyng supports two modes of supplying passphrase secrets: Standard input and the --passcmd option. The former can accept a secret from a pipe or redirect because when auth is necessary it is always the first input prompt. However, the prompt may not always occur when --authmin value > 0 is used since the passphrase may not be needed for repeat invocations of Wyng.

Persistence of cached archive.ini & archive.salt:

Authentication schemes in general can only verify the authenticity for an object at any point in time; they aren't well suited to telling us if that object (i.e. a backup archive) is the most recent update, and so they are vulnerable to rollback attacks that replace your current archive with an older version (in Wyng this is related to replay attacks, but not downgrade attacks). Wyng guards against such attacks by checking that the time encoded in your locally cached archive.ini isn't newer than the one on the destination/remote; Wyng also displays the last archive modification time whenever you access it.

Protecting and Verifying Archive Authenticity:

With encryption enabled, Wyng provides a kind of built-in verification of archive authenticity; this is because it uses an AEAD cipher mode. However, custom verification (BYOV) is also possible with Wyng and even works on non-encrypted archives. All you need to do is sign the 'archive.ini' file from the top archive directory after executing any Wyng command that changes the archive (i.e. arch-init, add, send, prune, delete, rename).

Subsequently, the steps to verify total archive authenticity would be to simply run wyng arch-check --dest <URL> (using Wyng's built-in authenticated encryption), or else using custom authentication based on GPG, for instance:

gpg --verify archive.ini.sig laptop1.backup/archive.ini && wyng arch-check --dest <URL>

Note that custom signature files should not be stored within the archive directory.

(Although volumes can be verified piecemeal with the wyng verify command, it is not suited to verifying everything within an archive.)

Tips & Caveats

• LVM users: Wyng has an internal snapshot manager which creates snapshots of volumes in addition to any snapshots you may already have on your local storage system. This can pose a serious challenge to lvmthin (aka thin-provisioned LVM) as the default space allocated for metadata is often too small for rigorous & repeated snapshot rotation cycles. It is recommended to at least double the existing or default tmeta space on each thin pool used with wyng send or wyng monitor; see the man page section Manually manage free metadata space of a thin pool LV for guidance on using the lvextend --poolmetadatasize command.

• To reduce the size of incremental backups it may be helpful to remove cache files, if they exist in your source volume(s). Typically, the greatest cache space consumption comes from web browsers, so volumes holding paths like /home/user/.cache can impacted by this, depending on the amount and type of browser use associated with the volume. Three possible approaches are to clear caches on browser exit, delete /home/user/.cache dirs on system/container shutdown (this reasonably assumes cached data is expendable), or to mount .cache on a separate volume that is not configured for backup.

• If you've changed your local path without first running wyng delete --clean to remove snapshots, there may be unwanted snapshots remaining under your old volume group or local directory. LVM snapshots can be found with the patterns *.tick and *.tock with the tag "wyng"; Btrfs/XFS snapshots can be found with sn*.wyng?. Deleting them can prevent unnecessary consumption of disk space.

• If you wish to make a "backup of the backup", i.e. duplicate an archive, its possible to do so with the following:

     mv destpath destpath-updating
     rsync -a --hard-links --delete sourcepath destpath-updating
     mv destpath-updating destpath

• The above rsync command can also efficiently update a duplicate archive, since it can delete files that are no longer present in the origin archive (cp cannot do this). However, note that there is risk of absentmindedly corrupting the duplicate archive if the update process ever ends in an error – this includes when your system is running rsync and it just crashes, thus displaying no error; all errors/interruptions must be handled. By renaming the archive with a suffix like '-updating', the above commands provide some precautionary "oops!" information, a reminder that your update had an error ...and to re-run it to make it right. This risk could be eliminated if Wyng were able to perform the duplicating itself; for plans on adding such a feature, see this issue.

Troubleshooting notes

• If you encounter an error during send/backup that the Btrfs path is "not a subvolume" its probably due to the --local path or pool ending in a dir not a subvolume. If you can't adjust the local path to end at an existing subvol, then a dir can be easily converted using btfs subvol create and mv commands. See the 'misc' project folder for an example dir-to-subvol conversion script.

• A major change in v0.8 is that for send and monitor Wyng will no longer assume you want to act on all known volumes if you don't specify any volumes. You must now use -a or --all, which now work for other commands as well. This change also enables adding new volumes while doing a complete backup, for instance: wyng -a send my-new-volume – updates every volume already in the archive plus backup 'my-new-volume' as well.

• Backup sessions shown in list output may be seemingly (but not actually) out of order if the system's local time shifts substantially between backups, such as when moving between time zones (including DST). If this results in undesired selections with --session ranges, its possible to nail down the precisely desired range by observing the output of list volumename and using exact date-times from the listing.

• Wyng locally stores information about backups in two ways: Snapshots alongside your local source volumes, and metadata under /var/lib/wyng. It is safe to delete Wyng snapshots without risking the integrity of backups (although send will become slower). However, as with all CoW snapshot based backup tools, you should never attempt to directly mount, alter or otherwise utilize a Wyng snapshot as this could (very likely) result in future backup sessions being corrupt (this is why Wyng snapshots are stored as read-only). If you think you have somehow altered a Wyng snapshot, you should consider it corrupt and immediately delete it before the next send. If you're in a pinch and need to use the data in a Wyng snapshot, you should first make your own copy or snapshot of the Wyng snapshot using cp --reflink or lvcreate -s and use that instead.

• Metadata cached under /var/lib/wyng may also be manually deleted. However, the archive.* root files in each 'a_*' directory are part of Wyng's defense against rollback attacks, so if you feel the need to manually reclaim space used in this dir then consider leaving the archive.* files in place.

• If data corruption in the archive is suspected, use wyng arch-check which will scan for errors and present you with options for recovery.

• If a volume becomes damaged and unrecoverable it may be necessary to delete it by its volume ID by using wyng delete --vid instead of the volume name.

Testing

• Wyng v0.4alpha3 and later no longer create or require the wyng.backup040/default directory structure. This means whatever you specify in --dest is all there is to the archive path. It also means accessing an alpha1 or alpha2 archive will require you to either include those dirs explicitly in your --dest path or rename '../wyng.backup040/default' to something else you prefer to use.

• Testing goals are basically stability, usability, security and efficiency. Compatibility is also a valued topic, where source systems are generally expected to be a fairly recent Linux distro or Qubes OS. Destination systems can vary a lot, they just need to have Python and Unix commands or support a compatible FUSE protocol such as sshfs(sftp) or s3.

Donations

Buy me a coffee!

If you like this project, monetary contributions are welcome and can be made through Liberapay or Buymeacoffee or Patreon.