v0.23.0 Default read-only, 255k object limit

By default dxfuse will run in read-only mode. -readOnly flag is deprecated. Experimental -readWrite added, but no upload guarantees are made, further development needed.

Raise the object limit from 10k --> 255k objects within a directory.

Continue prefetching sequential data even if all of the bytes in a previous prefetch window have not been read. Required for running plink with dxfuse.

v0.22.4 go modules

Migrate to go modules. No longer require cloning and building from within $GOPATH

Pull in lastest dxda dx_http changes

Rebuild with the latest dxda and dx_http changes. Notably dnanexus/dxda@1827fc3 to retry 502 errors when running inside a DNAnexus job.

Improved bulk lookup

• Use /system/findDataObjects route for bulk object describe calls, this significantly improves performance
  when the user has access to many projects.

Re-adding support for the `uid` and `gid` command line flags

• Re-adding support for the uid and gid command line flags.
• Allowing the older use of mounting with sudo, although, this is discouraged. The new command:

dxfuse MNT your_project

The older method:

sudo dxfuse -uid $(id -u) -gid $(id -g) MNT your_project

Eliminate sudo

• Eliminate the use of sudo for starting up the filesystem. Normal user permissions are now sufficient to start and stop the filesystem. This assumes that the fuse device (/dev/fuse) is open for read/write access to regular users.
• Use github actions for continuous integration (CI/CD)
• Improved explanation for why mmap needs to use PRIVATE mappings.

Bug fix release

• Fixed bug that occurs when a data object has properties or tags that include the apostrophe (`) character.
• Simplified the filesystem start-up mechanism.

Better upload performance

• Improved upload performance and added a benchmark for it. The implementation section includes a table comparing dxfuse upload to dx upload. The main bottleneck for FUSE is that IOs are passing through the kernel in small synchronous chunks (128KB). This means that large file copies are slow.
• Upgrade to golang version 1.14
• Fixed use of defer when there are several such statements inside one function.
• Fixed documentation for manifest mode, which is in the expert-options.

Overwriting files

• Experimental support for overwriting files. There is a limit of 16 MiB on a file that is undergoes modification. This is because it needs to first be downloaded in its entirety, before allowing any changes. It will then be uploaded to the platform. This is an expensive operation that is required because DNAnexus files are immutable.

• Removed support for hard links. The combination of hard-links, cloning on DNAx, and writable files does not work at the moment.

• Improvements to extended attributes (xattrs). The testing tool we use is xattr, which is native on MacOS (OSX), and can be installed with sudo apt-get install xattr on Linux. Xattrs can be written and removed.

Tags and properties are namespaced. For example, if zebra.txt is a normal text file with no DNAx tags or properties then xattr -l will print out all the tags, properties, and extra attributes that have no POSIX equivalent. This is split into three namespaces: base, prop, and tag.

$ xattr -l zebra.txt

base.state: closed
base.archivalState: live
base.id: file-Fjj89YQ04J96Yg3K51FKf9f7

Add a property named family with value mammal

$ xattr -w prop.family mammal zebra.txt

Add a tag africa

$ xattr -w tag.africa XXX zebra.txt

Extended Attributes

• Showing archived and open files; these were previously hidden. Trying to read or write from an archived
  or non-closed file will cause an EACCES error.
• Presenting file properties and tags as extended attributes. To list all attributes for a file you can do:

$ getfattr -d -m - FILENAME

The getattr utility is part of the ubuntu attr apt package. It can be installed with:

$ sudo apt-get install attr