VFS on Linux

[tobiasKaminsky]

VFS on Linux needs to have a proper support on KDE/Gnome first, so first step would be to create and commit such a RFC.
Then monitor it, so that we can afterwards implement it.

As many Linux users also use CLI, maybe we can also enhance our nextcloud to sync/upload/download, e.g.:
nextcloudcmd myVirtualImage.png.nextcloud -> downloads and opens it

[vasyugan]

For consistency, VFS support on Linux should not be bound to any specific desktop environment. A situation where nautilus or dolphin show some file as present yet command line utilities or even graphical programmes that don't use the GNOME/KDE libraries don't would be rather odd and irritating.

[FlexW]

@vasyugan @tobiasKaminsky At the moment it is planned to register a custom xattr and read that when opening files. That would still need support from the program that reads the file, but it would not be bound to GNOME or KDE. A real vfs like on Windows and macOS would need support from the kernel side I think.

[johnnyasantoss]

pCloud does an excellent job at using virtual fs to browse/sync files. I think that it would be a great addition to the Nextcloud Linux Client if it could do the same.
They created their own fs it might be a good reference https://github.com/pcloudcom/pclsync

[vasyugan]

A real vfs like on Windows and macOS would need support from the kernel side I think.

Couldn't such a thing be implemented using FUSE? This way, you would not have to get code into the kernel, I guess. In other words: Isn't that exactly what FUSE can do for you?

[mgallien]

A real vfs like on Windows and macOS would need support from the kernel side I think.

Couldn't such a thing be implemented using FUSE? This way, you would not have to get code into the kernel, I guess. In other words: Isn't that exactly what FUSE can do for you?

FUSE is not a good solution when network is involved because the normal file API you end up using to access FUSE filesystems is not able to cope with network errors.
Currently we have an experimental VFS feature on all platforms that is using some suffix appended to files when they are virtual empty placeholders.
You can activate it by manually editing your configuration file ($HOME/.config/Nextcloud/nextcloud.cfg) adding showExperimentalOptions = true in the first section.
You can then use the settings dialog to enable it.
Keep in mind that this experimental and not considered stable enough for daily usage.
Reports are more than welcome (bugs, performance issues, suggestions, ...).

[AditiThirdEye]

Hello @FlexW , @mgallien ,
If the PR has not been done yet, can I work on this thread?

Regards,
Aditi

[mgallien]

Hello @FlexW , @mgallien ,
If the PR has not been done yet, can I work on this thread?

Regards,
Aditi

Hello Aditi feel free to contribute by opening a PR
you can find some documentation on our workflow here wiki
especially the page about Pull requests

[tennox]

For reference, here's a related fuse prototype for Owncloud:
https://github.com/jnweiger/OCFFS

(also, for anyone searching for virtual file support, check this)

[vasyugan]

Currently we have an experimental VFS feature on all platforms that is using some suffix appended to files when they are virtual empty placeholders.

I suppose that this only works with GTK/QT applications, right? I definitely would need something that also works in the shell. I very often find that I can get things done quicker and with more ease in the shell.

[KaeTuuN]

@mgallien @tobiasKaminsky

1. Is there any (Dev) Documentation on how this feature Works on Linux?
2. Is there any Roadmap for this feature?
3. How can I support here with tests?

My employer is searching for a solution like nextcloud, but a must have is a VFS for Mac and Linux.

[tobiasKaminsky]

Currently there is no such thing for Linux.
First we implemented it on Windows and then we want to focus on MacOS, as there is a proper cloud API already.
Then we will start with Linux

[sinni800]

When I enable the experimental support that is there, it crashes when i hit the "enable virtual file support" button.
The application segfaults and the last line in the log is this:

2022-01-08 10:10:33:653 [ warning default unknown:0 ]: QObject::connect(QDialog, OCC::AccountSettings): invalid nullptr parameter

[aviallon]

Using FUSE for such a feature is clearly the right thing to do here, in my opinion.

[Cloud-Kid]

Any update on this ? I use Arch + Gnome and the client really lack of this feature :(

[vasyugan]

Using FUSE for such a feature is clearly the right thing to do here, in my opinion.

As @mgallien writes, fuse is unable to cope with network errors, so it would need to be fixed first.

[aviallon]

Using FUSE for such a feature is clearly the right thing to do here, in my opinion.

As @mgallien writes, fuse is unable to cope with network errors, so it would need to be fixed first.

Correct me if I'm wrong, but I think kio-fuse is able to cope with network errors, and actually implements webdav support.
Couldn't we look at what they currently do?

[SuperSandro2000]

I did a little research and at least for dolphin/kio the xattr support does not seem to be great. Also there is no universal xattr support so many tools and programs seem to strip them on save.

ref https://userbase.kde.org/Baloo

[hardwareadictos]

Is showExperimentalOptions still available on v3.4.4? I'm on Fedora 36 but i am not able to see it under settings rear adding the line on the config file and restarting.

[mgallien]

Is showExperimentalOptions still available on v3.4.4? I'm on Fedora 36 but i am not able to see it under settings rear adding the line on the config file and restarting.

yes should still work as we did not change anything there

[hardwareadictos]

Is showExperimentalOptions still available on v3.4.4? I'm on Fedora 36 but i am not able to see it under settings rear adding the line on the config file and restarting.

yes should still work as we did not change anything there

¿Maybe a Fedora issue? On F35 it was working fine.

[tennox]

¿Maybe a Fedora issue? On F35 it was working fine.

Same for me - my solution was to switch from flatpak version to AppImage

Works now :)

[yajo]

If using flatpak, the file can be found in ~/.var/app/com.nextcloud.desktopclient.nextcloud/config/Nextcloud/nextcloud.cfg

[delete-your-github-today]

Please implement this as a FUSE filesystem. We need full compatibility with all VFS features. It's certainly possible to do this despite network delays/errors. rclone's FUSE support with its full caching mode is already effectively the functionality I want, it would just be nice to have this built-in to the nextcloud client.

[BloodyIron]

Not a current blocker for me but I foresee $futureMe caring about this significantly, as well as my clients. Here's hoping this can come to fruition and be generally available as a default ;)

[flamingm0e]

I still don't understand why this is still an issue. Owncloud has it. I'm running NC and OC in parallel and OC just has the better client overall.

[r2evans]

@flamingm0e have you experienced any issues running the OC desktop client against an NC server (or vice-versa)?

[flamingm0e]

@flamingm0e have you experienced any issues running the OC desktop client against an NC server (or vice-versa)?

Sorry. I didn't mean to insinuate I was running both clients against a single NC server. I have both servers running:
NC AIO and OCIS.

[ados8]

I still don't understand why this is still an issue. Owncloud has it. I'm running NC and OC in parallel and OC just has the better client overall.

Yeah it's a shame, open source software that ironically runs the best on proprietary commercial software that is Windows.
Further to the irony it appears Mac OS well get support first, for now we dream.

[Fuseteam]

well if OC has it, and the code is open source, then someone from the community could open a pull request to add to NC, no?

any reference to how OC does it on, or since when?

[Fuseteam]

hmmm as far as i can find owncloud uses placeholder files with a custom extension on mac and linux: https://github.com/owncloud/client/wiki/Virtual-Files

I presume that they replace this file once the user actually tries to open the file 🤔

[antrmn]

I presume that they replace this file once the user actually tries to open the file 🤔

That's basically how it currently works using the beta branch of Nextcloud Client on Linux.

A suitable example may be onedriver which basically uses FUSE, but as you can see in its README (see Known Issues) it does not integrate well with file managers and has a few of rough edges, so I can't blame NC's developers for wanting a better solution.

Unfortunately it appears that no steps have been made in estabilishing a proper "Cloud Files API" for the Linux Desktop

[Fuseteam]

Sounds like we need an standard API on linux for it, tho i'm not sure is this would be portal or a protocol issue 🤔

EDIT: Ok actually looking at the known issues of onedriver, it seems to be a GNOME issue.........and a microsoft issue
So to me it sounds like an issue should raised against GNOME's nautilus—

[johnjaylward]

Would it make sense to use a project like GDFS as a baseline and implement a similar FUSE base filesystem that would work similarly? ... Not that I know if this would work or not. Just saw it may be an option.

Preferably, reliance on systemd would be optional

[jthoward64]

Don't know whether this will help anyone, but I was able to configure my Nextcloud and Google Drive in a different way which I find works much better than using the Nextcloud Desktop (even with experimental features enabled).

1. Make sure rclone is installed on your machine

2. Run 'rclone configure' and configure your Nextcloud drive under the WebDAV section (Nextcloud is one of the choices of the menu that follows)

3. Create an empty folder to mount the rclone drive (in this case, mkdir /home//Nextcloud)

4. Run this or even make an sh file to run at startup:
   `rclone mount  <nameofyourmount>: /home/<user>/Nextcloud --daemon --vfs-cache-mode full --vfs-cache-max-age 12h`

This will mount your Nextcloud (as configured in step 2) within the folder you've created. The cache mode is excellent and there are many more options for customizing the command itself.

Honestly, I wonder if just adding an optional rclone dependency and automatically managing rclone mounts would be the way to do this for Linux

[Fuseteam]

that does look promising: https://rclone.org/commands/rclone_mount/#vfs-virtual-file-system
rclone even supports macos and windows too

[Lichtblitz]

rclone is an entirely different use case. It may fit many people and will not fit many more. For example rclone caches until the cache expires. It will not sync changes from the cloud during that time. If collaborating with others this may quickly become a deal breaker for offline availability.
Secondly, there is no option for on demand sync. I have devices that wouldn't even fit my whole cloud storage if I wanted to. Plus the plethora of unnecessary syncing activity.
Microsoft implemented a close to perfect solution with the API they introduced directly in Windows. OneDrive uses the same API with the same great and seamless integration, that third parties use as well. Dropbox, Box, Google drive, etc.
Having a similar API baked into Linux without applications and users having to know about special files and providers would be awesome.

[phirestalker]

rclone is an entirely different use case. It may fit many people and will not fit many more. For example rclone caches until the cache expires. It will not sync changes from the cloud during that time. If collaborating with others this may quickly become a deal breaker for offline availability. Secondly, there is no option for on demand sync. I have devices that wouldn't even fit my whole cloud storage if I wanted to. Plus the plethora of unnecessary syncing activity. Microsoft implemented a close to perfect solution with the API they introduced directly in Windows. OneDrive uses the same API with the same great and seamless integration, that third parties use as well. Dropbox, Box, Google drive, etc. Having a similar API baked into Linux without applications and users having to know about special files and providers would be awesome.

According to the documentation at https://rclone.org/commands/rclone_mount/#vfs-cache-mode-full it uses sparse files. This means it will not download the whole file and only the parts of it that are needed. You can also set a short caching time to save disk space. There is even a mode with no caching, although some applications can not work with this mode.

Cache expiration is configurable. It will even poll for changes at a configurable interval. You can make it 1s. On demand sync is not necessary as it is polling for changes and syncing them constantly.

I am unsure of the syncing activity. Since the folks working on Linux do not seem too interested in creating a universal VFS implementation, rclone can be a good alternative. I would be grateful if Nextcloud would consider using rclone in the background until Linux implements the necessary API.

Currently, I use the built-in WebDAV functionality in KDE. Unfortunately, this is not available in any non-KDE apps or from the terminal. I would have used rclone if this functionality was missing from KDE.

[Fuseteam]

Secondly, there is no option for on demand sync. I have devices that wouldn't even fit my whole cloud storage if I wanted to.

that is actually what vfs-cache-mode full is with rclone, it does not download the actual the data, just the metadata needed to download the data on demand

Since the folks working on Linux do not seem too interested in creating a universal VFS implementation, rclone can be a good alternative.

if i'm reading/understanding this correctly it seems the universal API for this on linux is FUSE, rclone is using that under the hood. if rclone isn't what we want, then an iniative for vfs solution on top of fuse would need to be started. If FUSE is unfit, then some from scratch filesystem would developed for the linux kernel, which is likely a multiyear endeavor

in both cases we'd need stakeholders, nextcloud developers/users, proton developers/users and others to come together and create the filesystem we need. That's the way things move in the linux world— there is no "they", only "we"

[lachlanshoesmith]

When I enable the experimental support that is there, it crashes when i hit the "enable virtual file support" button. The application segfaults and the last line in the log is this:

2022-01-08 10:10:33:653 [ warning default unknown:0 ]: QObject::connect(QDialog, OCC::AccountSettings): invalid nullptr parameter

did anyone end up figuring out what causes this? having this problem on 3.14.0daily on my laptop, dont on my similarly-setup desktop for some reason. running arch with hyprland fwiw, can reproduce using the same steps of adding enableExperimentalOptions=true to nextcloud.cfg and then trying to enable virtual files support in the client

edit: 3.12.2git (2:3.12.2-1) seems to work.

[antrmn]

Maybe openVFS could be considered as a project proposal for initiatives like Outreachy Internships or Google Summer of Code? GNOME was selecting project proposals for Outreachy just a month ago.

This could provide an opportunity for openVFS to be explored further

[thibaultmol]

I'm also kind of confused why the current virtual file system implementation that you can enable using showExperimentalOptions is still so hidden away. If there are bugs, then what are the bug reports/issue numbers that are blocking it from being rolled as a stable feature?

(I think some of the above mentioned solutions would also be nice, but i'm mainly wondering why the current implementation hasn't shipped as stable for years now)

[bluefoxconsultant]

Personally I've never been able to make the experimental VFS work.
It created placeholder files as I expected it to, however upon trying to open those files, nothing happened (tried through the Open menu in applications as well as launching the file directly from Dolphin).
I've found rclone to generally be a vastly superior solution, although there are some slowdowns when using Google Drive (especially when listing directories and files).

[woiwoiwoiwoi]

For me, it's been working flawlessly for 2-3 years on Debian and OpenSUSE Tumbleweed. I've also wondered why it's still so hidden away.

[langfingaz]

For me, it's been working flawlessly for 2-3 years on Debian and OpenSUSE Tumbleweed. I've also wondered why it's still so hidden away.

What desktop environment are you using? I assume the support varies between different DEs and if one is using QT, GTK or other applications.

[Fuseteam]

For me, it's been working flawlessly for 2-3 years on Debian and OpenSUSE Tumbleweed. I've also wondered why it's still so hidden away.

i believe the comment above yours answers that question

[r2evans]

It created placeholder files as I expected it to, however upon trying to open those files, nothing happened (tried through the Open menu in applications as well as launching the file directly from Dolphin).

@bluefoxconsultant

This might be due to something external to NC. I suggest looking into how default-applications are defined in your distro to see if .nextcloud is correctly defined. I don't know how most distros do it, but after strace-ing mimeopen .nextcloud, I found it looking at /usr/share/mime/globs which includes

...
image/x-3ds:*.3ds
audio/x-ms-wma:*.wma
application/vnd.nextcloud:*.nextcloud
text/x-python3:*.py
application/vnd.lotus-1-2-3:*.123
...

which should match the placeholder with the associated mime type application/vnd.nextcloud. That type is matched to an executable via /usr/share/applications/mimeinfo.cache (or a ~/.local/share/applications or several other places to find multiple mimeinfo.cache files), where

...
application/vnd.ms-wpl=vlc.desktop;
application/vnd.ms-xpsdocument=org.gnome.Evince.desktop;
application/vnd.nextcloud=com.nextcloud.desktopclient.nextcloud.desktop;
application/vnd.oasis.opendocument.base=libreoffice-base.desktop;
application/vnd.oasis.opendocument.chart=libreoffice-calc.desktop;

And that points to /usr/share/applications/com.nextcloud.desktopclient.nextcloud.desktop (ditto on ~/.local/share/...), which ultimately points to the Executable that is called when something tries to open a *.nextcloud file.

[Desktop Entry]
Categories=Utility;X-SuSE-SyncUtility;
Type=Application
Exec=nextcloud %u
Name=Nextcloud Desktop
Comment=Nextcloud desktop synchronization client
GenericName=Folder Sync
Icon=Nextcloud
Keywords=Nextcloud;syncing;file;sharing;
X-GNOME-Autostart-Delay=3
MimeType=application/vnd.nextcloud;x-scheme-handler/nc;
Actions=Quit;

(Long story, thanks for walking all of it with me.)

So if you can verify that all of that is correctly defined, and that something like mimeopen somefile.docx.nextcloud works up until the point that nextcloud %s is attempted, then the problem is certainly with the NC desktop utility. If it doesn't get that far, then the linux-hack for VFS (.nextcloud file extension) may not be why it is not working on your system.

[woiwoiwoiwoi]

What desktop environment are you using? I assume the support varies between different DEs and if one is using QT, GTK or other applications.

Gnome.

[MrRinkana]

A workaround for KDE users is adding the nextcloud account to dolphin (system settings -> accounts) as that provides a way to get "virtual" files, although it seems to work more akin to a network share, such that files always get downloaded (possibly some temp. caching) rather than being stored locally after opening them once. It has some benefits like easily uploading folders etc. This is what I do since:

For me the nc-client just crashes when trying to enable the experimantal-VFS, I don't see any syck option during install or adding a sync connection, I can only add it afterwards by clicking the three dots on an existing sync-connection (and then it crashes).

[antrmn]

My previous comment:

Maybe openVFS could be considered as a project proposal for initiatives like Outreachy Internships or Google Summer of Code? GNOME was selecting project proposals for Outreachy just a month ago.

This could provide an opportunity for openVFS to be explored further

I reiterate my previous suggestion, this time bringing into consideration the following article: GNOME STF 2025: Call for Ideas and Contractors

We’re currently putting together a Sovereign Tech Fund application for next year, both collecting ideas for projects and investigating what we’ll be able to staff. We’ve already got a list of focus areas and ideas that fit, and we’re in the process of having meetings with various teams of maintainers across the project. However, we also want to open this up to the wider community.

Our focus areas are:

[...]
Platform features in GNOME, Freedesktop, Flatpak, portals, and related projects. For instance: xdg-intents, various Flatpak portal improvements, session save/restore, notifications, doze/wakeup/power management, search, text-to-speech, autofill, etc.
[...]

If you have any ideas in these areas, and/or would be available to work on them on a contract basis, please contact us (stf@gnome.org), or leave a comment below.