Technical documentations

[thoughtsunificator]

Hi,

Other than doc/Adapters.md there's not much documentation on the technical side of things.

I'd be grateful if you could provide some technical documentation.

Also, I see that there are some (user) guides on floccus.org which are not available when cloning the repo, unfortunately, so I'd recommend keeping all the documentation in the same place, on this repository, preferably on the doc/ folder which already serves this purpose. The documentation could be split into both a user and developer manual in doc/ and build floccus.org off the user manual.

Thank you

[marcelklehr]

Hey @thoughtsunificator

Good idea! To what end would you like to read such a technical documentation? As a guide for contributing, or as a description of how floccus works for users?

You can find the website source code here: https://github.com/floccusaddon/nuxt-website

[thoughtsunificator]

Hi @marcelklehr,

Actually both, I need to be aware of both the user and developer side of things. I'm actually making a bookmark managing system, and it looks like some parts of floccus are exactly what I need to complete its user experience, actually.

Having a technical documentation would help a lot, not only as a guide for contributing, but also to verify that the technical choices are the right fit for my plans.

My hope to provide some kind of "core", the core of floccus, that could be plugged pretty much anywhere with little to no assumptions about the stack.

Thanks

[jhgit]

I will add that I am interested from a potential contributor point of view.

For example, having a standalone .xbel decrypter tool would be nice.

And I would like to understand how the profile is stored on the various platforms to investigate storing that encrypted when saved to media - I think it would be nice to not have the passphrase for an encrypted .xbel file in plain text (either in the exported .json profile or in the addon's internal storage format). This might necessitate user entry of a passphrase / password before syncing can occur, of course.

So maybe a bit about source code organization and architecture and debugging techniques. Just a nudge would be fine. Others can add to the docs to fill in details perhaps.

[marcelklehr]

I'd be very happy to accept contributions from you 🎉

For decrypting stuff, here's Crypto implementation: https://github.com/floccusaddon/floccus/blob/develop/src/lib/Crypto.ts

And I would like to understand how the profile is stored on the various platforms to investigate storing that encrypted when saved to media - I think it would be nice to not have the passphrase for an encrypted .xbel file in plain text (either in the exported .json profile or in the addon's internal storage format). This might necessitate user entry of a passphrase / password before syncing can occur, of course.

I'm not sure if this will be useful. Encryption at rest is a laudable goal, but I'm afraid this will complicate things both from a dev and from a UX perspective. The benefit would be to protect users from stolen credentials although, I would argue, if the attacker has access to the storage where credentials are stored they already have access to the device and can install all sorts of malware to intercept the same credentials were they encrypted.

[jhgit]

Here's a use case for encryption (not the only one)... I want to import my profiles, so I stick it on a USB drive. I lose the USB drive on the way home (or whatever). Plus if I have an encrypted profile(s), I can put that on my cloud storage, so I don't have the same sync problem I have with bookmarks all over the place (i.e., trying to sync more than one floccus profile). BUT... there's alway gnupg for protecting the .json (or tarball of multiple profiles) on the cloud storage, I suppose. It's not that often I'll have to do imports (although we'll see - I've only just recently started using floccus).

I know what you're saying regarding someone having access to my storage, but it's a bar to hurdle. And it helps with casually nosy admins (but not the truly determined ones, of course, as you point out).

But we digress from the topic of this post...

[marcelklehr]

Let me write down a bit of an intro to the source:

The core of floccus is in https://github.com/floccusaddon/floccus/tree/develop/src/lib Especially interesting to get an overview are probably the interfaces: https://github.com/floccusaddon/floccus/tree/develop/src/lib/interfaces :

There's the Account interface which describes what in the UI is now called a profile. AccountStorage describes the storage layer. Adapter is the interface that all backend adapters implement, along with Resource, which is the description of anything that holds bookmarks ie. an adapter or the browser bookmarks or the browser tabs or the bookmarks storage for the native app. And then there's the serializer interface which describes objects that take bookmarks folders and serialize them, and can also deserialize that representation again into a proper folder tree.

The meat of all sync operations is Bookmark and Folder objects of course, you'll find these here: https://github.com/floccusaddon/floccus/blob/develop/src/lib/Tree.ts (should maybe move these into separate files). These are combined into an Item Type. What's interesting to note is also that since v5 all items also have a location, which is either 'server' or 'local'. This is handy to avoid using client-side IDs on the server and vice versa.

For syncing, we run a Scanner over the two trees to create a diff which is a glorified array of actions which, when applied, transform tree 1 into tree 2. In order for all of this stuff to work, we need to map between IDs of items on the server and IDs of items on the client, which is what https://github.com/floccusaddon/floccus/blob/develop/src/lib/Mappings.ts is for. The whole sync logic is encapsuled in three strategies (using the strategy pattern). In there, there's a crude implementation similar to operational transformation that transforms actions from the two trees against each other to produce a new diff that can be applied to the opposite tree in a sane way to produce a synced state.

The logic specific to the native apps can be found in https://github.com/floccusaddon/floccus/tree/develop/src/lib/native while the logic specific to browsers is in https://github.com/floccusaddon/floccus/tree/develop/src/lib/browser .

The User interface is in https://github.com/floccusaddon/floccus/tree/develop/src/ui

Tests are in https://github.com/floccusaddon/floccus/blob/develop/src/test/test.js

[thoughtsunificator]

Thanks for sharing.

Another example of an information, which I believe is missing, is from a user perspective is the need to install a browser extension to enable the syncing of bookmarks. Simply explaining what the extension does in background would be great, I think.

I know there's a lot of documentation already, but, if you can see any value in that, I'd be glad to compile it and add what I believe is missing.

Edit: Just looking at the issues I managed to find one that mention an error "E017", maybe there could be a table of error codes with a description and steps that explain how to fix the error.

[marcelklehr]

Another example of an information, which I believe is missing, is from a user perspective is the need to install a browser extension to enable the syncing of bookmarks. Simply explaining what the extension does in background would be great, I think.

Mh. I don't understand. Can you elaborate?

maybe there could be a table of error codes with a description and steps that explain how to fix the error.

Each error code is accompanied with a description in the language of the user that explains what the problem is. I think if anything, these messages should be improved rather than creating a separate resources for this. :)