From df373c7b95eed2cf84fbb80876fa68928084564e Mon Sep 17 00:00:00 2001 From: Ivan Ivanov Date: Thu, 10 Oct 2024 14:08:15 +0300 Subject: [PATCH 1/2] Improvements over actions available on the layer for QFieldCloud What I have realized is that we are missing a structural description how the whole thing is supposed to work. Main pain points: - QGIS project is described multiple times in the docs - layer actions are described multiple times in the project --- .../reference/qfieldcloud/projects.en.md | 39 +++- .../reference/qfieldcloud/system.en.md | 175 +++++++++--------- 2 files changed, 123 insertions(+), 91 deletions(-) diff --git a/documentation/reference/qfieldcloud/projects.en.md b/documentation/reference/qfieldcloud/projects.en.md index e40b37725..c9ece0469 100644 --- a/documentation/reference/qfieldcloud/projects.en.md +++ b/documentation/reference/qfieldcloud/projects.en.md @@ -3,19 +3,34 @@ title: Projects tx_slug: documentation_reference_qfieldcloud_projects --- -Projects are the main data containers on QFieldCloud. Users can create any number of projects. Projects must contain a single `.qgs`/`.qgz` QGIS file, and may in addition contain any combination of geospatial files -- GeoPackages, Shapefiles, TIFs -- or data files such as photos, PDFs etc. Files cannot be shared between projects. +Projects are the main data containers on QField and QFieldCloud. +Users can create any number of projects. +Projects must contain a single `.qgs`/`.qgz` QGIS file, and may in addition contain any combination of geospatial files -- GeoPackages, Shapefiles, TIFs -- or data files such as photos, PDFs etc. +Files cannot be shared between projects, unless [localized layers](../../how-to/outside-layers.md) are used. -QFieldCloud projects have a name and an owner. The owner of a project is a QFieldCloud user or an organization. No two projects can use the same pair . +QFieldCloud projects have a name and an owner. +The owner of a project is a QFieldCloud user or an organization. +No two projects can use the same pair `owner_name` and `project_name`. + +Projects can be marked as either public or private. +Private projects are accessible only to users added to a project as project collaborators. +Public projects are visible to, and can be downloaded by, any QFieldCloud user. -Projects can be marked as either public or private. Private projects are accessible only to users added to a project as project collaborators. Public projects are visible to, and can be downloaded by, any QFieldCloud user. ## Creating a project -A project can be created in two different ways: either using the QFieldCloud web interface or using QFieldSync in QGIS. +A project can be created in multiple ways: +- via QFieldCloud web interface; +- via QFieldSync in QGIS; +- via QFieldCloud-SDK; + ## Files -Files are the skeleton on which QFieldCloud project works. To make a QFieldCloud project alive users need to upload at least a single QGIS project file in the `.qgs` or `.qgz` file formats. All geospatial files must be uploaded using the same relative paths as on one's computer. If external SVG or raster symbology is used, users must upload the corresponding files too. +Files are the skeleton on which QFieldCloud project works. +To make a QFieldCloud project alive users need to upload at least a single QGIS project file in the `.qgs` or `.qgz` file formats. +All geospatial files must be uploaded using the same relative paths as on one's computer. +If external SVG or raster symbology is used, users must upload the corresponding files too. !!! note QFieldCloud does not support projects stored in a GeoPackage (`.gpkg`) files (but users can still use GeoPackage files to store datasets for their projects). @@ -34,9 +49,21 @@ project │ ├── bees-20220404121212.jpg │ ├── bees-20220405040506.jpg │ └── fields-20220405040607.jpg -└── project.qgs +├── project.qgs +├── project_attachments.zip +└── project.qml ``` + +The file in a QGIS project can be in one of the following groups by their purpose: + +- **QGIS project file** - a `.qgs` or `.qgz` project file. +- **QGIS sidecar files** - the utility files to the QGIS project file, such as `*_attachments.zip` or other sidecar files. +- **Data source files** - all your vector and raster data, such as `.gpkg`, `.tif`, `.mbtiles` or other data source files. +- **Attachments** - all your additional project data, such as `.jpg`, `.pdf` or other files. +- **QField plugins** - all your QField plugins, usually `.qml` files. + + ## File versions QFieldCloud uses file versioning. This allows users to restore to a previous version of any modified file. Files and file versions can be found under the **Files** section of one's projects. Subscriptions plans allow a different number of versions per file. See the qfield.cloud [pricing page for further details](https://qfield.cloud/pricing.html). diff --git a/documentation/reference/qfieldcloud/system.en.md b/documentation/reference/qfieldcloud/system.en.md index 91cedb313..a55418246 100644 --- a/documentation/reference/qfieldcloud/system.en.md +++ b/documentation/reference/qfieldcloud/system.en.md @@ -4,102 +4,107 @@ tx_slug: documentation_reference_qfieldcloud_system --- # QFieldCloud System Documentation - The aim of this document is to provide an overview of the QFieldCloud system to - understand the underlaying logic and technology. + +The aim of this document is to provide an overview of the QFieldCloud system to understand the underlaying logic and technology. + ## Entities and Concepts + ### QGIS Project -A QGIS project is a *.qgs* or *.qgz* file. A Project is created on -QGIS Desktop and uploaded to QFieldCloud using the QGIS's plugin -QFieldSync. Before the uploading of the QGIS project, it is necessary -for each layer of the QGIS project an "action" that determines how -QFieldSync and QField should treat the layer. There are the two types -of actions that can be setup - one for QFieldCloud and one for the -traditional cable export. - -This information is saved within the QGS project as layer's -*customProperty*, with the *QFieldSync/action* key. - -The available actions are: - -| Action internal name | Name showed in the UI | -|----------------------|-----------------------| -| OFFLINE | Consolidate | -| NO_ACTION | Live layer | -| REMOVE | Ignore layer | -| COPY | Copy | -| KEEP_EXISTING | Keep Existing | - -This would be the behavior of QFieldSync with the different -layer actions: - -| Action | File based layer | Not file based layer | -|---------------|------------------------------------------------------------------|----------------------------------------| -| OFFLINE | Create a consolidated copy of the data | Create a consolidated copy of the data | -| NO_ACTION | N/A | No action on the layer | -| REMOVE | Remove the layer from the project | Remove the layer from the project | -| COPY | Make source path relative and copy the file | N/A | -| KEEP_EXISTING | Make source path relative and copy the file if it does not exist | N/A | - - This is the behavior of QFieldCloud (`libqfieldsync`) with the - layers: - -| Action | File based layer | Not file based | -|---------------|-----------------------------------------------------------------------------------------------------------------|------------------------------------------------------------------------------------------------| -| OFFLINE | Create consolidated copy of the data on pull, apply delta file on push to original data source | Create consolidated copy of the data on pull, apply delta file on push to original data source | -| NO_ACTION | N/A | No action on the layer | -| REMOVE | Remove the layer from the project | Remove the layer from the project | -| COPY | Make source path relative and create copy of the data on pull, apply delta file on push to original data source | N/A | -| KEEP_EXISTING | Make source path relative and create copy of the data on pull, apply delta file on push to original data source | N/A | - -This is the behavior of QField with the layers: - -| Action | File based layer | Not file based layer | -|---------------|------------------------------------|--------------------------------------------| -| OFFLINE | Create and push deltafile | N/A (it's always file based at this point) | -| NO_ACTION | N/A | Edit the online (live) database | -| REMOVE | N/A (the layer is no longer there) | N/A (the layer is no longer there) | -| COPY | Create and push deltafile | N/A | -| KEEP_EXISTING | Create and push deltafile | N/A | - -In summary, for with QFieldCloud: - -- *NO_ACTION* is used for online layers that are located on a server - accessible via the Internet and that are modified directly by - QField. -- *HYBRID* means that a geopackage will be generated on the - server (or directly on the desktop for file-based layers) and - downloaded by clients. The client will generate deltafiles of the changes. -- *OFFLINE* is used for example to work with local databases not - visible by QFieldCloud which are consolidated before being - loaded from the desktop to the server and are not synchronized - with the original data by QFieldCloud. -- *REMOVE* will simply remove the layer from the project. -- *KEEP_EXISTENT* will not be used for QFieldCloud syncronizations. - -From QFieldSync it will be possible to update a project already -loaded on QFieldCloud. In the event that the changes concern only -styles, forms etc. but not the structure of the layers, the -project on the server will simply be updated. -If there are changes in the layers structure, the project will be -reset on the server (delta files will be deleted) and for each -client it will be necessary to download the updated version of the -project before being able to push new changes. - -### QFieldCloud Project - Is composed of one and only one QGIS project and the possible - related files (e.g. geopackages, images, ...) included the offline - or hybrid data package. + +In QFieldCloud context a QGIS project refers to all data files that are required for properly functioning QGIS project. +Read more about [QGIS projects](./projects.md). + +### Layer action + +Each layer in the QGIS project can be configured with an "layer action". +The action determines how QFieldSync and QField should treat the layer. + +There are the two actions that can be configured: cloud action and cable action. +They work are applied in the QFieldCloud and traditional cable export context respectively. + +The following actions are available and will be explained in more detail below: + +| Name showed in the UI | Work mode | Spatial type | +|---------------------------------|---------------|--------------| +| Offline editing | cloud & cable | Vector | +| Directly access data source | cloud & cable | Any | +| Remove from project | cloud & cable | Any | +| Copy | cable | Any | +| Keep existing (Copy if missing) | cable | Vector | + + +#### Cloud action configuration + +The cloud action set in QFieldSync is applied by QFieldCloud only at the moment of packaging a project for QField. + +This is the behavior of QFieldCloud (`libqfieldsync`) with the layers: + +| Action | File based layer | Service-based layer (e.g. WMS) | Database layers (Postgres) | +|-----------------------------|------------------------------------------------------------------------------------------------|-----------------------------------|--------------------------------------------------------------------------------------------------| +| Offline editing | Create consolidated copy of the data on pull, apply delta file on push to original data source | N/A | Create a consolidated copy of the data on pull, apply delta file on push to original data source | +| Directly access data source | Create a read-only copy in an individual GeoPackage | No action on the layer | No action on the layer | +| Remove from project | Remove the layer from the project | Remove the layer from the project | Remove the layer from the project | + +In summary, with QFieldCloud: + +- *Offline editing* means that a offline copy of the data will be generated by QFieldCloud and downloaded to QField. +The data from *Offline editing*-layers will all be stored as multiple layers within a single `data.gpkg`. +Whenever a modification is made on QField, a JSON structure called a *Change* (or Delta) will be generated only for the features that have been modified, only for the attributes or the geometry that have been modified. +On push, the original data source will be updated, including the database for database layers. +- *Directly access data source* is mainly used for service-based layers that are located on a internet accessible server. +The data in these layers are directly modified by QField. +Examples are WFS, WM(T)S-layers or layers coming from a database such as PostGIS layers. +If the layer is file-based it will be in read-only mode. +- *Remove from project* will simply remove the layer from the project (not package it for QField). + +From QFieldSync it will be possible to update a project already loaded on QFieldCloud. + +In the event that the changes concern only styles, forms etc. but not the data or the structure of the layers, the QGIS project file (`.qgs`/`.qgz`) on the server will simply be updated. + +If there are changes in the data or the structure of the layers, then the new data source files will be updated. + +Note that changing the structure (or schema) of the layers such as adding, removing or renaming attributes, changing not null constraints, etc. might have effects. +If such changes are pushed to QFieldCloud, please make sure all QField users have pushed their changes in advance. +Otherwise it might be impossible for QFieldCloud to apply these Changes/Deltas from older version of the QGIS project. + + +#### Behaviour of QField + +Depending on the actions set for each layer in QFieldSync for QFieldCloud or Cable export, QField will act as follows: + +| Action | File based layer | Service-based layer (e.g. WMS, database) | Notes | +|---------------------------------|---------------------------|--------------------------------------------|--------------------------------------| +| Offline editing | Create and push a Change | N/A | | +| Directly access data source | Layer is readonly | Edit the online online database | | +| Remove from project | N/A | N/A | the layer is not available on QField | +| Copy | Create and readonly | N/A | | +| Keep existing (Copy if missing) | Create and readonly | N/A | | + + +### Technical names for actions + +The technical names for the available actions in QFieldSync are: + +| Name showed in the UI | Action internal name | +|---------------------------------|----------------------| +| Offline editing | OFFLINE | +| Directly access data source | NO_ACTION | +| Remove from project | REMOVE | +| Copy | COPY | +| Keep existing (Copy if missing) | KEEP_EXISTING | + + ## Use Cases -### Hybrid +### Offline editing in the field, QFieldCloud connected to the database Hybrid editing mode with synchronization on the server !![Hybrid editing mode](../../assets/images/hybrid-schema.png) -### Offline database +### Offline editing in the field, QFieldCloud not connected to the database Offline editing mode with desktop synchronization !![Offline editing mode](../../assets/images/offline-schema.png) From d472ecdbdca99d95895fcec2a120f3f9a090a725 Mon Sep 17 00:00:00 2001 From: Ivan Ivanov Date: Thu, 10 Oct 2024 15:04:45 +0300 Subject: [PATCH 2/2] Apply suggestions from code review --- documentation/reference/qfieldcloud/projects.en.md | 4 ++-- documentation/reference/qfieldcloud/system.en.md | 10 +++++----- 2 files changed, 7 insertions(+), 7 deletions(-) diff --git a/documentation/reference/qfieldcloud/projects.en.md b/documentation/reference/qfieldcloud/projects.en.md index c9ece0469..108c9dbb3 100644 --- a/documentation/reference/qfieldcloud/projects.en.md +++ b/documentation/reference/qfieldcloud/projects.en.md @@ -27,7 +27,7 @@ A project can be created in multiple ways: ## Files -Files are the skeleton on which QFieldCloud project works. +Files are the skeleton on which QFieldCloud projects work. To make a QFieldCloud project alive users need to upload at least a single QGIS project file in the `.qgs` or `.qgz` file formats. All geospatial files must be uploaded using the same relative paths as on one's computer. If external SVG or raster symbology is used, users must upload the corresponding files too. @@ -55,7 +55,7 @@ project ``` -The file in a QGIS project can be in one of the following groups by their purpose: +The files in a QGIS project can be in one of the following groups by their purpose: - **QGIS project file** - a `.qgs` or `.qgz` project file. - **QGIS sidecar files** - the utility files to the QGIS project file, such as `*_attachments.zip` or other sidecar files. diff --git a/documentation/reference/qfieldcloud/system.en.md b/documentation/reference/qfieldcloud/system.en.md index a55418246..240a6783b 100644 --- a/documentation/reference/qfieldcloud/system.en.md +++ b/documentation/reference/qfieldcloud/system.en.md @@ -13,16 +13,16 @@ The aim of this document is to provide an overview of the QFieldCloud system to ### QGIS Project -In QFieldCloud context a QGIS project refers to all data files that are required for properly functioning QGIS project. +In the QFieldCloud context, a project refers to all data files that are required for a properly functioning QGIS project. Read more about [QGIS projects](./projects.md). ### Layer action -Each layer in the QGIS project can be configured with an "layer action". +Each layer in the QGIS project can be configured with a "layer action". The action determines how QFieldSync and QField should treat the layer. There are the two actions that can be configured: cloud action and cable action. -They work are applied in the QFieldCloud and traditional cable export context respectively. +They work in the QFieldCloud and traditional cable export context respectively. The following actions are available and will be explained in more detail below: @@ -59,7 +59,7 @@ Examples are WFS, WM(T)S-layers or layers coming from a database such as PostGIS If the layer is file-based it will be in read-only mode. - *Remove from project* will simply remove the layer from the project (not package it for QField). -From QFieldSync it will be possible to update a project already loaded on QFieldCloud. +With QFieldSync in QGIS it will be possible to update a project already loaded on QFieldCloud. In the event that the changes concern only styles, forms etc. but not the data or the structure of the layers, the QGIS project file (`.qgs`/`.qgz`) on the server will simply be updated. @@ -77,7 +77,7 @@ Depending on the actions set for each layer in QFieldSync for QFieldCloud or Cab | Action | File based layer | Service-based layer (e.g. WMS, database) | Notes | |---------------------------------|---------------------------|--------------------------------------------|--------------------------------------| | Offline editing | Create and push a Change | N/A | | -| Directly access data source | Layer is readonly | Edit the online online database | | +| Directly access data source | Layer is readonly | Edit the online database | | | Remove from project | N/A | N/A | the layer is not available on QField | | Copy | Create and readonly | N/A | | | Keep existing (Copy if missing) | Create and readonly | N/A | |