Skip to content

Commit

Permalink
export models validate models
Browse files Browse the repository at this point in the history
numbering
entry on export page
  • Loading branch information
signedav committed Mar 18, 2024
1 parent 471b455 commit fdfd4e7
Show file tree
Hide file tree
Showing 5 changed files with 47 additions and 13 deletions.
Binary file added docs/docs/assets/validation_exportmodels.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
27 changes: 22 additions & 5 deletions docs/docs/user_guide/export_workflow.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,12 @@
You can export your data from an physical database into an `xtf` file (INTERLIS transfer file). Select *Export data from an existing database* in the intro page of the wizard.
## Database Selection
You can export your data from an physical database into an `xtf` file (INTERLIS transfer file).

The wizard is started over the toolbar icon or *Database > Model Baker > Import/Export Wizard*.

![wizard intro](../assets/workflow_wizard_intro.png)

Select *Export data from an existing database*.

## 1. Database Selection

First you have to select the database schema or file to export your data from.

Expand All @@ -9,9 +16,10 @@ For more detailed description of the connection parameters, see the description

When the database or the schema / file does not exist, a warning will appear.

## Export data
## 2. Export data

![wizard export data](../assets/workflow_wizard_export_data.png)

### XTF File

Set the `xtf` file where you want to export your data to.
Expand All @@ -20,13 +28,22 @@ Set the `xtf` file where you want to export your data to.

You can filter the data *either* by models *or* - if the database considers [Dataset and Basket Handling](../../background_info/basket_handling/) - by datasets *or* baskets. You can choose multiple models/datasets/baskets. But only one kind of filter (`--model`, `--dataset`, `--basket`) is given to the ili2db command (it would make no conjunction (AND) but a disjunction (OR) if multiple parameters are given (what is not really used). A conjunction can still be done by selecting the smallest instance (baskets)).

## Run ili2db Sessions
### Export Models

The export models do not define the data that should be exported, but in what *format* they should be exported. This is relevant if you use *extended* models: You have your data stored in your extended model, but you export it in the *format* of the base model.

![export base](../assets/workflow_wizard_export_data_base.png)

!!! Note
When no model is selected, the data are exported in the format of the model it is saved in. Multiple model selection makes sense here, when you have multiple models extended.

## 3. Run ili2db Sessions

In the next step you can run the export in one single ili2db command.

With the ![run arrow_button](../assets/arrow_button.png) button next to *Run* the options are provided to run the command without any validation of your data or to edit the command manually before running it.

## Data Validation
### Data Validation

If you did not choose *Run without constraints* on your export session, then the data are validated against their INTERLIS models. If this validation did not succeed, then the export will fail.

Expand Down
20 changes: 12 additions & 8 deletions docs/docs/user_guide/import_workflow.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,13 +8,14 @@ To [choose data files and models to import or generate a new database](#source-s
!!! Note
As well the Wizard opens when you drop a file to QGIS with the extension `xtf`, `ili` or `xml` (`xml` only when it's dropped with a file having one of the other extensions).

## 1. Source Selection

## Source Selection
First page offers you to select the sources to import.

![wizard source selection](../assets/workflow_wizard_source_selection.png)

### INTERLIS Models

Sources can be INTERLIS models stored in `ili` files you select from your local system (with the file browser ![file_browser](../assets/file_browser_button.png) or with drag'n'drop).

As well you can search for INTERLIS models in the [repositories](../../background_info/repositories) or [custom model directories](../plugin_configuration/#custom-model-directories). Since model names should be given strictly as they were spelled in INTERLIS files, once you start typing, autocompletion will help you discover available models with their correct spelling. Add them wiht the ![plus button](../assets/plus_button.png).
Expand All @@ -25,11 +26,12 @@ As well you can search for INTERLIS models in the [repositories](../../backgroun
This leads you to a process to create the physical database in *PostgreSQL*, *MSSQL* or *GeoPackage*.

### Transfer and Catalogue Files (Data Files)

The `xtf` is an INTERLIS transfer file, containing spatial and/or alphanumeric data stored in XML format. Catalogues or Codelists are technically the same but have usually `xml` as extension.

The selection of data files leads you to the process to import the data into a physical database.

## Database Selection
## 2. Database Selection

In any case you are requested to set the connection to your database.

Expand All @@ -43,8 +45,10 @@ In any case you are requested to set the connection to your database.
- **User Password** Set the password for the database user.
- **Execute data management tasks with superuser login from settings** If checked, it uses the superuser login (set in the [plugin configurations](../plugin_configuration/) for data management tasks like the creation of the schema etc.

## Import of INTERLIS Model
## 3. Import of INTERLIS Model

### Model selection

There are several ways the Model Baker wizard detects INTERLIS models to import.

- Read from the selected local `ini` file.
Expand Down Expand Up @@ -103,23 +107,23 @@ You can define `sql` scripts that runs before and after the (schema) import.

A `toml` or `ini` file can contain values for [meta attributes](../../background_info/meta_attributes/) (like `qgis.modelbaker.dispExpression`) instead of having them directly in the `ili` file.

## Run ili2db Sessions
## 4. Run ili2db Sessions

In the next step you can run all the sessions to create your physical model. If having multiple models selected that are received from the repositories, then they are passed in one command. You can run the commands one by one or all together.

![wizard schema import sessions](../assets/workflow_wizard_schema_session.png)

With the ![run arrow_button](../assets/arrow_button.png) button next to *Run* the options are provided to run the command without checking constraints or to edit the command manually before running it.

## Create Baskets
## 5. Create Baskets

In case you don't have data to import into your default dataset and want to collect ***fresh*** data in QGIS, the baskets have to be created as well. It's up to you for what topics you want to create baskets. Model Baker suggests you what baskets should be created and reasonalbe values for the BID (value in the `t_ili_tid`) but you might need. to edit them.

![wizard basket page](../assets/workflow_wizard_default_baskets_page.png)

If you don't know what it is about, check more details [here](../../background_info/basket_handling/#creation-of-baskets) or just press *Create baskets* and then *Next*.

## Import of INTERLIS Data
## 6. Import of INTERLIS Data

After the physial model is generated or you selected an existing database to import your transfer or catalogue files to containing the models already, you will see the page to set up your data imports.

Expand Down Expand Up @@ -151,7 +155,7 @@ Check more information about the catalogues on the *UsabILIty Hub* [here](../../

If you want to delete the data in the database first, you can check the corresponding box. On using baskets, the ili2db parameter `--replace` is executed instead of `--update`. On not using baskets, the parameter `--deleteData` is added to the command. Note that on using baskets, only the data from the corresponding dataset is deleted, whereas on not using baskets all data from the schema is deleted.

## Generate the QGIS Project
## 7. Generate the QGIS Project

In case you want to generate your project from an existing database, you will need to [set the connection parameters](#database-selection) first.

Expand Down Expand Up @@ -185,7 +189,7 @@ Choose your optimization strategy in the checkbox:

For more information about the optimization of extended models, see the [corresponding chapter](../../background_info/extended_models_optimization).

### OID Values
## 8. OID Values

Often the models definition requires cross-system unique identificators. So called OIDs, what are represented in the physical database as the `t_ili_tid` column. Find a clear definition and more details about them in the [corresponding chapter](../../background_info/oid_tid_generator).

Expand Down
13 changes: 13 additions & 0 deletions docs/docs/user_guide/validation.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,12 +3,21 @@ You can validate your physical data against the INTERLIS models directly in QGIS
![validation](../assets/validation.png)

## Database

The database connection parameter are emitted from the currently selected layer. Mostly this is representative for the whole project, since mostly a project bases on one single database schema/file. In case of multiple used database sources, it's possible to *switch* between the validation results when switching the layers.

## Filters

You can filter the data being validated *either* by models *or* - if the database considers [Dataset and Basket Handling](../../background_info/basket_handling/) - by datasets *or* baskets. You can choose multiple models/datasets/baskets. But only one kind of filter (`--model`, `--dataset`, `--basket`) is given to the ili2db command (it would make no conjunction (AND) but a disjunction (OR) if multiple parameters are given (what is not really used). A conjunction can still be done by selecting the smallest instance (baskets)).

## Validate in the base model

This is relevant if you use *extended* models: You have your data stored in your extended model, but might want to validate it in the *format* of the base model only.

![validate base](../assets/validation_exportmodels.png)

## Skip Geometry Errors

![validation_skipgeometryerrors](../assets/validation_skipgeometryerrors.png)

When the checkbox is activated, geometry errors are ignored and the validation of AREA topology is disabled. Errors like those will not be listed.
Expand All @@ -21,6 +30,7 @@ When the checkbox is activated, geometry errors are ignored and the validation o
In the backend the parameters `--skipGeometryErrors` and `--disableAreaValidation` are set.

## Configuration File

It's possible to enable/disable and name constraints of the current model via meta attributes. For the configuration of those, see the chapter [Set Meta Attributes in the Config File](#set-meta-attributes-in-the-config-file) below.

Add it to the validation by selecting it via file browser.
Expand All @@ -33,6 +43,7 @@ You can save this path to your project (to the project variables) with the ![run
You can also use a configuration file from the [ilidata repositories](../../background_info/usabilityhub/technical_concept). Just add the ilidata-key (like `ilidata:<key>`) as the path.

## Results

After running the validation by pressing the ![checkmark](../assets/checkmark_button.png) the results are listed.

With *right click* on the error a menu is opened with the following options:
Expand All @@ -50,6 +61,7 @@ On automatic pan and zoom, the coordinates are taken into account if they are pr
Since ili2db sometimes on non-geometry errors provides the coordinates as well, there could be a confusion when it zooms or pans to the coordinates there. Still it's preferable to not zoom or pan to the coordinates on geometry errors when they provide an OID. Currently the validator cannot make a difference between those error-types.

## Using of Meta Attributes in the Validation

As well as configuring [meta attributes](../../background_info/meta_attributes/) used for the physical database implementation and for QGIS project generation, meta attributes can be used for additional configuration of the validation like e.g. disable specific checks generally or on specific objects as well as naming of the constraints.

### Set Meta Attributes in the ILI File
Expand Down Expand Up @@ -165,6 +177,7 @@ See for all the global configurations the official [documentation of ilivalidato
Validation with deactivated validation is useful because it checks if everything is ok with the technical aspects (like `t_typ`, `t_id` etc).

## ili2db with `--validate` in the background

On running the validation `ili2db` is used in the background with the parameter `--validate`. This means no export of the data is needed. The output is parsed by Model Baker and provided in the result list.

Entries of the type `Error` and `Warning` are listed.

0 comments on commit fdfd4e7

Please sign in to comment.