Add support for ODKCentral Datasets (Entity Lists) & Entities

[spwoodcock]

Is your feature request related to a problem? Please describe.

• ODK Entities are a new feature described here: https://forum.getodk.org/t/entity-based-data-collection/38115
  • In summary, an entity is a feature in the world that can be returned to over multiple forms.
  • An entity would be an OSM Feature in our case.
  • Entities would be stored in ODKCentral and linked to associated forms.
• ODKCollect is moving in the direction of having both a form-based flow, and an entity-based flow.
  • The entity-based flow is useful for us, as we could potentially open ODKCollect with an entity selected (an OSM Feature), after which we can then open and fill out a form related to the entity (and return to the same entity for another form later too).
• Entities are listed under an extra level called Datasets (also called Entity Lists), so I would propose the link to FMTM to be:

Project (FMTM organisation) --> Dataset (FMTM project, level where forms are linked) --> Entity (OSM Feature)

Projects are linked 1:1 to FMTM organisations, with user management.
Datasets are linked 1:1 to FMTM projects.
Entities are linked 1:1 to OSM Features within a project.

Correct me if I am wrong!

Describe the solution you'd like

• Modify ODKProject.py to link to FMTM Organisations, instead of Projects.
• Add ODKDataset.py to link to FMTM Projects:
  • Pretty much port the functionality of ODKProject.py currently, but to use Datasets instead.
  • https://docs.getodk.org/central-api-dataset-management/
  • Link forms to Datasets instead of Projects.
• Add ODKEntity.py to manage entities (each feature to be mapped in our project):
  • Creation / modification / deletion of entities in ODKCentral.
  • https://docs.getodk.org/central-api-entity-management/
  • Entities are linked to OSM Features via the data body params.
  • Once an Entity is made under a Dataset, it will be linked to the relevant form.

Additional context

I need to do some further reading to fully wrap my head around this.
Does anyone have thoughts?

[robsavoye]

Having been talking with the primary developer of Entities & Datasets in ODK Central a week ago, I'm not sure if it'll work for the FMTM data flow. I don't think opening ODK Collect with an OSM feature is useful for FMTM, we want to collect data on multiple features. OSM data is very dynamic, and often changes frequently, so the Datasets would get out of sync pretty quickly.

[spwoodcock]

A user will fill out a form for one feature at a time no?

[robsavoye]

Right, but there are multiple features in each task, so what's the point ? Something I'd like to see in ODK Collect is changing the color of their icons as we map using select_one_from_file. That may require entities or a dataset so an FMTM task could be mapped over several days.

Also some XLSForm have repeat sections, so multiple features in a single submission.

[spwoodcock]

If it's possible, I was thinking that a user can click on a feature in FMTM, then link to a Task entity, but with the OSM ID preselected for the feature they clicked.

The second case you mentioned makes things more tricky.

[spwoodcock]

The issue is that Collect has quite limited map functionality, and their release cadence is slow (ensuring compatibility, very thorough testing etc).

Ideally we want to handle the map in FMTM, then open Collect by intent (clicking a button).
That's the best user experience we could have.

The discussions around this https://forum.getodk.org/t/launch-odkcollect-from-a-browser-with-feature-pre-selected/43190/5
has suggested that launching with data in the URL is not an approach Collect is likely to go for.
More likely they will support launching by intent using an Entity selector.

But you raise a very good point about frequently changing OSM data.
The 1:1 mapping of OSM feature to entity may not always be perfect:

• It would use the OSM ID, so any changes to geometry are fine.
• But if the feature is changed (e.g. an incorrectly mapped building is split into multiple features), then it's a problem.

Perhaps the correct approach would be linking task areas to Entities, then seeing if we can get Collect to support an additional parameter on intent for opening on a specific OSM Feature within the Entity.

We have two options going forward:

1. See which direction the ODK team take launching by intent, then make FMTM work around that.
2. Scrap the idea of launching by intent & instead only use inbuilt Collect features (perhaps involving some PRs).

Interested to hear if @konishon has any thoughts on this too.

[robsavoye]

As long as the OSM ID is preserved through any data transformations, conflation is relatively easy.