Changes not applied - Unable to find feature

[miili]

A bunch of changes failed with Unable to find feature and could not be applied.

The changes came in in small chunks (3 deltas). Applying the change again does fail with the same error.

This is related to #324

[suricactus]

Can you give me any details about username/projectname and ideally, delta ids that you want to know more about. Also what is the data format of the layers, GeoPackage, PostGIS?

[miili]

Dataformat is GeoPackage. Change eff4a084-aef5-4b9b-8efa-321fb23f0ab0 Project Eifel-Large-N/Re-Deployment

[miili]

@suricactus when you see patch dd6fff05-b9ad-47c8-a253-8ae54c405c10 the sourcePk and localPk is the same. However the sourcePk should be 383.

Is this due to an outdated QField client app?

[miili]

I have 8 pages of changes here and none of them were properly applied!

Status says "applied" but they were not applied in the GeoPackage downstream. Yesterday's work had differing sourcePk and localPk.

What is going on? This is pretty critical! 💣

[miili]

Now that QfieldCloud is offered as a paid service (which I am 💯 supportive of) bugs like this should be ironed out!! I see hundreds of changes from today which appear applied, but are not. We were confident to use qfieldcloud in production, but I am getting a little nervous now.

[miili]

All that was changed in-between the working versions was the styling in the project file.

[suricactus]

Hi, @miili . This is definitely something that is of high importance.

when you see patch dd6fff05-b9ad-47c8-a253-8ae54c405c10 the sourcePk and localPk is the same. However the sourcePk should be 383.

How do you know the source PK is wrong in this one? Adding a bit more context will help finding the root cause.

EDIT: looking at the delta contents, I assume the sourcePk should not be 383, but 219?

Is this due to an outdated QField client app?

Should not be the case, can you please share the version you are using?

[miili]

Mostly 2.7.1 is used in the field by the different teams, but not by all teams in the field.
I suspect it is a problem with the project.

The matching sourcePk seems to be a common issue: #324 (comment)

[suricactus]

Seems there is some root problem might be from QgsOfflineEditing core plugin. Will keep you updated what are our findings.

[miili]

See for example these features:

• Only the diamond shaped features should have been modified: Move feature, update attributes
• (probably due to bad sourcePk another feature was moved and edited

This already is a big mess. We probably have to move back to paper for the week to come 😞

[GlaDal]

Hi,

Maybe I have the same problem as yours :

I wrote a ticket and open a discussion about this problem because as miili, it's quiet critical because I can't put Qfield in production anymore...
Project URL : https://app.qfield.cloud/a/SOPRECO/CCGP_CBT_3-0/
Discussion : https://github.com/opengisch/QField/discussions/4015

Without talking about this other problem : https://github.com/opengisch/QField/discussions/4007
Both cumulated, it's harder and harder to be confident, happily, I'm postive attitude ! :-)

[miili]

I uploaded the GeoPackage from the beginning of the working day, re-applied all changes from the day (~500 deltas), and reproduced the same bad state. See job b6e81eef.

Something is really odd!

[miili]

After more investigation, I am pretty confident that the localPk and sourcePk are referring to the same fid_1, where instead sourcePk should point to fid.
Subsequently the wrong feature pointing to fid_1 is modified and everything is messed up.

How to access the API and download all the changes?

[suricactus]

@miili thanks for doing the deep parallel investigation. We have identified the issue as I have posted 4 hours ago. All the team resources are focused on fixing this issue, so expect results very soon.

We know how important is data for our users and we are looking for a solution where you do not need to download the changes and apply them yourself. Since the issue escalated a few hours ago, it will take a bit of time to be evaluated and implemented, but I completely understand your urgency. Please give us a window of a few hours (midnful it's midnight in the teams timezone) and we will provide you with all the information you need.

After we make sure we prevent this from hapening again, we can also provide you with all your changes.

Hope makes you feel a bit better, apologies for the inconvenience.

[miili]

Great, thank you!

I got all the deltas from the API and will use the day to apply changes locally. If needed I can share the scripts.

[suricactus]

Thanks for the heads up. Will keep you posted on our side too.

[mbernasocchi]

Hi @miili we we've clearly identified what the issue is and have assessed the impacted projects. The issue seem to be related to a very specific use case and we have 23 users affected by the issue out of 45K users.
We are now investigating what project configuration leads to the issue.
we'll keep you updated as soon as we've more information.
cheers Marco

[miili]

Thanks for the heads up. What exotic configuration did we use for our project?

[stcz]

Hello together,
My Projects are also affected of this bug and i can reproduce it too.
A Workaround to fix this is to delete and download the Project again.

If you need more Information or betatesting (i have a selfhosted cloud currently), where i could apply a patch e.g.

[suricactus]

Can you please send me an email at ivan@opengis.ch for additional information?

[BoswachterMarc]

Perhaps I am also involved...
All the 'not applied' changes are because of "unable to find feature". I tried to re-apply them, but that didn't work out.

Apart from that, some features of collaborators and myself where registered twice. I don't understand how this could happen in the field.

[kiri03]

Here is another user who has the same problems when synchronizing. The error "Unable to find feature" appears many times, but this entity does exist in the project.

[kiri03]

When one of my organization's editors makes changes to the project logs I get the error: "Unable to find feature" and the patch shows up as "Not_applied", but the feature is still in the project.
It's a bit frustrating since we're paying for a service that doesn't seem to be working as it should.

From what I have been able to verify the error occurs when trying to sync changes made on the phone with a theme loaded that is different from the original project that was uploaded to QFieldCloud.
In my case, I have a project in QGIS with a theme loaded by default that I upload to the QField cloud.
If once successfully uploaded, I download it on the phone and change any record, the changes are applied correctly on the server. But if on the phone I change the default theme to another one that is already defined and then I make a change in the same record as before, when trying to upload the changes to the server an "Unable to find feature" error occurs.

[kiri03]

I don’t know if it has anything to do with it, but the theme that was producing the errors refers to a symbology that has embedded SVGs in the project and they are rotated with the value that a specific field has.

[graymondon]

Same problem here... Several changes not applied because "Unable to find feature". I checked and the features are there (id are matching).

Any workaround before patching this ?

[GlaDal]

Same problem here !
I put images of the related changes details and job details failures (I don't understand that message : "empty sourceLayerId from localLayerId"

I don't know if it's in relation with themes uses (See post here), I have in this project 6 differents themes.
This same problem was posted here on March 19th.

Thanks for helping

Are we only 23 users to have this problem ?

[GlaDal]

#opengisch/qfieldsync#508

[sth-HMS]

I'm also affected:

[gisuser0]

{
  "conflicts": null,
  "delta_file_id": "4bd43c74-e88b-4ae8-b004-eaa91dbeeea8",
  "delta_id": "01042462-700b-45d6-8ba2-3453ccaa0361",
  "delta_index": 0,
  "e_type": "ERROR",
  "feature_pk": "{9a8a24b4-6fbe-4a1d-aca2-05425f951cb8}",
  "layer_id": "test_01c1f56e_701b_44fc_b85a_5a400914d8be",
  "method": "patch",
  "modified_pk": null,
  "msg": "Unable to find feature",
  "provider_errors": null,
  "status": "status_apply_failed"
}

The problem persists and apparently without any reproducible logic.
Unfortunately, it makes data collection unreliable. Is there any news?