Transition experimental performance optimizations to FHIR Core

[ndegwamartin]

Describe the Issue

• SQLite crashes with 'Expression tree is too large (maximum depth 1000)' google/android-fhir#2561
• Sync fails with 'too many SQL variables' in select google/android-fhir#2559
• Caching the Questionnaires and the corresponding StructureMaps - PR Caching questionnaires and their SM #3461
  • Write back, can sync the first time they are used
  • Write through, can we do the caching in the background? Could cache on Q/SM change, e.g. after initial sync, then first load will be faster
• Use infinite scroll on search results instead of fetching entire list #3439
• Optimize the Database Search API google/android-fhir#2668
  • Parallelize any independent processing that happens after a search query e.g. when the SDK fetches the FHIR Resources from the DB they are returned as JSON documents. These need to be mapped to HAPI FHIR Structures which is a separate operation
• Reduce transaction(hog) time for GET that might return large results by fetching in batches #3440
• Expose an interface to db.withtransaction through FhirEngine google/android-fhir#2531
  When saving a questionnaire, run as single transaction, and not having to wait for other parts of related operations when queued in other transactions

[pld]

@ndegwamartin for the last one in this list, parallelization, do we have existing code that's implementing this? Can you share a link to this? Related, should we open an SDK issue for this or is this something that should live in the client (ie fhircore)

[ndegwamartin]

@ndegwamartin for the last one in this list, parallelization, do we have existing code that's implementing this? Can you share a link to this? Related, should we open an SDK issue for this or is this something that should live in the client (ie fhircore)

The code should reside on the SDK - The custom changes are bundled on this branch, they can be cherry. picked for PRs or discussions so that the SDK team can review first https://github.com/opensrp/android-fhir/tree/test-removed-select-transaction

[pld]

Cool, do we have an issue on Android FHIR SDK for that? On the Android FHIR SDK release we default to are we included the changes in that branch / should we be?

[ndegwamartin]

Cool, do we have an issue on Android FHIR SDK for that? On the Android FHIR SDK release we default to are we included the changes in that branch / should we be?

The latter is part of the focus of this ticket

[pld]

@ndegwamartin can you split infinite scroll into a new issue?

[f-odhiambo]

@LZRS @ndegwamartin Share an update here - Aug 5th

[qiarie]

@ndegwamartin @LZRS Should we consider adding a debounce interval to the register search if not already there?

[ndegwamartin]

@ndegwamartin @LZRS Should we consider adding a debounce interval to the register search if not already there?

Yeah we can explore this and evaluate its impact. Depending on the user patterns it may reduce the race conditions for the db connections as well as provide faster responses for users who enter more search characters. The latter may not be noticeable now though because of the scrolled pagination enhancement.