[Survey sync] Sync fails silently

[gino-m]

07:15:55.580  D  Background survey sync failed
                 java.lang.IllegalStateException: FirebaseFirestore has already been started and its settings can no longer be changed. You can only call setFirestoreSettings() before calling any other methods on a FirebaseFirestore object.
                 	at com.google.firebase.firestore.FirebaseFirestore.setFirestoreSettings(FirebaseFirestore.java:267)
                 	at com.google.android.ground.persistence.remote.firebase.FirebaseFirestoreProvider$1.invokeSuspend(FirebaseFirestoreProvider.kt:30)
                 	at com.google.android.ground.persistence.remote.firebase.FirebaseFirestoreProvider$1.invoke(Unknown Source:8)
                 	at com.google.android.ground.persistence.remote.firebase.FirebaseFirestoreProvider$1.invoke(Unknown Source:2)
                 	at com.google.android.ground.util.AsyncSingletonProvider.get(AsyncSingletonProvider.kt:33)
                 	at com.google.android.ground.persistence.remote.firebase.GroundFirestoreProvider$1.invokeSuspend(GroundFirestoreProvider.kt:28)
                 	at com.google.android.ground.persistence.remote.firebase.GroundFirestoreProvider$1.invoke(Unknown Source:8)
                 	at com.google.android.ground.persistence.remote.firebase.GroundFirestoreProvider$1.invoke(Unknown Source:2)
                 	at com.google.android.ground.util.AsyncSingletonProvider.get(AsyncSingletonProvider.kt:33)
                 	at com.google.android.ground.persistence.remote.firebase.FirestoreDataStore.db(FirestoreDataStore.kt:56)
                 	at com.google.android.ground.persistence.remote.firebase.FirestoreDataStore.access$db(FirestoreDataStore.kt:47)
                 	at com.google.android.ground.persistence.remote.firebase.FirestoreDataStore$loadSurvey$2.invokeSuspend(FirestoreDataStore.kt:59)
                 	at com.google.android.ground.persistence.remote.firebase.FirestoreDataStore$loadSurvey$2.invoke(Unknown Source:8)
                 	at com.google.android.ground.persistence.remote.firebase.FirestoreDataStore$loadSurvey$2.invoke(Unknown Source:4)
                 	at kotlinx.coroutines.intrinsics.UndispatchedKt.startUndispatchedOrReturn(Undispatched.kt:78)
                 	at kotlinx.coroutines.BuildersKt__Builders_commonKt.withContext(Builders.common.kt:167)
                 	at kotlinx.coroutines.BuildersKt.withContext(Unknown Source:1)
                 	at com.google.android.ground.persistence.remote.firebase.FirestoreDataStore.loadSurvey(FirestoreDataStore.kt:59)
                 	at com.google.android.ground.repository.SurveyRepository$loadAndSyncSurveyWithRemote$2.invokeSuspend(SurveyRepository.kt:113)
                 	at com.google.android.ground.repository.SurveyRepository$loadAndSyncSurveyWithRemote$2.invoke(Unknown Source:8)
                 	at com.google.android.ground.repository.SurveyRepository$loadAndSyncSurveyWithRemote$2.invoke(Unknown Source:4)
                 	at kotlinx.coroutines.intrinsics.UndispatchedKt.startUndispatchedOrReturnIgnoreTimeout(Undispatched.kt:89)
                 	at kotlinx.coroutines.TimeoutKt.setupTimeout(Timeout.kt:151)
                 	at kotlinx.coroutines.TimeoutKt.withTimeoutOrNull(Timeout.kt:107)
                 	at com.google.android.ground.repository.SurveyRepository.loadAndSyncSurveyWithRemote(SurveyRepository.kt:111)
                 	at com.google.android.ground.domain.usecases.survey.SyncSurveyUseCase.invoke(SyncSurveyUseCase.kt:36)
                 	at com.google.android.ground.persistence.sync.SurveySyncWorker.doWorkInternal(SurveySyncWorker.kt:53)
                 	at com.google.android.ground.persistence.sync.SurveySyncWorker.access$doWorkInternal(SurveySyncWorker.kt:33)
                 	at com.google.android.ground.persistence.sync.SurveySyncWorker$doWork$2.invokeSuspend(SurveySyncWorker.kt:43)
                 	at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
                 	at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:108)
                 	at kotlinx.coroutines.internal.LimitedDispatcher$Worker.run(LimitedDispatcher.kt:115)
                 	at kotlinx.coroutines.scheduling.TaskImpl.run(Tasks.kt:103)
                 	at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.kt:584)
                 	at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.executeTask(CoroutineScheduler.kt:793)
                 	at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.runWorker(CoroutineScheduler.kt:697)
                 	at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:684)

[shobhitagarwal1612]

Is it possible that this is not being initialized multiple times, but is being initialized late?

[shobhitagarwal1612]

If that's the case, then swallowing the exception won't help. Is my understanding correct?

[gino-m]

If that's the case, then swallowing the exception won't help. Is my understanding correct?

We don't call firestoreSettings = anyplace else, so I don't see how it could be a problem of late initialization, unless I'm missing something?

[shobhitagarwal1612]

The exception says that the setting can't be updated once it is started. I'm not 100% sure whether their definition of started means initialised or something else

[gino-m]

Fair point. Tbh I'm not sure how to repro this issue; it might be because the provider was used from two components, and thus initialized two times. #2380 will log an error if this happens, so hopefully that will give us more insight.

[gino-m]

This occurred the first time I tried to collect data with the Lacuna survey, causing data to not be synced.

I put the app to in the background and killed the app several times while collecting data. I wonder if this is why the error was reproduced. Restarting the app the data eventually syncs on the next attempt.

Here's the full stack trace:

2024-11-15 19:59:11.272 29127-29168 LocalMutationSyncWorker com.google.android.ground            D  Local mutation sync failed
                                                                                                    java.lang.IllegalStateException: FirebaseFirestore has already been started and its settings can no longer be changed. You can only call setFirestoreSettings() before calling any other methods on a FirebaseFirestore object.
                                                                                                    	at com.google.firebase.firestore.FirebaseFirestore.setFirestoreSettings(FirebaseFirestore.java:267)
                                                                                                    	at com.google.android.ground.persistence.remote.firebase.FirebaseFirestoreProviderKt.setFirestoreSettings(FirebaseFirestoreProvider.kt:42)
                                                                                                    	at com.google.android.ground.persistence.remote.firebase.FirebaseFirestoreProviderKt.access$setFirestoreSettings(FirebaseFirestoreProvider.kt:1)
                                                                                                    	at com.google.android.ground.persistence.remote.firebase.FirebaseFirestoreProvider$1.invokeSuspend(FirebaseFirestoreProvider.kt:30)
                                                                                                    	at com.google.android.ground.persistence.remote.firebase.FirebaseFirestoreProvider$1.invoke(Unknown Source:8)
                                                                                                    	at com.google.android.ground.persistence.remote.firebase.FirebaseFirestoreProvider$1.invoke(Unknown Source:2)
                                                                                                    	at com.google.android.ground.util.AsyncSingletonProvider.get(AsyncSingletonProvider.kt:33)
                                                                                                    	at com.google.android.ground.persistence.remote.firebase.FirestoreDataStore.db(FirestoreDataStore.kt:55)
                                                                                                    	at com.google.android.ground.persistence.remote.firebase.FirestoreDataStore.applyMutations(FirestoreDataStore.kt:92)
                                                                                                    	at com.google.android.ground.persistence.sync.LocalMutationSyncWorker.processMutations(LocalMutationSyncWorker.kt:119)
                                                                                                    	at com.google.android.ground.persistence.sync.LocalMutationSyncWorker.access$processMutations(LocalMutationSyncWorker.kt:44)
                                                                                                    	at com.google.android.ground.persistence.sync.LocalMutationSyncWorker$processMutations$2.invokeSuspend(Unknown Source:15)
                                                                                                    	at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
                                                                                                    	at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:108)
                                                                                                    	at kotlinx.coroutines.internal.LimitedDispatcher$Worker.run(LimitedDispatcher.kt:115)
                                                                                                    	at kotlinx.coroutines.scheduling.TaskImpl.run(Tasks.kt:103)
                                                                                                    	at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.kt:584)
                                                                                                    	at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.executeTask(CoroutineScheduler.kt:793)
                                                                                                    	at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.runWorker(CoroutineScheduler.kt:697)
                                                                                                    	at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:684)

[gino-m]

Issues found so far:

• After the app is killed and reopened during data collection, FirestoreUuidGenerator is initalizing FirebaseFirestore before the async provider is called to set the appropriate settings.
• The async singleton FirebaseFirestore is initialized by the background worker, even after it's already been initialized.