Added on_gain_focus, on_lose_focus, on_show & on_hide handlers on toga.Window

[proneon267]

Implements the APIs described in #2009.

on_gain_focus, on_lose_focus, on_show& on_hide handles are available both as properties and also as initialization parameters in toga.Window.

Only tested on WinForms and gtk. This will take sometime to complete for all backends.

Fixes #2009

PR Checklist:

• All new features have been tested
• All new features have been documented
• I have read the CONTRIBUTING.md file
• I will abide by the code of conduct

[freakboy3742]

Thanks for the contribution - this is another feature that is going to hit up against #2058 (and #2075); as such, I'm hesitant to merge it without tests.

I'm also hesitant because it doesn't currently have a Cocoa implementation (that should be easy enough, but it's worth flagging); and it's not 100% obvious how this would behave on mobile. My immediate reaction (and the one raised in #2006) is that gain/lose focus might link into application lifecycle hooks (so when the app comes into the foreground, that's the "gain focus" event for both the window and the app), but there's an open question of how those signals interact with tablet platforms that allow split-screen and other multi-app modes. This is an area where some additional design is required.

[proneon267]

Yes, I agree with you. This should not be merged currently. I will need some time to write the implementation for other backends. Like #1930, I will wait until the audits are merged, and will write the tests thereafter.

As for additional design for tablet modes, I agree with you. I will research more about it and will discuss with you while implementing for mobile platforms.

[proneon267]

@mhsmith I need your guidance on android side. According to: https://developer.android.com/guide/components/activities/intro-activities#onpause

The system calls onPause() when the activity loses focus and enters a Paused state. This state occurs when, for example, the user taps the Back or Recents button.

On the Emulator(Android 12):

When starting the app, the following events are triggered: onCreate()->onStart()->onResume() . But, when I press either the Back or Recents button, onPause() event is not triggered. No other events are triggered.

When I select the app by pressing the Recents button, only onStart()->onResume() are triggered. onRestart() is not triggered.

D/MainActivity: onStart() start
I/python.stdout: Toga app: onStart
D/MainActivity: onStart() complete
D/MainActivity: onResume() start
I/python.stdout: Toga app: onResume
D/MainActivity: onResume() complete
I/OpenGLRenderer: Davey! duration=18074ms; Flags=1, FrameTimelineVsyncId=10773, IntendedVsync=2419137327998, Vsync=2419137327998, InputEventId=0, HandleInputStart=2419139817470, AnimationStart=2419139847710, PerformTraversalsStart=2419139888290, DrawStart=2419223580770, FrameDeadline=2419170661330, FrameInterval=2419139788630, FrameStartTime=16666666, SyncQueued=2419225272400, SyncStart=2419226783620, IssueDrawCommandsStart=2419227470230, SwapBuffers=2419249275450, FrameCompleted=2437213784760, DequeueBufferDuration=21300, QueueBufferDuration=360800, GpuCompleted=2437213784760, SwapBuffersCompleted=2419251729810, DisplayPresentTime=0,

When I press the home button, neither onPause() or onStop() events are triggered. No other events are triggered.

On a Physical Device(Android 13):

When starting the app, the following events are triggered: onCreate()->onStart()->onResume() . But, when I press either the Back or Recents button, onPause() event is not triggered. Instead the following events are triggered:

D/VRI[MainActivity]: onFocusEvent false
D/VRI[MainActivity]: dispatchAppVisibility visible:false
D/BufferQueueProducer: [VRI[MainActivity]#0(BLAST Consumer)0](id:55c400000000,api:1,p:21956,c:21956) disconnect: api 1
D/BufferQueueConsumer: [VRI[MainActivity]#0(BLAST Consumer)0](id:55c400000000,api:0,p:-1,c:21956) disconnect
D/VRI[MainActivity]: setWindowStopped stopped:true
D/ActivityThread: do gfx trim 40 success

When I select the app by pressing the Recents button, only onStart()->onResume() are triggered. onRestart() is not triggered. Instead the following events are triggered:

D/VRI[MainActivity]: dispatchAppVisibility visible:true
D/VRI[MainActivity]: setWindowStopped stopped:false
D/MainActivity: onStart() start
I/python.stdout: Toga app: onStart
D/MainActivity: onStart() complete
D/MainActivity: onResume() start
I/python.stdout: Toga app: onResume
D/MainActivity: onResume() complete
I/Quality : Skipped: false 0 cost 5.253646 refreshRate 8332212 bit true processName com.example.helloworld
D/BufferQueueConsumer: [](id:55c400000001,api:0,p:-1,c:21956) connect: controlledByApp=false
E/IPCThreadState: attemptIncStrongHandle(57): Not supported
D/BufferQueueProducer: [VRI[MainActivity]#1(BLAST Consumer)1](id:55c400000001,api:1,p:21956,c:21956) connect: api=1 producerControlledByApp=true
D/VRI[MainActivity]: registerCallbacksForSync syncBuffer=false
D/VRI[MainActivity]: Received frameCommittedCallback lastAttemptedDrawFrameNum=1 didProduceBuffer=true syncBuffer=false
D/VRI[MainActivity]: draw finished.
D/VRI[MainActivity]: onFocusEvent true

When I press the home button, neither onPause() or onStop() events are triggered. But, the following events are triggered:

D/VRI[MainActivity]: onFocusEvent false
D/VRI[MainActivity]: dispatchAppVisibility visible:false
D/BufferQueueProducer: [VRI[MainActivity]#0(BLAST Consumer)0](id:55c400000000,api:1,p:21956,c:21956) disconnect: api 1
D/BufferQueueConsumer: [VRI[MainActivity]#0(BLAST Consumer)0](id:55c400000000,api:0,p:-1,c:21956) disconnect
D/VRI[MainActivity]: setWindowStopped stopped:true
D/ActivityThread: do gfx trim 40 success

As, you can see, onFocusEvent, dispatchAppVisibility visible and setWindowStopped stopped show consistent behaviors.

Why are the documented Activity lifecycle events not being triggered as per the documentation?

[proneon267]

Regarding the gain/lose focus on mobile platforms like android:

From my testing, the app will lose focus when either the Home or Recent App List buttons are pressed. On pressing the back button, the app also loses focus and sends the user to the home screen.

In split screen mode (like dual app mode), suppose there are two apps A and B. App A will gain focus when the user touches the A app's screen. The focus is lost when the user touches B app's screen or interacts with the system launcher or presses the Home or Recent App List Button.

In floating window mode, the app will gain focus when the user touches the app's screen. The focus is lost when the user touches anything outside the app, like interacting with the system launcher or another app.

In iOS like the cocoa, there exists UIApplicationDelegate, which has methods like: applicationDidBecomeActive and applicationWillResignActive

But, there needs to be another handler to differentiate between the states when (the app is not visible to the user & is not receiving inputs) and (when the app is visible to the user & is receiving inputs).

Hence, I would like to propose other additional handlers, on_background/foreground to call the handler when the app is in background/foreground and not visible/ visible to the user.

What do you guys think?

Also, without confirmation from @mhsmith regarding the Activity life events triggering behavior, I cannot proceed with the android implementation. Hence, I was thinking about working on the iOS implementation first.

[mhsmith]

When I press the home button, neither onPause() or onStop() events are triggered.

That's because those methods aren't included in the Android template, either in IPythonApp or in MainActivity. They should be added to both places, but ideally with an emptydefault implementation in order not to break any third-party implementations of IPythonApp. And all the existing methods should become default as well.

In split screen mode (like dual app mode), suppose there are two apps A and B.

See this page for how this is notified on Android.

But, there needs to be another handler to differentiate between the states when (the app is not visible to the user & is not receiving inputs) and (when the app is visible to the user & is receiving inputs).

Every API has a maintenance and testing cost, so I'd prefer not to add additional events unless there's a clear need for them, especially if they're only applicable to certain platforms.

[proneon267]

Thank you for helping. I will add default implementations for the remaining methods in the Android template and will submit a PR there after getting a stable behavior.

I agree with you that additional events will incur more maintenance. I feel that the on_background and on_foreground events are mostly needed for mobile platforms. But there is a need for distinction between [not visible state] and [visible but not receiving inputs state] of an app.

For example, the app should be put to a sleep mode(not updating the layout or text) when it is in background or [not visible state].
But when it is in a [visible but not receiving inputs state](like in a multi app screen mode), the app should update the layout or text, so that the user can get the latest updated information.

What do you think?

[proneon267]

I have tested android implementation both on a physical device and on the emulator.

I have submitted a PR at beeware/briefcase-android-gradle-template#69 so that the app focus event can be detected.

[proneon267]

Completed implementations of all the platforms and also added a test in the window example app.

I will write the tests after the audits are merged. But I think this PR is ready for a review.

Also, the CI android testbed is failing on its own for some reason.

[freakboy3742]

What you've done here looks like a good pass at implementing the API as proposed in #2009; however, I think we're hitting an area where we need more design before we proceed.

The detail you've dug up as part of the Android implementation has opened a bunch of design questions about what "focus" even means at the App/Window level. What are we actually trying to achieve with these signal handlers? Is "visibility" a better metaphor than "focus" in this case? Is there any use case for a literal "focus" event on a window? Do we need to differentiate between an app that is "visible", but isn't currently accepting input events, and an app that isn't accepting input events? How does the rest of the app lifecycle map into these events (on all platforms)?

Rather than pressing forward with an implementation, I think we need to step back and come up with a consistent design for these app/window lifecycle events, and work out how they map onto all the platforms we're targeting.

[proneon267]

Researching some more on the topic, it seems like we need 3 categories of events:

• Input Focus -> [Receiving Inputs] or [Not Receiving Inputs]
• Visibility -----> [Visible to User] or [Not Visible to User]
• Hover

The following are the states associated with the event categories mentioned above and their implications for other event categories:

Input Focus ---> [Receiving Inputs]
  |                               | ->[Visible to User]
  |
  |---------------> [Not Receiving Inputs]
                                 | ->[Visible to User] or
                                 | ->[Not Visible to User]

Visiblity -> [Visible to User]
  |                      | ->[Receiving Inputs] or
  |                      | ->[Not Receiving Inputs]
  |
  |---------> [Not Visible to User]
                         | ->[Not Receiving Inputs]

Hover ->[Interacting with a Pointing Device or Mouse] & [Visible to User] & [Not Receiving Inputs]

Use Cases:

• The use case for Input Focus might be to show a highlight effect indicating to the user that it is receiving input.

• The use case for Visibility might be to:

  • Put the app to a sleep mode(not updating the layout or text) when it is [Not Visible to User].
  • Update the layout or text when it is [Visible to User], so that the user can get the latest information.

• So far, I am not sure what the use case for Hover might be. Maybe showing effects that would imply to the user, clicking on it, will make it to receive input focus.

Who should have which event categories:

• Toga.Window should have:
  • Input Focus: Since, multiple windows may be present, hence each window should trigger Input Focus event when they receive/lose input focus.
  • Visibility: Same reason as above
  • Hover: Same reason as above
• Toga.App doesn't really need any of the event categories since it can just iterate through the Toga.App.windows list and add handler methods to the event handlers of the windows. But, we can add them if it would make intuitive sense for the user.

APIs are not much of a problem as the available platform APIs can be properly mapped onto the above described event categories.

[proneon267]

If resolving window states will make testing this PR easier... then that PR should be completed first.

:-) It seems like I have entangled myself in a web of interrelated planned PRs and APIs. Window State #1884 depends on Screen API #1930 and Screen API depends on DPI Scaling Fix #2155. Now for proper testing of this PR #2096, this will depend on Window State #1884.

So the flow of completing these PRs would be #2155 => #1930 => #1884 => #2096.

[proneon267]

TODO: Add on_resize handler after completion of window states API #1884

Done in #2364

[mhsmith]

on_resize doesn't seem related to the other events, and should also be easier to design and implement, so I'd recommend doing that in a separate PR of its own.

[proneon267]

on_resize doesn't seem related to the other events, and should also be easier to design and implement, so I'd recommend doing that in a separate PR of its own.

Sure. I'll do it in a separate PR.