[WIP] Kim FSM operations timeout

[koala7659]

Description

The operations processed by the KIM service in the Runtime CR controller reconciliation loop are interrupted after the timeout period has elapsed. KIM is configured with parameters:

• provisioning-timeout - for shoot creation timeout
• update-timeout - for shoot update timeout
• deprovisioning-timeout - for Shoot deletion timeout

Implementation
The moment every new operation starts by Runtime controller - a new annotation is added to RuntimeCR with the current timestamp. Example:

kyma-project.io/runtime-operation-started: "2024-08-09T05:06:02Z"

During each reconciliation cycle - the current time is checked against saved timestamp and configuration parameter.

When operation succeeds on Gardener side before timeout - the Runtime CR status is switched in Ready state and timestamp annotation is removed from Runtime CR.

When operation does on Gardener side before timeout - the Runtime CR status is switched in Failed state and timestamp annotation remains on Runtime CR to prevent any further reconciliation attempts.

Recovery
If the timeout during Update function occurs - user (KEB) can fix broken configuration that has timeout by manually deleting annotation in the Runtime instance:
kyma-project.io/runtime-operation-started:
In such a case Runtime reconciler will Patch Gardner shoot with fixed configuration and will start next upgrade cycle and wait for operation to be completed on Gardener side.

Output

Provisioning timeout:

 status:
    conditions:
    - lastTransitionTime: "2024-08-08T11:17:23Z"
      message: Shoot creation timeout
      reason: ShootCreationTimeout
      status: "False"
      type: Provisioned
    state: Failed

Deprovisioning timeout:

 status:
    conditions:
    - lastTransitionTime: "2024-08-08T11:17:23Z"
      message: Runtime deprovisioning timeout
      reason: ShootDeletionTimeout
      status: "False"
      type: Deprovisioned
    state: Failed

Upgrade timeout:

 status:
    conditions:
    - lastTransitionTime: "2024-08-08T11:17:23Z"
      message: Shoot reconcile timeout
      reason: ShootProcessingTimeout
      status: "False"
      type: Provisioned
    state: Failed

Related issue(s)
#324

[koala7659]

Closing as we decided this functionality will not be part of KIM and we will use Gardener shoot state to determine Runtime CR status. All timeouts if occurred will be will be propagated.