Create endpoints for identifying different forms of connectivity

[nucleardreamer]

Needed for Wi-Fi connect and the SDK. This is an interesting topic because it covers a number of different offline cases, I think we need to discuss what "offline" actually means and what we want to setup as far as endpoint go. We have a few different situations to detect:

• WAN disconnected: you can still reach your local network but unable to resolve anything outside. This would effect the calls to the SDK
• LAN disconnected: you have no physical link to your own network. This effects wifi-connect and SDK calls.
• Supervisor disconnected: the supervisor container has crashed or is in a state of restart, meaning those endpoints won't work from our express backend
• Container/machine disconnected: You are on the front-end, but our actual container is not accessible or the machine is in an inaccessible state. This would be from being rebooted (stopping out backend container), or our backend crashing in a loop due to an error.

Polling APIs might be an ineffective solution to get real time information around all these states. We might want to roll WS (socket.io perhaps?) to provide an enum value for the state of why we are offline so that the UI can respond accordingly

[maggie44]

• WAN disconnected: you can still reach your local network but unable to resolve anything outside. This would effect the calls to the SDK

I think this is what we meant by whether there is internet connectivity. I.e. can it reach the outside world/Google/Balena.

• LAN disconnected: you have no physical link to your own network. This effects wifi-connect and SDK calls.

Wi-Fi connect checks whether WLAN0 is connected to a network or not and bases all its responses on that. It doesn't actually. You could be connected to Wi-Fi and not have internet, but it would still deem it as connected. This is great for wi-fi connect as it only wants to know about WLAN0 connectivity, not the outside world. In this sense, Wi-Fi connect is quite self sufficient, not sure if it needs to be included in the consideration for this conversation (Wi-Fi connect does have an endpoint for checking for internet connectivity, but we want people to be able to run the UI without Wi-Fi connect installed if they so choose, so it's not much use here).

Not sure what the advantage is in knowing about LAN or wi-fi connectivity over polling the internet. Seems like it may just be one more step. I guess it could be quicker response times, but so minimal I'm not sure if it's work adding this feature.

• Supervisor disconnected: the supervisor container has crashed or is in a state of restart, meaning those endpoints won't work from our express backend

There is a /ping endpoint in the Supervisor routes we can use for this.

• Container/machine disconnected: You are on the front-end, but our actual container is not accessible or the machine is in an inaccessible state. This would be from being rebooted (stopping out backend container), or our backend crashing in a loop due to an error.

I think the user would only know about this scenario when they next click something, so might just be easier to handle the errors and show a disconnected notification rather than any sort of polling of an endpoint.

Considering Wi-Fi connect is largely looking after itself, I wonder if it may be viable to use Axios interceptors to handle errors rather than to trigger polls of certain endpoints to check for connectivity. Referring here mostly to internal such as Supervisor and Expressjs (which should be up, rare that it wouldn't, maybe first boots and things but polling all the time may be excessive if the downtime is minimal). A dns resolution or something for internet connectivity may be helpful though to decide what shows and what doesn't?

Edit:

Or not any of what I just said then: https://stackoverflow.com/a/54573024/16019434

[maggie44]

For reference, here is something I used a while back for another project that seemed to work pretty well. May want to rethink it now, but food for thought:

const apiIsReady = ref<boolean>()
    const apiPath = ref<string>(
      'http://' + window.location.hostname + ':9090/v1/supervisor/state'
    )
    const errorMessage = ref<string>(t('api_unavailable'))
    const loadingSpinner = ref<boolean>(true)
    // If on the captive portal, the check is skipped to allow welcome message to display
    if (!window.location.pathname.includes('captive_portal')) {
      // Check the state of the device to decide on page to display
      AxiosOverride.get(apiPath.value, { timeout: 2000 })
        .then(function (response) {
          if (response.data.message === true) {
            apiIsReady.value = true
          } else {
            void waitApi()
          }
        })
        .catch(function () {
          void waitApi()
        })
    } else {
      apiIsReady.value = true
    }
    // Function for calling delays throughout
    function delay(ms: number) {
      return new Promise((resolve) => setTimeout(resolve, ms))
    }
    // Loop for rechecking when the API is ready
    async function waitApi() {
      const redirecting = ref<boolean>(false)
      // Show message indicating the page will auto-reload
      apiIsReady.value = false
      // Loop x number of times checking for the device to be ready
      for (let i = 0; i < 10; i++) {
        await delay(5000)
        const xhr = new XMLHttpRequest()
        xhr.open('GET', apiPath.value)
        xhr.timeout = 2000
        xhr.onreadystatechange = async function () {
          if (xhr.readyState === XMLHttpRequest.DONE) {
            // If successful status code
            if (xhr.status === 0 || (xhr.status >= 200 && xhr.status < 400)) {
              // If the API response returns true to indicate api is ready
              if (JSON.parse(xhr.responseText).message) {
                redirecting.value = true
                // A short delay to allow the device to settle
                await delay(3000)
                // Reload the page
                apiIsReady.value = true
              }
            }
          }
        }
        // Send another request if not already redirecting
        if (!redirecting.value) {
          xhr.send()
        } else {
          return
        }
      }
      // If no success after x loops, display a different error
      errorMessage.value = t('api_down')
      loadingSpinner.value = false
      $q.notify({
        type: 'negative',
        message: `${t('error')} API is down or device state is False.`
      })
      // Try the original route eventually to avoid a bug prevnting complete access
      await delay(30000)
      apiIsReady.value = true
    }

[tmigone]

I haven't been involved much with local-ui recently, regardless, here are my 2c...

WAN disconnected
Wether or not we have internet access. This is useful information!

I feel like we can and should avoid the responsibility of figuring out if a device has connectivity or not. The supervisor is already making periodic requests to the balena API, so making yet another service that polls google, a dns server, or an extra api seems wasteful (also I remember a lot of support threads where users complain about all that outgoing requests, this would only increase that).

I don't know enough about the inner state that's kept by the supervisor, but I imagine we can replicate the model we have for determining device status on the API. If you ignore the VPN, the API is just checking wether a device has "recently" heartbeat-ed the API to figure out if the device is online or offline (https://www.balena.io/docs/learn/manage/device-statuses/). Can we not do the same thing but on the other end? Have the supervisor keep track of the "API status" and use that as a proxy for "internet access": device has internet access if it has "recently connected to the API successfully", offline if it hasn't. Of course this assumes connecting to balena infrastructure means universal internet access but I think it's a fair assumption to make. cc @cywang117 please tell me why this is a bad idea :P

LAN disconnected
Polling the network gateway should be cheap and enough though I can't come up with a good use case for knowing this information.

Supervisor disconnected and Container/machine disconnected
IMO we should treat both this cases as an error/exception. If either of them are down/unreachable then something has gone woefully wrong and the device probably needs to be looked at?

[maggie44]

device has internet access if it has "recently connected to the API successfully", offline if it hasn't.

Doesn’t the supervisor have configurable values that change the poll rate to the api? If a user had the vpn disabled and a high poll rate then this would be quite out of date info?

I have lost track a bit of the goal. Wouldn’t a simple poll to Google or dns on first render of the page be sufficient? It would only happen once to decide which components to show and would be among a bunch of other polls to populate the device ui so in the grand scheme of things not really costly. It’s not the backend polling regularly either, only when some loads the page. If the device then goes offline unexpectedly then I imagine errors are ok, not sure if we want to handle every rate occurrence. If you pull the lan cable out of your device then yes, you will get errors. The user will just have to refresh the page.

LAN for information could be on the networks page and only called when needed to load that page. There are two types of check, those required to render the page and those used as a feature or function. For rendering the page, probably just an internet check, and then any issues connecting to the API not related to internet connectivity could be in the error handlers for that endpoint?

Supervisor down would be a Balena os issue, not sure we need to anticipate that happening or handle for it, instead assume best case scenario. Fortunately the front end and backend of this device are all in one container so shouldn’t be any reason not to access our own backend. The fact you reached the ui means it must be up. And again any later failures seem unlikely other than for brief container updates and not sure how important it is to handle for those outside of standard error handling for Axios (I.e. no polling).

If the internet dependent features started as disabled (not visible) and then activate if a successful internet response returns then it’s also going to have minimal impact on the load time. It can keep rendering rather than wait for the response (although a clever internet check could be done in ms).