Allow to configure custom network adapter labels

[milan-zededa]

EVE already offers adapter labels uplink and freeuplink to match
a specific group of ports to provide external connectivity for a network
instance.

We should allow users to define and assign their own adapter labels and
support the following new network instance use-cases:

1. Switch network instance with multiple ports: user should be able to
   select a group of ports, all of which will be bridged together under
   one switch NI. EVE should then automatically run the STP protocol for
   the bridge to avoid bridge loops and broadcast storms.
2. Local network instance with multiple ports: the routing table on the
   host (EVE) side will contain routes from all selected ports and will
   be used to select output port for the given flow.
   For (not only default) route we should:
   a) allow the user to pick the output port(s). Can be more than one
   for multipath routes.
   b) periodically test port connectivity and failover between ports
   when currently used one has broken connectivity
   c) allow to enable cost-based port selection
   d) allow to enable cellular signal strength based selection.
   Applicable only among wwan ports.
   a and b/c/d could be combined together to allow the user to select a
   subset of NI ports for a route and then let EVE to pick one of them
   based on the probing results and other selection criteria.
   We should also allow to limit a port-forwarding rule to only a subset
   of ports used by a local network instance. This can be done by
   defining a new ACEMatch type adapter where value is adapter label
   to match.

Currently, EVE supports a variant of the second use-case for hard-coded
uplink and freeuplink labels. It would make sense to unify the behaviour
between these existing labels and the user-defined ones. This means that
we would make small change to the current implementation:
when uplink or freeuplink is selected for a local network instance,
the NI routing table will have IP routes from all matching (mgmt)
ports and the default route will be selected by connectivity testing.
This is different to the present behaviour, where EVE installs only
routes of the port selected by connectivity testing into the NI routing
table. But lets say that there are 2 mgmt ports: eth0 and eth1, and
connectivity testing selected eth0. Then even if app is accessing hosts
which are directly inside the eth1 subnet, traffic will be routed via eth0,
which can be confusing. Also, user-defined static routes for eth1 are
not present in the routing table and therefore inactive when eth0 is
selected. The idea is to apply connectivity testing to only default
route selection.

Another change in behaviour is wrt. the inbound (port forwarding) rules.
Without the newly introduced adapter label specified (e.g. when EVE is
upgraded but config remains the same), the inbound rules will be applied
to all the ports used by a network instance. With uplink labels this
means that port forwarding will be enabled on all the management ports,
not just on the one with the default route activated. However, this new
behaviour is actually more practical for the user. Accessing applications
will be possible through any of the mgmt ports, without having to know
which port is being used by the NI for the default route. After all,
NI with the uplink label is configured to use whichever mgmt port is
working, meaning that the user should ensure that opening ports on any
of them is safe in their environment. It therefore makes no difference
if port forwarding rules are always activated on all the managment ports
vs. only on one of them.

Lastly, to allow cost-based port selection for routes with multiple
output ports, some of which are possibly app-shared and not enabled
for management, we should allow the controller to set cost to also
app-shared ports (this change only affects the controller, EVE API
already does not limit cost attribute to only mgmt ports).

[eriknordmark]

@milan-zededa I think it makes sense generalizing what we have. However, for the connectivity testing we currently at least partially depend on them being management ports - we check connectivity to the controller. We also check connectivity to the default router. I wonder whether that default router check would be sufficient for non-manegement ports. (If not we have to expose the ability to configure probe target IPs and/or URLs to the controller, UI, etc.)

[eriknordmark]

Don't we need STP plus BPDU guard? Or will STP catch the common loops?
( I know from experience that STP doesn't catch loops if a port or cable is defective and only passes traffic in one direction, but that is an uncommon case.)

[milan-zededa]

It probably make sense to automatically enable BPDU guard on every application VIF connected to the switch NI. These are effectively leaf (edge) ports and we do not expect them to participate in STP. It would also make sense if these ports were immediately set to the forwarding state. However, I do not see feature similar to CISCO's proprietary "Fast-Enabled port" being available on the Linux bridge.
Perhaps it also make sense to allow the user to select a subset of physical ports which are also expected to be leafs, or where the downstream bridges are not fully trusted. This can also be a string attribute expecting a port label (selecting zero or more ports). But in that case, we will also have to allow the user to re-enable ports that were previously blocked. Due to the declarative nature of the config, this would have to be a counter attribute.

[milan-zededa]

@milan-zededa I think it makes sense generalizing what we have. However, for the connectivity testing we currently at least partially depend on them being management ports - we check connectivity to the controller. We also check connectivity to the default router. I wonder whether that default router check would be sufficient for non-manegement ports. (If not we have to expose the ability to configure probe target IPs and/or URLs to the controller, UI, etc.)

True, I will think about how to allow configuring probing endpoint for any port, and ideally also how to unify it with the cellular connectivity testing (https://github.com/milan-zededa/eve-api/blob/adapter-labels/proto/config/netconfig.proto#L91-L111),

[naiming-zededa]

How will the static routing implementation be affected this change?

[milan-zededa]

How will the static routing implementation be affected this change?

Currently, NI routing table contains only routes corresponding to the single selected port (selected at the given moment by connectivity testing). This also applies to static IP routes, i.e. we do not install static routes that at the given moment do not point to the selected port.
What I would like to change, is that the port connectivity testing would only apply to the default route selection. Link-local and (non-default) static IP routes of all ports matched by NI label would always be present in the NI routing table.
@naiming-zededa Do you think this may break user expectation or not be suitable for some scenarios?

[naiming-zededa]

@milan-zededa I think this is fine. Only that all the ip routes can be applied onto various interfaces with gateway address matching their interface prefixes, and default route most likely will be propagated since there is mix of mgmt and app-shared, all the connected routes can be propagated if the knob is selected.