-
Notifications
You must be signed in to change notification settings - Fork 2k
Border router using USB CDC ECM (ethernet over USB)
This is a generic guide explaining how to set up a border router using the USB CDC ECM feature from RIOT. This allows for setting up a 6LoWPAN border gateway router with global connectivity provided over USB ethernet. The end result should be a setup with two nodes, one acting as a gateway for the other. Both nodes should be able to ping each other and ping the host computer the gateway is connected to.
When using ethos
and gnrc_uhcpc
with the gnrc_border_router
example, these steps are applied automatically by either the host side scripting or by the gnrc_uhcpc.c
code.
Technically nothing obstructs from using this tooling with USB CDC ECM, however the steps are shown here manually with explanation to give the reader insight into the requirements for routing and connectivity
Shell commands are provided for a Linux based operating system. macOS should follow similar steps.
- Commands requiring administrative access are prefixed with
#
. - Commands that do not require administrative access are prefixed with
$
. - Commands that must be executed on a RIOT node are prefixed with
>
.
On Linux all network related commands are using the iproute2 tools.
- A computer acting as the USB host.
- A board with a
periph_usbdev
driver and a radio. - A board with at least a radio.
- Two IPv6 prefixes. This guide is using
2001:db8:1::/64
and2001:db8:2::/64
Boards based on the ATSAMR21G18A MCU or the nRF52840 are suitable.
This includes the samr21-xpro
and the nrf52840dk
boards (note that the nrf52840-preview-dk
boards have a flawed usb peripheral).
As a host computer both Linux and macOS should work. Windows is currently not supported out of the box as a CDC-ECM driver appears to be missing by default.
The board with both the periph_usbdev
available and a radio available (henceforth called The Border Router) should be connected to the host with both the debug interface for programming and shell. The other node (henceforth called The Node) only needs connectivity via the debug/serial interface.
Administrative permissions might be required on the host computer to set up proper routing.
The Border Router is based on a slightly modified example of the gnrc_border_router
.
The main set of modules required is:
-
usbus_cdc_ecm
: for USB CDC ECM functionality -
auto_init_usbus
: to automatically initialize the USB stack -
gnrc_netdev_default
: to pull in the default gnrc modules including radio drivers -
auto_init_gnrc_netif
: to automatically initialize the GNRC network stack -
gnrc_icmpv6_echo
: not strictly required but useful here for testing withping6
-
shell
andshell_commands
: to have the shell ready and useful for interaction.
Furthermore in the makefile, GNRC_NETIF_NUMOF
should be at least set to 2 to allow for both the CDC ECM netif interface and the wireless interface.
Also don't forget to specify an USB vendor ID and product ID like:
USB_VID ?= YOUR_VID # Replace with your vendor ID
USB_PID ?= YOUR_PID # Replace with your product ID
CFLAGS += -DUSB_CONFIG_VID=0x$(USB_VID) -DUSB_CONFIG_PID=0x$(USB_PID)
This is a basic stripped makefile that results in a working border router.
Don't forget to adapt the BOARD
, add any custom radio driver (e.g. mrf24j40, cc2420) and replace USB_VID
and USB_PID
The Border Router makefile
APPLICATION = usb_cdc_ecm_border_router
BOARD ?= samr21-xpro
RIOTBASE ?= $(CURDIR)/../..
GNRC_NETIF_NUMOF := 2
USEMODULE += usbus_cdc_ecm
USEMODULE += auto_init_usbus
USEMODULE += gnrc_netdev_default
USEMODULE += auto_init_gnrc_netif
USEMODULE += gnrc_icmpv6_error
USEMODULE += gnrc_sixlowpan_border_router_default
USEMODULE += gnrc_udp
USEMODULE += gnrc_rpl
USEMODULE += auto_init_gnrc_rpl
USEMODULE += gnrc_icmpv6_echo
USEMODULE += shell
USEMODULE += shell_commands
# Optional, but might be useful
USEMODULE += ps
USEMODULE += netstats_l2
USEMODULE += netstats_ipv6
USEMODULE += netstats_rpl
USB_VID ?= YOUR_VID # Replace with your vendor ID
USB_PID ?= YOUR_PID # Replace with your product ID
CFLAGS += -DUSB_CONFIG_VID=0x$(USB_VID) -DUSB_CONFIG_PID=0x$(USB_PID)
DEVELHELP ?= 1
include $(RIOTBASE)/Makefile.include
The Node can use the gnrc_networking
example as that fits the requirements.
The most notable differences with The Border Router are:
-
gnrc_sixlowpan_router_default
instead ofgnrc_sixlowpan_border_router_default
: No border router functionality and settings required, router (as in multi-hop router) is enough for The Node. - No USB-related modules: Not required as The Node won't be hooked up to a host computer.
This is a basic stripped makefile for The Node:
The Node makefile
APPLICATION = node
BOARD ?= samr21-xpro
RIOTBASE ?= $(CURDIR)/../..
USEMODULE += gnrc_netdev_default
USEMODULE += gnrc_sixlowpan_router_default
USEMODULE += gnrc_icmpv6_errorWhen using `ethos` and `gnrc_uhcpc` with the `gnrc_border_router` example, these steps are applied automatically by either the [host side scripting](https://github.com/RIOT-OS/RIOT/blob/master/dist/tools/ethos/setup_network.sh) or by the [`gnrc_uhcpc.c` code](https://github.com/RIOT-OS/RIOT/blob/master/sys/net/gnrc/application_layer/uhcpc/gnrc_uhcpc.c#L57).
Technically nothing obstructs from using this tooling with USB CDC ECM, however the steps are shown here manually with explanation to give the reader insight into the requirements for routing and connectivity
USEMODULE += gnrc_udp
USEMODULE += gnrc_rpl
USEMODULE += gnrc_icmpv6_echo
USEMODULE += auto_init_gnrc_netif
USEMODULE += auto_init_gnrc_rpl
USEMODULE += shell
USEMODULE += shell_commands
# Optional but useful
USEMODULE += ps
USEMODULE += netstats_l2
USEMODULE += netstats_ipv6
USEMODULE += netstats_rpl
DEVELHELP ?= 1
include $(RIOTBASE)/Makefile.include
When using ethos
with the gnrc_border_router
example, a number of network configuration is applied automatically by the scripts.
Unfortunately, this is currently not available for USB CDC ECM.
Some manual configuration is required.
This can be done both manually via the shell, or included in the main.c
code.
The guide here will use the shell to configure networking.
Adding code to the main.c
file to configure networking for The Border Router is intentionally left as an exercise to the reader.
First the network connectivity is set up between the USB host computer and The Border Router.
When plugging in The Border Router into the host, Something along the following lines will appear in dmesg
:
Kernel log:
# dmesg
…
[514571.217526] usb 3-9.2.3: new full-speed USB device number 69 using xhci_hcd
[514571.295639] usb 3-9.2.3: New USB device found, idVendor=1209, idProduct=0001, bcdDevice= 0.00
[514571.295640] usb 3-9.2.3: New USB device strings: Mfr=3, Product=2, SerialNumber=0
[514571.295641] usb 3-9.2.3: Product: USB device
[514571.295642] usb 3-9.2.3: Manufacturer: RIOT-os.org
[514571.305205] cdc_ether 3-9.2.3:1.0 usb0: register 'cdc_ether' at usb-0000:00:14.0-9.2.3, CDC Ethernet Device, 5a:8b:05:3a:4e:45
[514571.328682] cdc_ether 3-9.2.3:1.0 enp0s20u9u2u3: renamed from usb0
[514571.345439] IPv6: ADDRCONF(NETDEV_UP): enp0s20u9u2u3: link is not ready
Shown here is the detection and enumeration of the new USB device.
If this step fails, the periph_usbdev
implementation for the board or the USBUS stack is not working correctly.
Linux or macOS repeatedly logging USB device resets is one of the signs of a failing USB device.
More interesting is the resulting name (enp0s20u9u2u3
here).
On most recent Linux based systems the name is generated based on the chain of USB controllers and hubs.
Comparing the name with the output of lsusb -t
USB device listing:
$ lsusb -t
…
/: Bus 03.Port 1: Dev 1, Class=root_hub, Driver=xhci_hcd/14p, 480M
|__ Port 9: Dev 6, If 0, Class=Hub, Driver=hub/4p, 480M
|__ Port 2: Dev 8, If 0, Class=Hub, Driver=hub/4p, 480M
|__ Port 3: Dev 69, If 0, Class=Communications, Driver=cdc_ether, 12M
|__ Port 3: Dev 69, If 1, Class=CDC Data, Driver=cdc_ether, 12M
…
In this case, the USB device is connected to the USB host controller at PCI address 00:1a.0, resulting in the enp0s20
part.
The USB device is connected via two hubs, the first one at port 9 of the USB host controller, the second one at port 2 of the first hub.
The device itself is connected to the port 3 of the second hub.
This results in the u9u2u3
part of the naming scheme, simply by chaining the port numbers of the hubs.
In this case, the CDC ECM functionality is at USB device interface 0 of the USB device (If 0
at the lsusb -t
output).
If this is not the case, the name gets an additional f
suffix with the USB device interface number.
This results in predictable, but sometimes long and hard to remember network interface names.
Don't worry about the IPv6: ADDRCONF(NETDEV_UP): enp0s20u9u2u3: link is not ready
message for now.
Inspecting the network device via ethtool
:
$ ethtool enp0s20u9u2u3
Settings for enp0s20u9u2u3:
Cannot get wake-on-lan settings: Operation not permitted
Current message level: 0x00000007 (7)
drv probe link
Link detected: yes
Shown here is that the network interface reports the link as detected. The RIOT USB CDC ECM implementation reports this together with the link speed to the host operating system.
If one of these steps fail or the interface doesn't show up at the host, either the host operating system doesn't have a CDC ECM driver or the CDC ECM implementation in RIOT doesn't adhere to what the driver expects.
Most hosts auto detect the CDC ECM functionality based on the bInterfaceClass
and bInterfaceSubClass
values and they don't require a specific USB vendor ID or USB product ID to function.
At this point it is assumed that the host reports a functional USB network interface.
A freshly booted Border Router should show something like this when executing ifconfig
on it:
Network configuration on The Border Router
> ifconfig
Iface 7 HWaddr: 74:3E Channel: 26 Page: 0 NID: 0x23
Long HWaddr: 79:67:18:60:42:0C:F4:3E
TX-Power: 0dBm State: IDLE max. Retrans.: 3 CSMA Retries: 4
AUTOACK ACK_REQ CSMA L2-PDU:102 MTU:1280 HL:64 RTR
RTR_ADV 6LO IPHC
Source address length: 8
Link type: wireless
inet6 addr: fe80::7b67:1860:420c:f43e scope: local VAL
inet6 group: ff02::2
inet6 group: ff02::1
inet6 group: ff02::1:ff0c:f43e
Statistics for Layer 2
RX packets 0 bytes 0
TX packets 0 (Multicast: 0) bytes 0
TX succeeded 0 errors 0
Statistics for IPv6
RX packets 0 bytes 0
TX packets 0 (Multicast: 0) bytes 0
TX succeeded 0 errors 0
Iface 8 HWaddr: 5A:8B:05:3A:4E:45
L2-PDU:1500 MTU:1500 HL:64 RTR
RTR_ADV Source address length: 6
Link type: wired
inet6 addr: fe80::588b:5ff:fe3a:4e45 scope: local VAL
inet6 group: ff02::2
inet6 group: ff02::1
inet6 group: ff02::1:ff3a:4e45
Statistics for Layer 2
RX packets 4 bytes 432
TX packets 1 (Multicast: 1) bytes 0
TX succeeded 0 errors 0
Statistics for IPv6
RX packets 4 bytes 376
TX packets 1 (Multicast: 1) bytes 64
TX succeeded 1 errors 0
Shown is a wireless interface as Iface 7
and a wired interface as Iface 8
, recognizable by the Link type
in the output.
Both have a link local address and a set of multicast subscription groups.
At this point it should be possible for
Verify that the host side reports the network interface as UP
or set it as UP
if necessary:
$ ip link show enp0s20u9u3
43: enp0s20u9u2u3: <BROADCAST,MULTICAST> mtu 1500 qdisc fq_codel state DOWN group default qlen 1000
link/ether 5a:8b:05:3a:4e:45 brd ff:ff:ff:ff:ff:ff
# ip link set up enp0s20u9u2u3 # Must be root for this
$ ip link show enp0s20u9u2u3
43: enp0s20u9u2u3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UNKNOWN mode DEFAULT group default qlen 1000
link/ether 5a:8b:05:3a:4e:45 brd ff:ff:ff:ff:ff:ff
To add the address on the host side:
# ip address add 2001:db8:1::1/64 dev enp0s20u9u2u3 # Must be done as root
$ ip address show enp0s20u9u2u3
43: enp0s20u9u2u3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UNKNOWN group default qlen 1000
link/ether 5a:8b:05:3a:4e:45 brd ff:ff:ff:ff:ff:ff
inet6 2001:db8:1::1/64 scope global
valid_lft forever preferred_lft forever
Of course, it is also possible to use NetworkManager or other configuration applications to configure this interface.
On the RIOT side use ifconfig
to add a static address to the interface.
Please make sure not to use the same address as used on the host side.
> ifconfig 8 add 2001:db8:1::2/64
success: added 2001:db8:1::2/64 to interface 8
> ifconfig 8
Iface 8 HWaddr: 5A:8B:05:3A:4E:45
L2-PDU:1500 MTU:1500 HL:64 RTR
RTR_ADV Source address length: 6
Link type: wired
inet6 addr: fe80::588b:5ff:fe3a:4e45 scope: local VAL
inet6 addr: 2001:db8:1::2 scope: global VAL
inet6 group: ff02::2
inet6 group: ff02::1
inet6 group: ff02::1:ff3a:4e45
inet6 group: ff02::1:ff00:2
Statistics for Layer 2
RX packets 17 bytes 2519
TX packets 1 (Multicast: 1) bytes 0
TX succeeded 0 errors 0
Statistics for IPv6
RX packets 17 bytes 2281
TX packets 1 (Multicast: 1) bytes 64
TX succeeded 1 errors 0
Note that both the requested address and the solicited-node multicast address have been added to the interface.
Ping between The Border Router and the host computer should now work:
> ping6 2001:db8:1::1
12 bytes from 2001:db8:1::1: icmp_seq=1 ttl=64 time=0.907 ms
12 bytes from 2001:db8:1::1: icmp_seq=2 ttl=64 time=0.917 ms
--- 2001:db8:1::1 PING statistics ---
3 packets transmitted, 2 packets received, 33% packet loss
round-trip min/avg/max = 0.907/0.912/0.917 ms
The first ping echo request might be lost due to delays caused by address resolving.
And from the host:
$ ping6 2001:db8:1::2 -c3
PING 2001:db8:1::2(2001:db8:1::2) 56 data bytes
64 bytes from 2001:db8:1::2: icmp_seq=1 ttl=64 time=1.07 ms
64 bytes from 2001:db8:1::2: icmp_seq=2 ttl=64 time=1.08 ms
64 bytes from 2001:db8:1::2: icmp_seq=3 ttl=64 time=1.12 ms
--- 2001:db8:1::2 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 4ms
rtt min/avg/max/mdev = 1.074/1.091/1.119/0.033 ms
At this point The Border Router is almost fully functional.
The Node itself requires no configuration. All configuration happens on The Border Router.
First an address from the other IPv6 prefix is added to the wireless interface of The Border Router:
> ifconfig 7 add 2001:db8:2:0:7b67:1860:420c:f43e/64
success: added 2001:db8:2::7b67:1860:420c:f43e/64 to interface 7
> ifconfig 7
Iface 7 HWaddr: 74:3E Channel: 26 Page: 0 NID: 0x23
Long HWaddr: 79:67:18:60:42:0C:F4:3E
TX-Power: 0dBm State: IDLE max. Retrans.: 3 CSMA Retries: 4
AUTOACK ACK_REQ CSMA L2-PDU:102 MTU:1280 HL:64 RTR
RTR_ADV 6LO IPHC
Source address length: 8
Link type: wireless
inet6 addr: fe80::7b67:1860:420c:f43e scope: local VAL
inet6 addr: 2001:db8:2:0:7b67:1860:420c:f43e scope: global VAL
inet6 group: ff02::2
inet6 group: ff02::1
inet6 group: ff02::1:ff0c:f43e
Statistics for Layer 2
RX packets 13 bytes 559
TX packets 0 (Multicast: 0) bytes 0
TX succeeded 0 errors 0
Statistics for IPv6
RX packets 13 bytes 832
TX packets 0 (Multicast: 0) bytes 0
TX succeeded 0 errors 0
The address selected here (2001:db8:2:0:7b67:1860:420c:f43e
) is chosen as such that it would match the auto-generated address for this interface.
The second step to take is to initialize the RPL routing protocol to enable multi-hop networking.
> rpl init 7
successfully initialized RPL on interface 7
> rpl root 0 2001:db8:2:0:7b67:1860:420c:f43e
successfully added a new RPL DODAG
These commands first initialize RPL for interface 7 (the wireless interface), then a new RPL DODAG instance is added with RPL instance ID 0
and 2001:db8:2:0:7b67:1860:420c:f43e
as prefix.
The prefix intentionally matches the address of the wireless interface here.
With these two actions, RPL is active on The Border Router and should transmit DODAG Information Objects.
The Node should receive the DODAG Information Objects from The Border Router and should configure an address based on the prefix. For example on The Node:
> ifconfig
Iface 6 HWaddr: 22:A0 Channel: 26 NID: 0x23
Long HWaddr: 6B:AE:96:BB:79:F5:22:A0
TX-Power: 0dBm L2-PDU:102 MTU:1280 HL:64 RTR
6LO IPHC
Source address length: 8
Link type: wireless
inet6 addr: fe80::69ae:96bb:79f5:22a0 scope: local VAL
inet6 addr: 2001:db8:2:0:69ae:96bb:79f5:22a0 scope: global VAL
inet6 group: ff02::2
inet6 group: ff02::1
inet6 group: ff02::1:fff5:22a0
inet6 group: ff02::1a
Statistics for Layer 2
RX packets 15 bytes 1060
TX packets 13 (Multicast: 11) bytes 877
TX succeeded 13 errors 0
Statistics for IPv6
RX packets 15 bytes 1340
TX packets 13 (Multicast: 11) bytes 1140
TX succeeded 13 errors 0
As shown, The Node now has both a link local address (fe80::69ae:96bb:79f5:22a0
) and a globally reachable address (2001:db8:2:0:69ae:96bb:79f5:22a0
).
At this point ping6
between the two RIOT boards should work.
From The Border Router:
> ping6 2001:db8:2:0:69ae:96bb:79f5:22a0
12 bytes from 2001:db8:2:0:69ae:96bb:79f5:22a0: icmp_seq=0 ttl=64 rssi=-61 dBm time=7.421 ms
12 bytes from 2001:db8:2:0:69ae:96bb:79f5:22a0: icmp_seq=1 ttl=64 rssi=-62 dBm time=8.081 ms
12 bytes from 2001:db8:2:0:69ae:96bb:79f5:22a0: icmp_seq=2 ttl=64 rssi=-62 dBm time=8.057 ms
--- 2001:db8:2:0:69ae:96bb:79f5:22a0 PING statistics ---
3 packets transmitted, 3 packets received, 0% packet loss
round-trip min/avg/max = 7.421/7.853/8.081 ms
And from The Node:
> ping6 2001:db8:2:0:7b67:1860:420c:f43e
12 bytes from 2001:db8:2:0:7b67:1860:420c:f43e: icmp_seq=0 ttl=64 rssi=-58 dBm time=8.387 ms
12 bytes from 2001:db8:2:0:7b67:1860:420c:f43e: icmp_seq=1 ttl=64 rssi=-59 dBm time=8.685 ms
12 bytes from 2001:db8:2:0:7b67:1860:420c:f43e: icmp_seq=2 ttl=64 rssi=-59 dBm time=9.027 ms
--- 2001:db8:2:0:7b67:1860:420c:f43e PING statistics ---
3 packets transmitted, 3 packets received, 0% packet loss
round-trip min/avg/max = 8.387/8.699/9.027 ms
However ping6
from The Node to the host computer does not work:
ping6 2001:db8:1::1
--- 2001:db8:1::1 PING statistics ---
3 packets transmitted, 0 packets received, 100% packet loss
Direct connectivity is working between The Node and The Border Router and between The Border Router and the host computer. However connectivity between The Node and the host computer is not working. This is because The Border Router does not know that the traffic from The Node must be routed to the host computer. Looking at the routing table of The Node:
> nib route
2001:db8:2::/64 dev #6
default* via fe80::7b67:1860:420c:f43e dev #6
It shows two routes, the first is the prefix (2001:db8:2::/64
) as local to the network interface and the second is a default route to link local address of The Border Router.
On The Border Router:
> nib route
2001:db8:1::/64 dev #8
2001:db8:2::/64 dev #7
2001:db8:2:0:69ae:96bb:79f5:22a0/128 via fe80::69ae:96bb:79f5:22a0 dev #7
Here it shows both prefixes, the first one from the wired interface and the second one from the wireless interface. Another route is available provided by RPL indicating via which next hop The Node is reachable. In a real multi-hop scenario, these routes show via which next node a different node, requiring multiple hops to reach, is reachable.
The missing bit in this routing table however is a default route, telling The Border Router via which to route any traffic it is not the final destination for.
On The Border Router:
> nib route add 8 ::/0 2001:db8:1::1
> nib route
2001:db8:1::/64 dev #8
2001:db8:2::/64 dev #7
2001:db8:2:0:69ae:96bb:79f5:22a0/128 via fe80::69ae:96bb:79f5:22a0 dev #7
default* via 2001:db8:1::1 dev #8
And the host computer requires information that the prefix 2001:db8:2::/64
is reachable via The Border Router, otherwise it would attempt to reply via it's own default gateway.
Usually this would result in the host computer sending the reply over it's own (regular) wired or WiFi interface.
To configure this additional route on Linux via the command line:
# ip route add 2001:db8:2::/64 via 2001:db8:1::2
$ ip -6 route show 2001:db8:2::/64
2001:db8:2::/64 via 2001:db8:1::2 dev enp0s20u9u2u3 metric 1024 pref medium
It is also possible to add this configuration to NetworkManager or a different network configuration application.
With these two additional settings, The Border Router is aware that traffic should be routed to the host computer. Furthermore, the host computer is aware that there is a second prefix which is reachable via The Border Router.
At this point ping6
between The Node and the host computer is fully functional:
From The Node:
> ping6 2001:db8:1::1
12 bytes from 2001:db8:1::1: icmp_seq=0 ttl=63 rssi=-59 dBm time=10.311 ms
12 bytes from 2001:db8:1::1: icmp_seq=1 ttl=63 rssi=-60 dBm time=10.155 ms
12 bytes from 2001:db8:1::1: icmp_seq=2 ttl=63 rssi=-60 dBm time=8.195 ms
--- 2001:db8:1::1 PING statistics ---
3 packets transmitted, 3 packets received, 0% packet loss
round-trip min/avg/max = 8.195/9.553/10.311 ms
And from the host computer:
$ ping6 2001:db8:2:0:69ae:96bb:79f5:22a0 -c3
PING 2001:db8:2:0:69ae:96bb:79f5:22a0(2001:db8:2:0:69ae:96bb:79f5:22a0) 56 data bytes
64 bytes from 2001:db8:2:0:69ae:96bb:79f5:22a0: icmp_seq=1 ttl=63 time=15.1 ms
64 bytes from 2001:db8:2:0:69ae:96bb:79f5:22a0: icmp_seq=2 ttl=63 time=17.1 ms
64 bytes from 2001:db8:2:0:69ae:96bb:79f5:22a0: icmp_seq=3 ttl=63 time=16.5 ms
--- 2001:db8:2:0:69ae:96bb:79f5:22a0 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 4ms
rtt min/avg/max/mdev = 15.123/16.234/17.125/0.838 ms
With this, the promised connectivity between the host computer and The Node is working.
Shown are the steps required to manually achieve the connectivity.
Automating this process on the startup of The Border Router is up to the reader.
Another option is to use the gnrc_uhcpc
code to delegate a prefix to the wireless interface of The Border Router.