OTA (Over The Air) Updates

There are currently two different ways to get a firmware onto a brain without needing to connect to it via the USB serial port interface. Before we get into those details let's have a chat about what a firmware is.

On our ESP32 platform, when you flash a board using the ids.py flash command you actually write multiple files to the board like a bootloader, non-volatile memory settings, ota settings, a partition map, an application image and even a filesystem image.

When we "OTA" a board we aren't doing the same operation. Instead we are only updating one of the two OTA application images that are in our partition map. There is a third application image, known as the factory image, which is what gets written when you use the idf.py flash tool. That's kind of a special image because it (hopefully) won't ever be overwritten by anything else. In reality malicious code code do whatever it wants, but we're so not concerned about security right now.

So okay, you've written some awesome new code, compiled it, where is the file you need? Well it's in the build directory and in our case because we named the project brain the file is build/brain.bin. If you pay attention to the first couple of messages that idf.py spits out when it runs you'll see the version that will be baked into the firmware. It will look something like this 0.0.1-450-gad9451b-dirty.

I haven't hunted down exactly where the IDF generates that number, but it's not really necessary because it's pretty self explanatory. The first part before the the first dash we configured somewhere. The next number, 450 in this example, seems to be the number of commits or something like that. Git doesn't really do monotonically increasing build numbers because of the nature of branches, but this is sort of an approximation of that. The next part is the short git hash, so you can tell exactly which commit created this firmware and the last part, the -dirty means that there were local changes. This is kind of totally exactly the right way to deal with version numbers in a git environment.

Alright we've got a file, and we know where to look to find what version number is baked into it - What should you do with these two pieces of information? Well, you should copy that brain.bin file you found in the build directory to a file with the same name as the version and ending in .bin placed in your home directory under sparklemotion/fw. So for our example here we would do the following (starting in the brain/sw/playa directory which is where you built the thing from)

cp build/brain.bin ~/sparklemotion/fw/0.0.1-450-gad9451b-dirty.bin

Boom! You're done. Now when Pinky runs as the same user (you), Pinky will see that firmware image, along with any other ones that have been placed in that directory, and will select the "latest" firmware by virtue of the second element of the version, which I'm pretty sure maps to number of commits.

At the moment there isn't anything fancy going on with version numbers or dirty versus not dirty. Pinky just does a split on the - characters, expects an integer, and hopefully things work out.

This mechanism is built on trust all the way through, so no one is checking the contents of these files, or if they're even runnable images. Espressif has a lot of cryptographic tools we could use to ensure that only "our" firmware gets runs on our brains, but yeah right. That's not the way we play.

The mechanism behind the system described above is that each brain fairly continuously broadcasts a BRAIN_HELLO message which includes it's baked in firmware string (as well as a similar string for the IDF version it was built with). Pinky uses these messages to find the brains in the first place and starts talking to them to tell them what to do. Pinky now has a USE_FIRMWARE message which it can send to a brain to tell it that "Hey, there's this new firmware at this URL, you probably want to be using it".

When a brain gets the USE_FIRMWARE message, it dutifully goes off and does what it has been told to do like a good slave and downloads the firmware then reboots. So as long as you have Pinky around, there is now a way he can push out new firmware to any brains he hears. This might get ugly if we have competing pinky instances with different firmwares, so um, yeah, let's try not to do that.

But wait, there's more!

What if you don't have a Pinky? How do you upload your new firmware without a cable? You connect to the Brain's AP and use it's web interface is how.

Right now, each Brain creates it's own soft access point with an SSID of brain_XXYYZZ where the XXYYZZ part is the last 3 octets of it's base mac address. On that network (which is unique to each brain) the address of the brain is http://10.10.10.10/

If you connect to that network and open that in a web browser you will get whatever interface was last flashed to the filesystem partition of the board. How do you upgrade this filesystem portion you ask? Well, right now you don't. The OTA mechanism is really only about getting the app binary updated. There's not any technical reason that we can't update the filesystem image - it's all just flash memory anyway - it's just not what's there out of the box for us. So yeah, we might get into a little drift between app image versus filesystem image versus factory image, but as is the theme with the Espressif parts, there's more capability here than we're likely to actually use.