Read *.txt configuration files from https://www.ardusimple.com/configuration-files/

[wdwalker]

Thank you for this great set of tools. It's helping me work with an ArduSimple simpleRTK2B board on Mac OS X.

I'm curious if it is possible to read *.txt configuration files from https://www.ardusimple.com/configuration-files/. In particular, the v1.13 "RAW data (PPK) over UART1 & USB at 1Hz" configuration file.

From what I can see, the PyGPIClient UI only allows loading from binary files, but I could be wrong and not looking in the right spots. I've also searched around for tools to convert the *.txt to a *.ubx, but haven't been able to find anything yet.

Thanks again for your tools - so helpful!

[semuadmin]

Hi @wdwalker ,

So, as you can probably see, each line of the *.txt configuration file provided by ArduSimple contains the UBX message class/id as a string, followed by the payload as space-delimited hexadecimals, e.g.

CFG-VALGET - 06 8B 44 01 01 00 etc.

You can convert this format into a binary *.ubx file fairly easily using helper functions provided by the pyubx2 library.

Unfortunately, there is an added complication. The *.txt file contains MON-VER and CFG-VALGET messages. These are GET messages i.e. they are sent from the receiver in response to configuration polls. In order to actually configure a receiver, you'd need to discard the MON-VER messages (these are for information only) and convert the CFG-VALGET messages into CFG-VALSET messages before sending these to the u-blox receiver (I'm guessing u-center does this internally).

The code example below illustrates how to convert *.txt files to *.ubx files but it doesn't currently do the necessary conversion from CFG-VALGET to CFG-VALSET. I'd need to look into this a little further - please leave it with me for now.

Sample code:

"""
txt2ubx.py

Simple illustration of how to convert *.txt UBX 
configuration files into binary *.ubx format.

Created on 23 Aug 2022

@author: semuadmin
"""

from pyubx2 import calc_checksum, val2bytes, UBX_HDR, U2


def txt2ubx(fname: str):
    """
    Convert txt configuration file to ubx

    :param fname str: txt config file name
    """

    with open(fname, "r", encoding="utf-8") as infile:
        with open(fname + ".ubx", "wb") as outfile:
            read = 0
            write = 0
            errors = 0
            for line in infile:
                try:
                    read += 1
                    parts = line.replace(" ", "").split("-")
                    payload = bytes.fromhex(parts[-1])
                    cls = payload[0:1]
                    mid = payload[1:2]
                    data = payload[2:]
                    payload = cls + mid + data
                    cksum = calc_checksum(payload)
                    ubx = UBX_HDR + payload + cksum
                    outfile.write(ubx)
                    write += 1
                except Exception as err:  # pylint: disable=broad-exception-caught
                    print(err)
                    errors += 1
                    continue

    print(f"{read} messages read, {write} messages written, {errors} errors")


txt2ubx("simpleRTK2B_FW132_Base-00.txt")

Hope this helps.

[wdwalker]

Thank you so much. This helps a lot. I did find it odd that the commands were 'get' commands instead of 'set' commands, but just assumed it might be some sort of convention or secret sauce. I notice the rest of their config files also use 'get' commands, so maybe your thought about u-connect doing the conversion internally is correct.

This particular config sets up the device to write logs to a microSD card on a shield attached the device. I'm going to see if I can get access to a Windows machine to try out the file.

[semuadmin]

TBH I'm not entirely clear why u-center does it this way, as opposed to simply storing the configuration as a series of binary CFG-VALSET messages (like PyGPSClient does in its Configuration Load/Save/Record facility). I'm guessing a text file may be seen as safer to transport than a binary file, or maybe it's to allow people to edit the config file more easily (assuming they understand the hexadecimal values to use), but it's frustrating to have to use u-center's config facility in order to apply it. As a binary *.ubx file, you could send it from any terminal utility on any platform.

I think the conversion is fairly simple to do, and I'll aim to add this functionality into PyGPSClient in the next release (1.3.25).

[semuadmin]

Hi @wdwalker

FYI I've uploaded a modified version of the code snippet above which now does the full conversion to a 'SET' configuration file:

https://github.com/semuconsulting/PyGPSClient/blob/RC-1.3.25/examples/txt2ubx.py

This utility takes a *.txt configuration file as an input and produces two *.ubx output files:

1. The *get.ubx file is a direct transcription of the input file into binary format. This can be parsed by PyGPSClient like any other 'GET' UBX receiver datalog file, but it cannot be used to configure the receiver.
2. The *set.ubx file converts all the CFG-VALGET messages into CFG-VALSET 'SET' commands, which can be used to configure the receiver. You could load it into PyGPSClient's Configuration Load/Save/Record facility and send it to the receiver from there, or simply send it directly to the receiver via any serial terminal utility.

You can also parse the SET file using PyGPSClient but remember to set the input mode to 'SET' rather than the default 'GET' - the setting for this is hidden in the drop-down serial configuration panel.

I've tested this using my own F9P and it all seems to work OK.

What I'll probably do in due course is incorporate this conversion functionality into PyGPSClient's existing Load/Save/Record facility, automatically detecting if the config file is in *.txt or *.ubx format.

[wdwalker]

You rock, @semuadmin! I just gave this a shot and the conversion seems to work well. Unfortunately, I'm not getting the end result I'm looking for (writing to the microSD card), but that's most likely on my side and not yours. I'll keep poking -- I must say it is nice to not have to use Windows. Thanks so much.

[semuadmin]

The acid test is whether you're getting ACK-ACK responses from the receiver when you upload the config. You should see as many ACKs as there are CFG-VALGET commands in the original *.txt file. If you use PyGPSClient's Load/Save/Record facility, for example, it will tell you how many commands there are in the file, and you can monitor the corresponding ACKs in the console after pressing 'send'. If you're definitely using the config file appropriate to your F9P firmware version but are seeing ACK-NAKs (or no ACK at all), by all means get back to me.

FYI If you fancy a sneak preview of PyGPSClient v1.3.25, which now includes automatic conversion of *.txt config files, here's the install command...

python3 -m pip install git+https://github.com/semuconsulting/pygpsclient.git@RC-1.3.25

Go to the CFG Configuration Load/Save/Record panel illustrated above, click the 'open file' icon and select the *.txt file you want. The utility should now automatically convert the file to a series of in-memory CFG-VALSET commands which can be uploaded to the receiver, or saved as a binary *.ubx config file.

[semuadmin]

Feature now available in v1.3.25