Trying to set configuration from configuration file

[MelvinWarnet]

Hi,

I use many Ublox GPS ZED-F9P and I want to configure them automatically. For instance, I use U-Center to 1-"Update the firmware" from a file on my computer, 2-"Load configuration" from a configuration file on my computer generated by U-Center in a .txt format.

I use an Ubuntu 18.04.5 version with python 3.6.9 so I use an old version of pyubx2 (1.2.14). I have already check examples "ubxconfigdb.py" and "ubxfile.py" but I don't really understand how they work and how I can use them to achieve my goal. It's important to precise that I just discover the UBX protocol so I don't really understand all its specifications.

I think there is already someone who make something similar, but I can't find him.

Thanks by advanced !

[semuadmin]

Hi @MelvinWarnet I'm converting this issue to a Discussion as it's not a code issue in pyubx2 as such.

[semuadmin]

Firstly, please be aware that neither PyGPSClient nor pyubx2 can update the firmware on any u-blox device. This involves the use of private, proprietary message types and procedures which u-blox will not permit me to include in these open source libraries. I have tried to remonstrate with u-blox, but they are adamant that this functionality be excluded.

[MelvinWarnet]

Okay, so I can give up the idea of update the firmware using a Python librairy but maybe there is a solution for using U-Center's command line on Ubuntu 18.04.5 ? Do you know something about it ?
Thanks

[semuadmin]

I'm not entirely clear what you mean by 'using U-Center's command line'.

The PyGPSClient GUI application is (more or less) an open source equivalent to u-center and runs on MacOS and most flavours of Linux. It includes comprehensive UBX configuration capabilities including the ability to record or load u-center *.ubx configuration files (also *.txt files as provided by ArduSimple for the F9P) - see the README for details.

If you want an actual CLI utility to use at the Ubuntu terminal, there is also the pygnssutils library which includes two very basic CLI utilities ubxsave and ubxload.

Hope this helps

PS Ubuntu 18.04 is now in extended support - recommend upgrading to Ubuntu 22.04 LTS if you can.

[MelvinWarnet]

When I talk about "U-center's command line", I mean that we can use the ucenter.exe directly in a cmd. If I can reproduce this on Ubuntu 18.04.5 in a terminal, i will be able to update de firmware and load a configuration without using the U-Center GUI.
I planified to use Wine to run the .exe on Ubuntu but i'm on a Nvidia Jetson Nano which can't use 32bits architecture, so i'm facing a new problem...

[semuadmin]

Okay, so I can give up the idea of update the firmware using a Python librairy

To clarify, it's perfectly possible to create a u-blox firmware update utility using Python, but you'd have to obtain the information directly from u-blox and they'll probably ask you to sign a Non-Disclosure Agreement (NDA). I'm not permitted to publish the information myself here.

[MelvinWarnet]

Okay I see, my program has to be Open-Source so I can't sign something like that.

[semuadmin]

I can't help you with firmware updates, but here's a simple pyubx2 code snippet you could run at the Ubuntu command line which uploads the contents of a u-center *.txt configuration file to a u-blox receiver at a designated serial port:

python3 load_txt_config.py port="/dev/ttyACM0" baud=38400 infile="ucenter.txt"

"""
load_txt_config.py

Loads contents of u-center *.txt configuration file into u-blox
receiver via designed serial port.

Usage:

   python3 load_txt_config.py port="/dev/ttyACM0" baud=38400 infile="ucenter.txt"

or, for dry run without updating receiver:

   python3 load_txt_config.py infile="ucenter.txt" dryrun=1

"""

from sys import argv

from serial import Serial

from pyubx2 import (
    POLL_LAYER_BBR,
    POLL_LAYER_FLASH,
    SET,
    SET_LAYER_BBR,
    SET_LAYER_FLASH,
    SET_LAYER_RAM,
    TXN_NONE,
    U1,
    UBXMessage,
    bytes2val,
    val2bytes,
)

CFG = b"\x06"
VALGET = b"\x8b"
VALSET = b"\x8a"


def load_txt(fname: str, output: Serial) -> int:
    """
    Load u-center format text configuration file.

    Any messages other than CFG-MSG, CFG-PRT or CFG-VALGET are discarded.
    The CFG-VALGET messages are converted into CFG-VALGET.

    :param str fname: fully qualified input file name
    :param Serial output: output serial stream
    :return: no of items read
    :rtype: int
    """
    # pylint: disable=too-many-locals, broad-exception-caught

    try:
        with open(fname, "r", encoding="utf-8") as file:
            i = 0
            for line in file:
                parts = line.replace(" ", "").split("-")
                data = bytes.fromhex(parts[-1])
                cls = data[0:1]
                mid = data[1:2]
                if cls != CFG:
                    continue
                if mid == VALGET:  # config database command
                    version = data[4:5]
                    layer = bytes2val(data[5:6], U1)
                    if layer == POLL_LAYER_BBR:
                        layers = SET_LAYER_BBR
                    elif layer == POLL_LAYER_FLASH:
                        layers = SET_LAYER_FLASH
                    else:
                        layers = SET_LAYER_RAM
                    layers = val2bytes(layers, U1)
                    transaction = val2bytes(TXN_NONE, U1)  # not transactional
                    reserved0 = b"\x00"
                    cfgdata = data[8:]
                    payload = version + layers + transaction + reserved0 + cfgdata
                    parsed = UBXMessage(CFG, VALSET, SET, payload=payload)
                else:  # legacy CFG command
                    parsed = UBXMessage(CFG, mid, SET, payload=data[4:])
                if parsed is not None:
                    # send CFG message to receiver
                    print(parsed)
                    if output is not None:
                        output.write(parsed.serialize())
                    i += 1
    except Exception:
        print(f"ERROR parsing {fname}!")
        return 0

    print(f"{i} configuration commands processed")
    return i


def main(**kwargs):
    """
    Main routine.
    """

    infile = kwargs.get("infile", "ucenter.txt")
    port = kwargs.get("port", "/dev/ttyACM0")
    baud = int(kwargs.get("baud", 38400))
    dryrun = int(kwargs.get("dryrun", 0))

    if dryrun:
        load_txt(infile, None)
    else:
        with Serial(port, baud, timeout=5) as stream:
            load_txt(infile, stream)


if __name__ == "__main__":

    main(**dict(arg.split("=") for arg in argv[1:]))

[MelvinWarnet]

I start to test your code and I start to have very good results ! Moreover, it permit to me to get a better understanding of the protocol !
Thank you very much, and I will come back here if I have any other questions

[semuadmin]

Glad it was useful. If you're happy with the response, I'd be grateful if you could tick the "Mark as answer" box. Thanks

[MelvinWarnet]

After testing deeply the code, (I modified it a little bit) I have two more interogations :

• How to set configurations in flash memory and not in RAM, ? Cause when I forced it to set in flash, the serial seems to be blocked and nothing more append
• In the UBX protocol I can read that VALSET message can't send more than 64 key-value pairs, but in my case some messages containt more than 600 pairs, so the solution is maybe to cut them in smaller parts
  I join you some file to better understanding the problem.
  test_conf.txt
  script.txt