cockpit: Implement PCP metrics channel in the Python bridge

[jelly]

Current ToDo's

• test failure on Arch exception
• TF failures, create a TEST VM without python3-pcp
• Aks PCP if they wanna upload a wheel.
• tox
• investigate coverage once more
• Debian-stable/testing lack python3-pcp when running test/image-prepare -q debian-testing it fails on pytest.
• Land this after tomorrow's 325 release

---

Follow ups, I'd suggest tackling:

• Dropping the cockpit-pcp package
• On demand install in metrics page, change it to installing pcp python3-pcp instead of cockpit-pcp
• Dropping the cockpit-pcp C code et all
• Deal with code which checks for the existence of cockpit-pcp (release-blocker)
• Consider fetch group API https://gist.github.com/jelly/6f4d683fcd3f8797945f5f90bf0fe5d6
• Drop networkmanager-basic journal allow for PCP (C bridge): test: drop cockpit-pcp specific journal allow #21043

[jelly]

Tox is meh :( There is no pcp wheel so need to build it and the PyPi version is outdated 5.0 😞

[jelly]

Looking again at the fetchGroup API, this looks useful but it is so freaking broken, for some reason it returns no instances for the first metric.. (why!)

#!/usr/bin/python

import cpmapi as c_api
from pcp import pmapi

from test.pytest.test_pcp import instances_archive

# archive_path = "/tmp/pytest-of-jelle/pytest-current/disk-archives0"
archive_path = "/var/log/pcp/pmlogger/t14s/20240723.index"
# archive_path = "/home/jelle/projects/cockpit/cockpit-pcp-python/testEnableNoData/archive3/20240719.09.40.index"
pmfg = pmapi.fetchgroup(c_api.PM_CONTEXT_ARCHIVE, archive_path)
context = pmfg.get_context()

# To work out:
# - omit instances
# - scale

kernel_all_load = pmfg.extend_indom("kernel.all.load", c_api.PM_TYPE_FLOAT)
mem_physmem = pmfg.extend_item("mem.physmem")
mem_avail = pmfg.extend_indom("mem.util.available", c_api.PM_TYPE_FLOAT)
swap_pagesout = pmfg.extend_indom("swap.pagesout", c_api.PM_TYPE_FLOAT)
network_interface_total_bytes = pmfg.extend_indom("network.interface.total.bytes")
t = pmfg.extend_timestamp()

for _ in range(5):
    pmfg.fetch()
    print("time: %s" % t())
    for icode, iname, value in kernel_all_load():
        print('kernel.all.load %s %s=%s' % (icode, iname, value()))
    print("mem.physmem: %s" % mem_physmem())
    print(mem_avail())
    for icode, iname, value in mem_avail():
        if iname is not None:
            print('mem.util.available %s %s=%s' % (icode, iname, value()))
    print("total.bytes %s" % network_interface_total_bytes())
    for icode, iname, value in network_interface_total_bytes():
        if iname is not None:
            print('network.interface.total.bytes %s %s=%s' % (icode, iname, value()))

Maybe something for a follow up, it seems nice

[jelle@t14s][~/projects/cockpit/cockpit-pcp-python]%python3 fetchgroup-pcp.py
time: 2024-07-23 16:05:32.873173
mem.physmem: 32620024
[]
total.bytes []
time: 2024-07-23 16:05:32.894293
kernel.all.load 1 1 minute=1.9800000190734863
kernel.all.load 5 5 minute=1.4700000286102295
kernel.all.load 15 15 minute=1.309999942779541
mem.physmem: 32620024
[(4294967295, None, <function fetchgroup.fetchgroup_indom.__call__.<locals>.<lambda>.<locals>.<lambda> at 0x715ea1f02c00>)]
total.bytes [(0, 'lo', <function fetchgroup.fetchgroup_indom.__call__.<locals>.<lambda>.<locals>.<lambda> at 0x715ea1f02ca0>), (1, 'enp0s31f6', <function fetchgroup.fetchgroup_indom.__call__.<locals>.<lambda>.<locals>.<lambda> at 0x715ea1f02e80>), (2, 'wlp0s20f3', <function fetchgroup.fetchgroup_indom.__call__.<locals>.<lambda>.<locals>.<lambda> at 0x715ea1f02f20>), (3, 'enp0s20f0u2u6', <function fetchgroup.fetchgroup_indom.__call__.<locals>.<lambda>.<locals>.<lambda> at 0x715ea1f02fc0>), (14, 'virbr0', <function fetchgroup.fetchgroup_indom.__call__.<locals>.<lambda>.<locals>.<lambda> at 0x715ea1f03060>), (19, 'virbr1', <function fetchgroup.fetchgroup_indom.__call__.<locals>.<lambda>.<locals>.<lambda> at 0x715ea1f03100>)]
Traceback (most recent call last):
  File "/home/jelle/projects/cockpit/cockpit-pcp-python/fetchgroup-pcp.py", line 38, in <module>
    print('network.interface.total.bytes %s %s=%s' % (icode, iname, value()))
                                                                    ^^^^^^^
  File "/usr/lib/python3.12/site-packages/pcp/pmapi.py", line 3102, in <lambda>
    (lambda i: (lambda: decode_one(self, i)))(i)))
                        ^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/site-packages/pcp/pmapi.py", line 3096, in decode_one
    raise pmErr(self.stss[i])
pcp.pmapi.pmErr: PM_ERR_VALUE Missing metric value(s)

[jelly]

@martinpitt @mvollmer an initial review would be appreciated, I am not super fond of the spaghetti soup, combining direct and archived metrics together kinda makes it messy

This is also the reason we can't easily combine the internalmetrics and the PCP metrics, the archive metrics are too different. Plus we have no Sampler() class

[martinpitt]

Thanks @jelly , great work! I have a lot of little things, but honestly I'm not too worried about the code structure -- there is a lot of bit shifting, data type conversion, and plumbing, and I trust that all of this is necessary. But the overall structure is relatively easy to follow.

I'm really looking forward to landing this!

[jelly]

This seems to go green! @martinpitt I re-added the manifest as the JS code uses this to detect if PCP is available and if it's missing all PCP tests simply fail.

A follow up PR will take care of that.

[martinpitt]

@jelly ack wrt. putting back the manifest. I think we agreed to let this land after tomorrow's release, then we have two weeks to do the follow-up cleanups.

Can you please take a look at the remaining open review comments such as this one? They are collapsed by default unfortunately (GitHub could really be more clever with resolved comments..)

[martinpitt]

@jelly FTR, the remaining threads are mostly questions, or little cleanups. They are ok for follow-ups.

[jelly]

@jelly FTR, the remaining threads are mostly questions, or little cleanups. They are ok for follow-ups.

I hopefully handled them all and updated the PR, it wasn't too much work.

[martinpitt]

Thanks! There was one outstanding issue about commenting the "unsupported metric" (string) case, but not important. It's here for archaeology. 😁

I retried some tests which looked like flakes. Thanks!

(Let's please land this after tomorrow's release)

[martinpitt]

🚀

[jelly]

Thanks! There was one outstanding issue about commenting the "unsupported metric" (string) case, but not important. It's here for archaeology. 😁

Added, let's not add new technical debt :)

I retried some tests which looked like flakes. Thanks!

(Let's please land this after tomorrow's release)

\o/