FreeBSD requests

Page for discussion of integration into FreeBSD.

Time series data

xo_emit() to have a custom time series output

Suggest some markup in xo_emit() for outputting a time series output. This would be useful in the case of "netstat 1", "iostat 1", and other tools that will emit periodic data. The goal here is to make it easy to graph data points for output. Suggest allowing user programs to override both the name of the key as well as the output format.

Why not just use a filter?

This could also be done via an output filter, however this would require using stdbuf(1) and might be complex for end users to implement as opposed to having this baked in.

Timeseries simple mode.

In addition it may be useful to have a simplified version. For example the output of netstat(1) when told to give time series data looks as follows:

/usr/src/contrib/libxo/libxo % netstat --libxo json,pretty 1
{
  "statistics": {
    "interface-statistics": [
      {
        "received-packets": 2,
        "received-errors": 0,
        "dropped-packets": 0,
        "received-bytes": 132,
        "sent-packets": 4,
        "send-errors": 0,
        "sent-bytes": 1080,
        "collisions": 0
      },
      {
        "received-packets": 2,
        "received-errors": 0,
        "dropped-packets": 0,
        "received-bytes": 132,
        "sent-packets": 6,
        "send-errors": 0, 

A more abbreviated and compact form might be worthwhile, example (note the ommission of "interface-statistics" and moving stats to a single line):

/usr/src/contrib/libxo/libxo % netstat --libxo json,ld 1
{
  "statistics": [
 { "received-packets": 2, "received-errors": 0, "dropped-packets": 0, "received-bytes": 132, "sent-packets": 4, "send-errors": 0, "sent-bytes": 1080, "collisions": 0 },
 { "received-packets": 2, "received-errors": 0, "dropped-packets": 0, "received-bytes": 132, "sent-packets": 6, "send-errors": 0, "sent-bytes": 1964, "collisions": 0 },
 { "received-packets": 1, "received-errors": 0, "dropped-packets": 0, "received-bytes": 66, "sent-packets": 4, "send-errors": 0, "sent-bytes": 1048, "collisions": 0 } 

API cleanup for json

Stefan Esser se@freebsd.org has this to say:

I have converted pciconf to use libxo (not yet committed, but available
for review on Phybricator as D1206).

During that work, I noticed that the libxo API is not optimal from a
user's PoV (i.e. the programmer, who work on the conversion of some
existing program to use libxo). The matching of opening and closing
tags (and list tags around instances) needs explicit state in the
program, although this state is maintained within libxo, just not
readily accessible to the programmer.


A simpler API could be to just indicate the start of a new container
or instance and to automatically generate the closing tags. This is
possible, if containers and instances do not automatically push state.


I have already sent a description of a proposed alternative API to
Phil Shafer (and the hackers list) some 2 weeks ago:

https://groups.google.com/d/topic/mailing.freebsd.hackers/zhl7ojBBYJ4


Today I sent another mail to Phil and Simon, to further explain the
proposal - I'll append that mail at the end of this message. Both
mails contain a description of the proposed API and an example of
its use (pciconf fragment and short synthetic example).


What's your opinion about this API? Could it help get syntactically
correct JSON output out of netstat with less effort?

Simplify list handling

When looking for ways to simplify conversion of programs to libxo I noticed, that much effort is required to close optional lists (which are omitted, if there are no list elements to display). Examples are the conversion of netstat or pciconf, which need conditionals and state handling to correctly emit xo_close_list() at the correct place.

Due to rules for well-formed JSON and XML output, xo_open/close_list() can generally be implied around the xo_open/close_instance() calls that generate the list elements. This API change is very simple (just insert a few tests and conditionally call xo_openlist() or xo_close_list() in xo_do_emit(), xo_open_instance() and xo_close_*() (and possibly xo_finish() too?) ...).