Add coverage report

    This commit implements the coverage build support. It can
be triggered by invoking Hefesto as follows:

        _ hefesto --coverage

    The command above would generate all reports under a directory
named as `reports`.

    Anyway, it is also possible of specifies the location where
the reports will be saved, as follows:

        _ hefesto --coverage --genhtml-outpath=/tmp/reports

    Addionally it parses the coverage measures (functions & lines)
and it displays this info at the end of a coverage build. Also
breaking the build if at least one of the two is lower than 75%.

.github/workflows/forge-specs.yml

@@ -1,14 +1,17 @@
 on: push
-name: Build
+name: Macgonuts CI
 jobs:
- Build-Task:
+ Build:
  runs-on: ubuntu-latest
  steps:
  - name: Install basic tools
  shell: bash
  run: |
  sudo apt-get install git
  sudo apt-get install gcc-9
+ sudo apt-get update
+ sudo apt-get install perl
+ sudo apt-get install lcov
  - name: Install Hefesto
  shell: bash
  run: |
@@ -21,6 +24,15 @@ jobs:
  sudo chown -R runner /usr/local/share/hefesto
  cd ../..
  rm -rf hefesto
+ - name: Install lcov-generator
+ shell: bash
+ run: |
+ git clone https://github.com/rafael-santiago/helios
+ cd helios
+ sudo -E hefesto --install=lcov-generator
+ sudo chown -R runner /usr/local/share/hefesto
+ cd ../
+ rm -rf helios
  - name: Clone project repo
  uses: actions/checkout@v3
  with:
@@ -29,5 +41,15 @@ jobs:
  shell: bash
  run: |
  cd src
- sudo -E hefesto
-
+ sudo -E hefesto --coverage
+ - name: Tar coverage report
+ shell: bash
+ run: |
+ sudo tar -cvf /home/libmacgonuts-coverage.tar src/reports/macgonuts-static-lib
+ sudo chown runner /home/libmacgonuts-coverage.tar
+ - name: Upload LCOV results
+ uses: actions/upload-artifact@v2
+ with:
+ name: libmacgonuts-coverage-report
+ path: /home/libmacgonuts-coverage.tar
+ retention-days: 7

.gitignore

@@ -2,3 +2,4 @@
 **/bin/*
 *.Forgefile-*
 *.o/
+coverage.info

README.md

@@ -1,4 +1,4 @@
-# Macgonuts ![ci-status](https://github.com/rafael-santiago/macgonuts/actions/workflows/forge-specs.yml/badge.svg)
+# Macgonuts ![ci-status](https://github.com/rafael-santiago/macgonuts/actions/workflows/forge-specs.yml/badge.svg) ![linux-function-coverage](https://img.shields.io/badge/function_coverage-0%25-red?logo=linux&logoColor=white) ![linux-line-coverage](https://img.shields.io/badge/line_coverage-0%25-red?logo=linux&logoColor=white) ![freebsd-function-coverage](https://img.shields.io/badge/function_coverage-0%25-red?logo=freebsd&logoColor=white) ![freebsd-line-coverage](https://img.shields.io/badge/function_coverage-0%25-red?logo=freebsd&logoColor=white)
 
 ``Macgonuts`` is an ``ARP/NDP`` swiss army knife to make ``MAC addresses`` going nuts on networks around!
 

doc/BUILD.md

@@ -12,6 +12,7 @@ fresh ``macgonuts`` binary to get your stuff done, you can give ``the low-cost b
  - [Installing Hefesto](#installing-hefesto)
  - [The low-cost build](#the-low-cost-build)
  - [The developer's build](#developers-build)
+ - [Extracting code coverage](#extracting-code-coverage)
  - [Installing the command line tool](#installing-the-command-line-tool)
 
 ## Getting newest macgonuts source code revision
@@ -56,6 +57,17 @@ you@somewhere-over-the-rainbow:~/hefesto/src# logout
 (redo login and you done)
 ```
 
+Now you need to install some conveniences for code coverage extractions, so you need to clone `Helios`
+and install `lcov-generator`:
+
+```
+you@somewhere-over-the-rainbow:~# git clone https://github.com/rafael-santiago/helios
+you@somewhere-over-the-rainbow:~# cd helios
+you@somewhere-over-the-rainbow:~/helios# hefesto --install=lcov-generator
+you@somewhere-over-the-rainbow:~/helios# cd ..
+you@somewhere-over-the-rainbow:~# rm -rf helios
+```
+
 You can also run the script ``get-hefesto.sh`` into ``src`` folder of ``Macgonuts``.
 
 [``Back``](#topics)
@@ -143,6 +155,36 @@ remembering you that your code is actually working and that ``TDD`` matters. :ra
 
 [``Back``](#topics)
 
+### Extracting code coverage
+
+``Macgonuts`` build gives support for code coverage extraction, it support ``gcov`` or ``llvm-cov``. You also need to
+have ``lcov`` well-installed more on that [here](https://github.com/linux-test-project/lcov).
+
+By using ``Hefesto`` we can easily extract the code coverage of ``Macgonuts`` by invoking ``Hefesto`` as follows:
+
+```
+you@somewhere-over-the-rainbow:~/macgonuts/src# hefesto --coverage
+```
+
+By default the report will be generated under ``src/reports`` directory. If you want to specify a directory to generate
+the reports you can pass the option ``--genhtml-outpath=<directory path>`` option:
+
+```
+you@somewhere-over-the-rainbow:~/macgonuts/src# hefesto --coverage \
+> --genhtml-outpath=/mnt/tdd/rocks
+```
+
+By design we are only extracting code coverage from ``libmacgnuts`` (the main project under ``src``).
+The ``cmd-tool`` is pretty hard for unit testing since it would involve run all attacks that this tool
+implements in form of commands (a.k.a tasks) from the github actions' runner. Sincerely, it would be not
+easy to do from a rather ``restricted-docker-velotrol-like`` [sic] environment. So, *C'est la vie!*
+
+> - Wait. What does *"velotrol"* is?!
+
+Well, a image will make you understand my point much better, [look](https://duckduckgo.com/?q=velotrol&t=h_&iax=images&ia=images)! :rofl:
+
+[``Back``](#topics)
+
 ## Installing the command line tool
 
 Having ``Hefesto`` well installed all you need is move to ``src`` toplevel subdirectory and run the following:

src/Forgefile.hsl

@@ -71,6 +71,18 @@ macgonuts-static-lib.epilogue() {
  $subprojects.add_item("test");
  $subprojects.add_item("cmd");
  if (build_projects($subprojects) == 0) {
+ var option type list;
+ $option = hefesto.sys.get_option("coverage");
+ if ($option.count() > 0) {
+ var report_path type string;
+ $report_path = hefesto.sys.make_path(get_coverage_report_dir(), "index.html");
+ var fcov type string;
+ $fcov = get_function_coverage($report_path);
+ var lcov type string;
+ $lcov = get_line_coverage($report_path);
+ hefesto.sys.echo("INFO: Code coverage : [ functions = " + $fcov + " % / lines = " + $lcov + " % ]\n");
+ do_break_when_low_coverage($fcov, $lcov);
+ }
  hefesto.sys.echo("INFO: Done.\n");
  } else {
  hefesto.project.abort(1);

src/build/toolsets.hsl

@@ -9,7 +9,9 @@ include ~/toolsets/gcc/gcc-lib.hsl
 include ~/toolsets/clang/clang-app.hsl
 include ~/toolsets/clang/clang-lib.hsl
 include ~/toolsets/common/utils/lang/c/dependency_scanner.hsl
+include ~/toolsets/common/utils/lang/c/lcov.hsl
 include ~/fsutil.hsl
+include ~/conv.hsl
 
 function installer() : result type none {
  var option type list;
@@ -51,6 +53,20 @@ function runtests(binary type string, args type string) : result type none {
  if (hefesto.sys.run($binary + " " + $args) != 0) {
  hefesto.project.abort(1);
  }
+ var option type list;
+ $option = hefesto.sys.get_option("coverage");
+ if ($option.count() > 0) {
+ var obj_output_dir type string;
+ $option = hefesto.sys.get_option("obj-output-dir");
+ if ($option.count() > 0) {
+ $obj_output_dir = $option.item(0);
+ } else {
+ $obj_output_dir = hefesto.sys.pwd();
+ }
+ if (generate_lcov_report($obj_output_dir) != 0) {
+ hefesto.project.abort(1);
+ }
+ }
 }
 
 function set_rootdir(change_to type string) : result type none {
@@ -171,6 +187,22 @@ local function build_accacia() : result type int {
  result $exit_code;
 }
 
+function get_coverage_report_dir() : result type string {
+ var report_path type string;
+ var option type list;
+ $option = hefesto.sys.get_option("genhtml-outpath");
+ var genhtml_outpath type string;
+ if ($option.count() == 0) {
+ $report_path = hefesto.sys.make_path(get_rootdir(),
+ hefesto.sys.make_path("/reports/", hefesto.project.name()));
+ } else {
+ $report_path = $option.item(0);
+ $report_path = hefesto.sys.make_path($report_path,
+ hefesto.project.name());
+ }
+ result $report_path;
+}
+
 local function build_submodule(subdir type string) : result type int {
  var oldcwd type string;
  $oldcwd = hefesto.sys.pwd();
@@ -189,6 +221,43 @@ local function build_submodule(subdir type string) : result type int {
  $build_options.replace("--libraries=.* ", "");
  $build_options.replace("--ldflags=.* ", "");
 
+ var coverage type list;
+ $coverage = hefesto.sys.get_option("coverage");
+ var report_path type string;
+
+ var projects2cov type list;
+ # INFO(Rafael): Add to this list all projects relevant to extract coverage info.
+ #
+ # By now we are only extracting coverage info from libmacgonuts.
+ # The cmd-tool is rather difficult for unit testing since it depends
+ # on promoting all attacks that its tasks implements. Automating the
+ # execution of all them into a docker-velotrol-based-environment [sic]
+ # would be a quixotic task. Even so, if you want to, good luck!
+ $projects2cov.add_item("macgonuts-static-lib");
+
+ if ($coverage.count() > 0
+ && $subdir == "test") {
+ var subproject type string;
+ $subproject = $subdir;
+ $subproject.replace("/", "-");
+ $coverage = hefesto.sys.get_option("genhtml-outpath");
+ var genhtml_outpath type string;
+ $report_path = get_coverage_report_dir();
+ if ($coverage.count() > 0) {
+ $report_path = $coverage.item(0);
+ $build_options.replace("--genhtml-outpath=.* ", "");
+ }
+ if ($projects2cov.index_of(hefesto.project.name()) > -1) {
+ $genhtml_outpath = " --genhtml-outpath=" + $report_path;
+ }
+ $build_options = $build_options +
+ " --gcda-search-path=.o," + hefesto.sys.make_path($oldcwd, ".o") + " " +
+ " --lcov-remove-patterns=*_tests.c,*src/test* " +
+ " --genhtml-rendering-options=--legend" +
+ $genhtml_outpath;
+
+ }
+
  var exit_code type int;
  $exit_code = hefesto.sys.run("hefesto " + $build_options);
 
@@ -197,6 +266,89 @@ local function build_submodule(subdir type string) : result type int {
  result $exit_code;
 }
 
+function do_break_when_low_coverage(function_coverage type string,
+ line_coverage type string) : result type none {
+ var low_nr type int;
+ $low_nr = is_low_cov($function_coverage);
+ if ($low_nr > 0) {
+ hefesto.sys.echo("ERROR: Low function coverage detected (it must be >= 75) : " +
+ $function_coverage + "%\n");
+ }
+ $low_nr = $low_nr + is_low_cov($line_coverage);
+ if ($low_nr > 0) {
+ hefesto.sys.echo("ERROR: Low line coverage detected (it must be >= 75) : " +
+ $line_coverage + "%\n");
+ }
+ if ($low_nr > 0) {
+ hefesto.project.abort(1);
+ }
+}
+
+local function is_low_cov(coverage type string) : result type int {
+ var dec type string;
+ $dec = $coverage;
+ $dec.replace("\\..*$", "");
+ result (str2int($dec) < 75);
+}
+
+function get_function_coverage(report_path type string) : result type string {
+ result get_coverage_result($report_path, "<td.*Functions:</td>");
+}
+
+function get_line_coverage(report_path type string) : result type string {
+ result get_coverage_result($report_path, "<td.*Lines:</td>");
+}
+
+function get_function_coverage_badge(value type string) : result type string {
+ result get_coverage_badge($value, "function coverage");
+}
+
+function get_line_coverage_badge(value type string) : result type string {
+ result get_coverage_badge($value, "line coverage");
+}
+
+local function get_coverage_badge(value type string, prefix type string) : result type string {
+ var uri type string;
+ $prefix.replace(" ", "_");
+ $uri = "https://img.shields.io/badge/" + $prefix + "-" + $value + "%25";
+ var dec type string;
+ $dec = $value;
+ $dec.replace("\\.*$", "");
+ var d type int;
+ $d = str2int($dec);
+ if ($d >= 90) {
+ $uri = $uri + "-lime";
+ } else if ($d >= 75) {
+ $uri = $uri + "-yellow";
+ } else {
+ $uri = $uri + "-red";
+ }
+ $uri = $uri + "?logo=" + hefesto.sys.os_name() + "&logoColor=white&style=for-the-badge";
+ result $uri;
+}
+
+local function get_coverage_result(report_path type string, pattern type string) : result type string {
+ var report_data type list;
+ $report_data = hefesto.sys.lines_from_file($report_path, ".*");
+ if ($report_data.count() == 0) {
+ result "NaN";
+ }
+ var r type int;
+ $r = 0;
+ while ($r < $report_data.count()) {
+ var report_line type string;
+ $report_line = $report_data.item($r);
+ if ($report_line.match($pattern) == 1 && ($r + 3) < $report_data.count()) {
+ $report_line = $report_data.item($r + 3);
+ $report_line.replace(".*\">", "");
+ $report_line.replace(" %</td>$", "");
+ result $report_line;
+ }
+ $r = $r + 1;
+ }
+ result "NaN";
+}
+
 local function build_cutest() : result type int {
  var oldcwd type string;
  $oldcwd = hefesto.sys.pwd();