Skip to content

Commit

Permalink
Add coverage report
Browse files Browse the repository at this point in the history
    This commit implements the coverage build support. It can
be triggered by invoking Hefesto as follows:

        _ hefesto --coverage

    The command above would generate all reports under a directory
named as `reports`.

    Anyway, it is also possible of specifies the location where
the reports will be saved, as follows:

        _ hefesto --coverage --genhtml-outpath=/tmp/reports
  • Loading branch information
rafael-santiago committed Jul 11, 2023
1 parent 5f97178 commit d2f1e2b
Show file tree
Hide file tree
Showing 4 changed files with 123 additions and 4 deletions.
30 changes: 26 additions & 4 deletions .github/workflows/forge-specs.yml
Original file line number Diff line number Diff line change
@@ -1,14 +1,17 @@
on: push
name: Build
name: Macgonuts CI
jobs:
Build-Task:
Build:
runs-on: ubuntu-latest
steps:
- name: Install basic tools
shell: bash
run: |
sudo apt-get install git
sudo apt-get install gcc-9
sudo apt-get update
sudo apt-get install perl
sudo apt-get install lcov
- name: Install Hefesto
shell: bash
run: |
Expand All @@ -21,6 +24,15 @@ jobs:
sudo chown -R runner /usr/local/share/hefesto
cd ../..
rm -rf hefesto
- name: Install lcov-generator
shell: bash
run: |
git clone https://github.com/rafael-santiago/helios
cd helios
sudo -E hefesto --install=lcov-generator
sudo chown -R runner /usr/local/share/hefesto
cd ../
rm -rf helios
- name: Clone project repo
uses: actions/checkout@v3
with:
Expand All @@ -29,5 +41,15 @@ jobs:
shell: bash
run: |
cd src
sudo -E hefesto
sudo -E hefesto --coverage
- name: Give permissions for runner to the artifacts
shell: bash
run: |
sudo tar -cvf /home/libmacgonuts-coverage.tar src/reports
sudo chown runner /home/libmacgonuts-coverage.tar
- name: Upload LCOV results
uses: actions/upload-artifact@v2
with:
name: libmacgonuts-coverage-report
path: /home/libmacgonuts-coverage.tar
retention-days: 7
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -2,3 +2,4 @@
**/bin/*
*.Forgefile-*
*.o/
coverage.info
42 changes: 42 additions & 0 deletions doc/BUILD.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@ fresh ``macgonuts`` binary to get your stuff done, you can give ``the low-cost b
- [Installing Hefesto](#installing-hefesto)
- [The low-cost build](#the-low-cost-build)
- [The developer's build](#developers-build)
- [Extracting code coverage](#extracting-code-coverage)
- [Installing the command line tool](#installing-the-command-line-tool)

## Getting newest macgonuts source code revision
Expand Down Expand Up @@ -56,6 +57,17 @@ you@somewhere-over-the-rainbow:~/hefesto/src# logout
(redo login and you done)
```

Now you need to install some conveniences for code coverage extractions, so you need to clone `Helios`
and install `lcov-generator`:

```
you@somewhere-over-the-rainbow:~# git clone https://github.com/rafael-santiago/helios
you@somewhere-over-the-rainbow:~# cd helios
you@somewhere-over-the-rainbow:~/helios# hefesto --install=lcov-generator
you@somewhere-over-the-rainbow:~/helios# cd ..
you@somewhere-over-the-rainbow:~# rm -rf helios
```

You can also run the script ``get-hefesto.sh`` into ``src`` folder of ``Macgonuts``.

[``Back``](#topics)
Expand Down Expand Up @@ -143,6 +155,36 @@ remembering you that your code is actually working and that ``TDD`` matters. :ra

[``Back``](#topics)

### Extracting code coverage

``Macgonuts`` build gives support for code coverage extraction, it support ``gcov`` or ``llvm-cov``. You also need to
have ``lcov`` well-installed more on that [here](https://github.com/linux-test-project/lcov).

By using ``Hefesto`` we can easily extract the code coverage of ``Macgonuts`` by invoking ``Hefesto`` as follows:

```
you@somewhere-over-the-rainbow:~/macgonuts/src# hefesto --coverage
```

By default the report will be generated under ``src/reports`` directory. If you want to specify a directory to generate
the reports you can pass the option ``--genhtml-outpath=<directory path>`` option:

```
you@somewhere-over-the-rainbow:~/macgonuts/src# hefesto --coverage \
> --genhtml-outpath=/mnt/tdd/rocks
```

By design we are only extracting code coverage from ``libmacgnuts`` (the main project under ``src``).
The ``cmd-tool`` is pretty hard for unit testing since it would involve run all attacks that this tool
implements in form of commands (a.k.a tasks) from the github actions' runner. Sincerely, it would be not
easy to do from a rather ``restricted-docker-velotrol-like`` [sic] environment. So, *C'est la vie!*

> - Wait. What does *"velotrol"* is?!
Well, a image will make you understand my point much better, [look](https://duckduckgo.com/?q=velotrol&t=h_&iax=images&ia=images)! :rofl:

[``Back``](#topics)

## Installing the command line tool

Having ``Hefesto`` well installed all you need is move to ``src`` toplevel subdirectory and run the following:
Expand Down
54 changes: 54 additions & 0 deletions src/build/toolsets.hsl
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@ include ~/toolsets/gcc/gcc-lib.hsl
include ~/toolsets/clang/clang-app.hsl
include ~/toolsets/clang/clang-lib.hsl
include ~/toolsets/common/utils/lang/c/dependency_scanner.hsl
include ~/toolsets/common/utils/lang/c/lcov.hsl
include ~/fsutil.hsl

function installer() : result type none {
Expand Down Expand Up @@ -51,6 +52,20 @@ function runtests(binary type string, args type string) : result type none {
if (hefesto.sys.run($binary + " " + $args) != 0) {
hefesto.project.abort(1);
}
var option type list;
$option = hefesto.sys.get_option("coverage");
if ($option.count() > 0) {
var obj_output_dir type string;
$option = hefesto.sys.get_option("obj-output-dir");
if ($option.count() > 0) {
$obj_output_dir = $option.item(0);
} else {
$obj_output_dir = hefesto.sys.pwd();
}
if (generate_lcov_report($obj_output_dir) != 0) {
hefesto.project.abort(1);
}
}
}

function set_rootdir(change_to type string) : result type none {
Expand Down Expand Up @@ -189,6 +204,45 @@ local function build_submodule(subdir type string) : result type int {
$build_options.replace("--libraries=.* ", "");
$build_options.replace("--ldflags=.* ", "");

var coverage type list;
$coverage = hefesto.sys.get_option("coverage");

var projects2cov type list;
# INFO(Rafael): Add to this list all projects relevant to extract coverage info.
#
# By now we are only extracting coverage info from libmacgonuts.
# The cmd-tool is rather difficult for unit testing since it depends
# on promoting all attacks that its tasks implements. Automating the
# execution of all them into a docker-velotrol-based-environment [sic]
# would be a quixotic task. Even so, if you want to, good luck!
$projects2cov.add_item("macgonuts-static-lib");

if ($coverage.count() > 0
&& $subdir == "test") {
var subproject type string;
$subproject = $subdir;
$subproject.replace("/", "-");
$coverage = hefesto.sys.get_option("genhtml-outpath");
var genhtml_outpath type string;
if ($coverage.count() == 0) {
$genhtml_outpath = " --genhtml-outpath=" + hefesto.sys.make_path(get_rootdir(),
hefesto.sys.make_path("/reports/", hefesto.project.name()));
} else {
$genhtml_outpath = $coverage.item(0);
$genhtml_outpath = " --genhtml-outpath=" + hefesto.sys.make_path($genhtml_outpath,
hefesto.project.name());
$build_options.replace("--genhtml-outpath=.* ", "");
}
if ($projects2cov.index_of(hefesto.project.name()) == -1) {
$genhtml_outpath = "";
}
$build_options = $build_options +
" --gcda-search-path=.o," + hefesto.sys.make_path($oldcwd, ".o") + " " +
" --lcov-remove-patterns=*_tests.c,*src/test* " +
" --genhtml-rendering-options=--legend" +
$genhtml_outpath;
}

var exit_code type int;
$exit_code = hefesto.sys.run("hefesto " + $build_options);

Expand Down

0 comments on commit d2f1e2b

Please sign in to comment.