Header files in install directories are filtered out of the results

[crdelsey]

If I have two packages in my workspace, one defining a class with implementation details in a header file and the other including that header file

my_ws
 - src
   - package_a
     - package_a.h
  - package_b
     - package_b.cpp (includes package_a/package_a.h)

Package_b picks up the package_a.h header from the workspace install directory. The .gcda file associated with package_b.cpp will refer to my_ws/install/package_a/package_a.h

However, any coverage data for files listed in the install directories is getting filtered out of the lcov-results. Only files in build and src have their data reported.

As a result, if package_a.h is only executed by cpp files in other packages, no coverage data is reported.

[jpsamper2009]

@crdelsey Are you building with --symlink-install? In theory, lcov is configured to follow symlinks (https://github.com/colcon/colcon-lcov-result/blob/master/colcon_lcov_result/verb/configuration/lcovrc#L31), which means it should find the headers in src in those cases. I'll have to look into exactly how lcov is generating gcno and gcda files for the headers with implementation, though.

Do you have a simple reproducer? If not, I'll try to create one. Thanks!

[crdelsey]

Yes. I'm building with --symlink-install

I've created a minimal example project here:
https://github.com/crdelsey/lcovissue

You'll see there is one extra line in the base lcov build. Basically, the package_a.h file shows up twice. It will show up under install for the lines run from package B tests, and it will show up under src for the lines run from it's own package A test.

[jpsamper2009]

@crdelsey I think I understand now. You expect that since package_b is calling a function in package_a, the coverage should reflect that package_a was touched during package_b's tests, correct? More concretely, you expect:

Coverage from package_a tests

and coverage from package_b tests

to appear as one file, correct?

---

• This behavior brings up a philosophical question of whether package_a should rely on the tests of package_b for package_a's coverage?
  • The ROS2 federated package model looks at each package as its own separate entity, so under this model, package_a is external from package_b.
  • package_a.h could just as well be stdio.h as far as package_b is concerned, so does it really make sense to generate coverage for stdio.h as part of package_b's tests?
  • This issue is not wrong, however, sincelcov-result does filter out the results of header files in the install directory when it configures lcov to not generate coverage for external packages (https://github.com/colcon/colcon-lcov-result/blob/master/colcon_lcov_result/verb/configuration/lcovrc#L17).
  • That same configuration also filters files in /usr/include/c++/
  • So again, I think it's a philosophical question whether you consider this configuration a bug or a feature 😄
  • The good news is that you can specify a custom lcovrc file with the --lcov-config-file option
  • The bad news is that I don't think lcov is able to recognize that the symlink in the install folder resolves to the package_a.h in src (which I thought it could) - unless gcc somehow can...

[crdelsey]

This behavior brings up a philosophical question of whether package_a should rely on the tests of package_b for package_a's coverage?

In a perfect world, having full coverage within a single package would be great. But even in a perfect world, running system tests is desirable, and it would be nice to know how much of the code is covered by the system tests as well.

Perhaps it would be appropriate to have two modes of operation, a per package report like you currently produce, and a per workspace report, like I am looking for.

The good news is that you can specify a custom lcovrc file with the --lcov-config-file option

I think I see a way to turn off the --no-external flag in the lcov-config-file so I can include the install folder. Unfortunately, that would also include /usr/include/c++ and other system headers, which would be undesirable. Is there a rc file setting I'm missing that would allow me to get the report I'm looking for?

The bad news is that I don't think lcov is able to recognize that the symlink in the install folder resolves to the package_a.h in src (which I thought it could) - unless gcc somehow can...

Yeah. I guess that is an issue for the lcov tool. Maybe if I could provide it a list of aliases...