Integration with C++ coverage analysis

[fkromer]

How can one integrate coverage analysis (e.g. gcov) for C++ nodes tested with pyrostest?

[joshuamorton]

Hey, so my intent with pyrostest is to be useful for larger integration style testing, where coverage is less useful. It looks like gcov uses the same output format as python tools like coverage.py, so someone could probably integrate them somehow, but its not currently on my radar.

[fkromer]

Hey, so my intent with pyrostest is to be useful for larger integration style testing, where coverage is less useful.

Your project makes node unit/integration testing really pythonic and looks promising. Are you planning to add support for action mocks, service mocks, check action and check service similar to the topic mock and check functionality? The fact that "check_topic call can only recieve a single message" is limiting but you are planning to address this as well, I guess.

A short explanation about the reason for my question: I search some workaround for rostest's limitation of not beeing able to support grey-box node unit testsing. (An explanation what I understand under grey-box node unit testing.) As a consequence you won't be able to achieve full coverage across the node to the library code level with rostest.

It looks like gcov uses the same output format as python tools like coverage.py, so someone could probably integrate them somehow, but its not currently on my radar.

Sure, that would be one later topic on my todo list as well.

[joshuamorton]

The fact that "check_topic call can only recieve a single message" is limiting but you are planning to address this as well, I guess.

Indeed, if I can figure it out, its proving to be harder to fix than I'd hoped.

Your project makes node unit/integration testing really pythonic and looks promising. Are you planning to add support for action mocks, service mocks, check action and check service similar to the topic mock and check functionality?

Its not on my radar, unfortunately. I think doing this correctly would require a major refactor of some core bits of the library, which perhaps should happen, but aren't things that I can really focus on (this project is one that I work on in my spare time, and mostly for use on buzzmobile).

A short explanation about the reason for my question: I search some workaround for rostest's limitation of not beeing able to support grey-box node unit testsing.

Ah, yeah I don't really think this library helps with that problem. Its really just a nice interface for making fakes for black box tests. Ros's multi-process architecture makes gray-box testing really, really annoying and IMO not worth it if your nodes are already black-box tested and unit/white-box tested internally.

[fkromer]

Indeed, if I can figure it out, its proving to be harder to fix than I'd hoped.
...
Its not on my radar, unfortunately.

Ah, that's sad. (Without providing all basic functionalities pyrostest is no real choice for me.)

... I think doing this correctly would require a major refactor of some core bits of the library, which perhaps should happen, but aren't things that I can really focus on (this project is one that I work on in my spare time, and mostly for use on buzzmobile).

I understand that. Unfortunately focus on private projects (how valuable the outcome would be) is limited.

Ah, yeah I don't really think this library helps with that problem. Its really just a nice interface for making fakes for black box tests. Ros's multi-process architecture makes gray-box testing really, really annoying and IMO not worth it if your nodes are already black-box tested and unit/white-box tested internally.

I continue to search for a solution for this, so true, really annoying problem as soon as it gets relevant and valuable enough.