Running several variations of the same test file

[dertarr]

Hello folks,

I have a module_under_test.c and its module_under_test.h that takes a macro parameter passed from a Makefile. Say, the macro has a name ENABLE_SUBSET and it would turn on or off specific code inside my module with an #ifdef directive.
I created a test_module_under_test.c with several test cases, and I would like to run the same test file twice: each time with a different value in ENABLE_SUBSET macro as a part of the test:all target. How would I do that?

Ideally, I could have a short test_module_under_test.yaml that would override a :define: section from the project.yaml for the specified test, and this configuration file could also have a section that instructs to run the test several times, each with a different config. Or, I could create a soft link test_module_under_test_2.c and set the required parameters to each file. As for now, both are likely not an option yet.

[mvandervoord]

If you're using one of the pre-releases for 1.0.0, the best method is to do the following:

Create two test files... a test_module_under_test.c and a test_module_under_test_2.c (or whatever you want to call them). The tests can be as different or similar as is needed for the different define settings. For either or both, you can add an entry to the defines list in your project.yml file which are file specific:

:defines:
  :test:
    '*': 
      - TEST
      - OTHER_STUFF
    test_module_under_test_2.c:
      - ENABLE_SUBSET

[dertarr]

Well, technically, it works. Although, it would be much handier to have a separate test_module_under_test.yaml file in the same folder with the test to override the required settings for tests individually. I believe it is better to have the specific setting as close to the target test file as possible so they are clearly visible to whoever has to debug a failing test. And another point is that project.yaml would grow immensely obese for large projects if every special option must go there.

PS. I am not sure if I should click "Mark as answer" to the previous post. It does resolve my issue for now, but maybe the devs would consider the enhancement I mentioned?

[mvandervoord]

The method you've mentioned, of having a specific yaml file of changes named after its corresponding C file, is the method we started with, actually. This method had its own collection of misunderstandings. Most of all, people assumed you could override any settings per module in this way. They were surprised when they couldn't, for example, change all their compiler settings per file or change CMock's features per file.

There were also complaints that it was hard for devs to know when a file has had special handling requested. Whereas now there is a central place where you can find all one-off modifications, in the past they would have to search through all their project directories to find them.

Having worked with both methods and seeing their pros and cons, I don't believe we're likely to switch back to the old method.

[dertarr]

Ok, I see. So, that is the answer. Thanks for your help!
BTW, is it possible to set a specific compile/link options in the same manner for a file? For example, when I want certain UT code to compile and run as 32-bits and other files as 64 bits?
Also, you mentioned pre-release versions. Would that approach change with the 1.0.0 release version?

[mvandervoord]

:) Yes, you can do the same thing with the :flags section.