✨ Preprocessing has more configuration options

Boolean true / false is superseded by `:none`, `:all`, `:tests`, and `:mocks`.

assets/project_as_gem.yml

@@ -13,7 +13,7 @@
 
  # optional features. If you don't need them, keep them turned off for performance
  :use_mocks: TRUE
- :use_test_preprocessor: TRUE
+ :use_test_preprocessor: :all
  :use_backtrace: :none
  :use_decorators: :auto #Decorate Ceedling's output text. Your options are :auto, :all, or :none
 

assets/project_with_guts.yml

@@ -13,7 +13,7 @@
 
  # optional features. If you don't need them, keep them turned off for performance
  :use_mocks: TRUE
- :use_test_preprocessor: TRUE
+ :use_test_preprocessor: :all
  :use_backtrace: :none
  :use_decorators: :auto #Decorate Ceedling's output text. Your options are :auto, :all, or :none
 

docs/BreakingChanges.md

@@ -7,7 +7,7 @@ These breaking changes are complemented by two other documents:
 
 ---
 
-# [1.0.0 pre-release] — 2024-06-19
+# [1.0.0 pre-release] — 2024-07-22
 
 ## Explicit `:paths` ↳ `:include` entries in the project file
 
@@ -45,6 +45,23 @@ In brief:
 
 Flags specified for release builds are applied to all files in the release build.
 
+## New `:project` ↳ `:use_test_preprocessor` configuration settings
+
+Ceedling’s preprocessing features have been greatly improved. Preprocessing is now no longer all-or-nothing with a simple boolean value.
+
+In place of `true` or `false`, `:use_test_preprocessing` now accepts:
+
+* `:none` disables preprocessing (equivalent to previous `false` setting).
+* `:all` enables preprpocessing for all mockable header files and test C files (equivalent to previous `true` setting).
+* `:mocks` enables only preprocessing of header files that are to be mocked.
+* `:tests` enables only preprocessing of your test files.
+
+## Preprocessing is temporarily unable to handle Unity’s parameterized test case macros `TEST_CASE()` and `TEST_RANGE()`
+
+Ceedling’s preprocessing abilities have been nearly entirely rewritten. In the process of doing so Ceedling has temporarily lost the ability to preprocess a test file but preserve certain directive macros including Unity’s parameterized test case macros.
+
+Note that it is now possible to enable preprocessing for mockable header files apart from test files.
+
 ## `TEST_FILE()` ➡️ `TEST_SOURCE_FILE()`
 
 The previously undocumented `TEST_FILE()` build directive macro (#796) available within test files has been renamed and is now officially documented. See earlier section on this.

docs/CeedlingPacket.md

@@ -1384,28 +1384,31 @@ In other words, a test function signature should look like this:
 Ceedling and CMock are advanced tools that both perform fairly sophisticated
 parsing.
 
-However, neither of these tools fully understand the entire C language,
+However, neither of these tools fully understands the entire C language,
 especially C's preprocessing statements.
 
-If your test files rely on macros and `#ifdef` conditionals, there's a good
+If your test files rely on macros and `#ifdef` conditionals, there’s a good
 chance that Ceedling will break on trying to process your test files, or,
-alternatively, your test suite will not execute as expected.
+alternatively, your test suite will build but not execute as expected.
 
 Similarly, generating mocks of header files with macros and `#ifdef`
-conditionals can get weird.
+conditionals can get weird. It’s often in sophisticated projects with complex 
+header files that mocking is most desired in the first place.
 
 Ceedling includes an optional ability to preprocess test files and header files
 before executing any operations on them. See the `:project` ↳
-`:use_test_preprocessor`). That is, Ceedling will expand preprocessor
-statements in test files before generating test runners from them and will
-expand preprocessor statements in header files before generating mocks from
-them.
+`:use_test_preprocessor` project configuration setting.
 
-This ability uses `gcc`'s preprocessing mode and the `cpp` preprocessor tool to
+When preprocessing is enabled for test files, Ceedling will expand preprocessor 
+statements in test files before generating test runners from them. When 
+preprocessing is enabled for mocking, Ceedling will expand preprocessor 
+statements in header files before generating mocks from them.
+
+This ability uses `gcc`’s preprocessing mode and the `cpp` preprocessor tool to
 strip down / expand test files and headers to their applicable content which
-can then be processed by Ceedling and CMock. They must be in your search path
-if Ceedling’s preprocessing is enabled. Further, Ceedling’s features are
-directly tied to these tools' abilities and options. They should not be
+can then be processed by Ceedling and CMock. These tools must be in your search 
+path if Ceedling’s preprocessing is enabled. Further, Ceedling’s features are
+directly tied to these tools' abilities and options. These tools should not be
 redefined for other toolchains.
 
 ### Execution time (duration) reporting in Ceedling operations & test suites
@@ -1958,7 +1961,7 @@ the examples).
 
 ```yaml
 :project:
- :use_test_preprocessor: TRUE
+ :use_test_preprocessor: :all
  :test_file_prefix: Test
 ```
 
@@ -1983,7 +1986,7 @@ rules (noted after the examples).
 
 ```yaml
 :project:
- :use_test_preprocessor: FALSE
+ :use_test_preprocessor: :none
 
 :plugins:
  :enabled:
@@ -1997,7 +2000,7 @@ Behold the project configuration following mixin merges:
 ```yaml
 :project:
  :build_root: build/ # From base.yml
- :use_test_preprocessor: TRUE # Value in support/mixins/cmdline.yml overwrote value from support/mixins/enabled.yml
+ :use_test_preprocessor: :all # Value in support/mixins/cmdline.yml overwrote value from support/mixins/enabled.yml
  :test_file_prefix: Test # Added to :project from support/mixins/cmdline.yml
 
 :plugins:
@@ -2388,18 +2391,22 @@ migrated to the `:test_build` and `:release_build` sections.
 * `:use_test_preprocessor`
 
  This option allows Ceedling to work with test files that contain
- conditional compilation statements (e.g. `#ifdef`) and header files you
- wish to mock that contain conditional preprocessor statements and/or
- macros.
+ conditional compilation statements (e.g. `#ifdef`) as well as mockable header 
+ files containing conditional preprocessor directives and/or macros.
 
  See the [documentation on test preprocessing][test-preprocessing] for more.
 
- With this option enabled, the `gcc` & `cpp` tools must exist in an
+ With any preprocessing enabled, the `gcc` & `cpp` tools must exist in an
  accessible system search path.
 
+ * `:none` disables preprocessing.
+ * `:all` enables preprpocessing for all mockable header files and test C files.
+ * `:mocks` enables only preprocessing of header files that are to be mocked.
+ * `:tests` enables only preprocessing of your test files.
+
  [test-preprocessing]: #preprocessing-behavior-for-tests
 
- **Default**: FALSE
+ **Default**: `:none`
 
 * `:test_file_prefix`
 
@@ -2616,7 +2623,7 @@ migrated to the `:test_build` and `:release_build` sections.
 :project:
  :build_root: project_awesome/build
  :use_exceptions: FALSE
- :use_test_preprocessor: TRUE
+ :use_test_preprocessor: :all
  :options_paths:
  - project/options
  - external/shared/options
@@ -3916,15 +3923,13 @@ Please see the discussion in `:defines` for a complete example.
 
 ## `:cmock` Configure CMock’s code generation & compilation
 
-Ceedling sets values for a subset of CMock settings. All CMock
-options are available to be set, but only those options set by
-Ceedling in an automated fashion are documented below. See CMock
-documentation.
+Ceedling sets values for a subset of CMock settings. All CMock options are
+available to be set, but only those options set by Ceedling in an automated
+fashion are documented below. See CMock documentation.
 
-Ceedling sets values for a subset of CMock settings. All CMock 
-options are available to be set, but only those options set by 
-Ceedling in an automated fashion are documented below. 
-See [CMock] documentation.
+Ceedling sets values for a subset of CMock settings. All CMock options are
+available to be set, but only those options set by Ceedling in an automated
+fashion are documented below. See [CMock] documentation.
 
 * `:enforce_strict_ordering`:
 
@@ -3944,7 +3949,7 @@ See [CMock] documentation.
 
 * `:defines`:
 
- Adds list of symbols used to configure CMock's C code features in its source and header 
+ Adds list of symbols used to configure CMock’s C code features in its source and header 
  files at compile time.
 
  See [Using Unity, CMock & CException](#using-unity-cmock--cexception) for much more on
@@ -3953,43 +3958,51 @@ See [CMock] documentation.
  To manage overall command line length, these symbols are only added to compilation when
  a CMock C source file is compiled.
 
- No symbols must be set unless CMock's defaults are inappropriate for your environment 
+ No symbols must be set unless CMock’s defaults are inappropriate for your environment 
  and needs.
 
  **Default**: `[]` (empty)
 
 * `:plugins`:
 
- To add to the list Ceedling provides CMock, simply add `:cmock` ↳ `:plugins` 
- to your configuration and specify your desired additional plugins.
+ To enable CMock’s optional and advanced features available via CMock plugin, simply add 
+ `:cmock` ↳ `:plugins` to your configuration and specify your desired additional CMock 
+ plugins as a list.
 
  See [CMock's documentation][cmock-docs] to understand plugin options.
 
  [cmock-docs]: https://github.com/ThrowTheSwitch/CMock/blob/master/docs/CMock_Summary.md
 
-* `:includes`:
+ **Default**: `[]` (empty)
+
+* `:unity_helper`:
+
+ A Unity helper is a specific header file containing 
 
  If `:cmock` ↳ `:unity_helper` set, prepopulated with unity_helper file
  name (no path).
 
- The `:cmock` ↳ `:includes` list works identically to the plugins list
- above with regard to adding additional files to be inserted within
- mocks as #include statements.
+* `:includes`:
+
+ In certain advanced testing scenarios, you may need to inject additional header files 
+ into generated mocks. The filenames in this list will be transformed in `#include` 
+ directives within every generated mock.
+
+ **Default**: `[]` (empty)
 
 ### Notes on Ceedling’s nudges for CMock strict ordering
 
-The last four settings above are directly tied to other Ceedling
-settings; hence, why they are listed and explained here.
+The preceding settings are tied to other Ceedling settings; hence, why they are 
+documented here.
 
-The first setting above, `:enforce_strict_ordering`, defaults
-to `FALSE` within CMock. However, it is set to `TRUE` by default 
-in Ceedling as our way of encouraging you to use strict ordering.
+The first setting above, `:enforce_strict_ordering`, defaults to `FALSE` within
+CMock. However, it is set to `TRUE` by default in Ceedling as our way of
+encouraging you to use strict ordering.
 
-Strict ordering is teeny bit more expensive in terms of code 
-generated, test execution time, and complication in deciphering 
-test failures. However, it's good practice. And, of course, you 
-can always disable it by overriding the value in the Ceedling 
-project configuration file.
+Strict ordering is teeny bit more expensive in terms of code generated, test
+execution time, and complication in deciphering test failures. However, it’s
+good practice. And, of course, you can always disable it by overriding the
+value in the Ceedling project configuration file.
 
 ## `:unity` Configure Unity’s features
 
@@ -4004,8 +4017,8 @@ project configuration file.
  To manage overall command line length, these symbols are only added to compilation when
  a Unity C source file is compiled.
 
- No symbols must be set unless Unity's defaults are inappropriate for your environment 
- and needs.
+ **_Note_**: No symbols must be set unless Unity's defaults are inappropriate for your 
+ environment and needs.
 
  **Default**: `[]` (empty)
 

docs/Changelog.md

@@ -9,7 +9,7 @@ This changelog is complemented by two other documents:
 
 ---
 
-# [1.0.0 pre-release] — 2024-07-17
+# [1.0.0 pre-release] — 2024-07-22
 
 ## 🌟 Added
 
@@ -143,6 +143,12 @@ This fix addresses the problem detailed in PR [#728](https://github.com/ThrowThe
 
 CMock can optionally mock inline functions. This requires ingesting, modifying, and rewriting a source hearder file along with then mocking it. Sophisticated header files with complex macros can require that the source header file be preprocessed before CMock then processes it a second time. In previous versions of Ceedling the preprocessing steps and handoff to CMock were not working as intended. This has been fixed.
 
+### Bug fix for test runner generation
+
+Issue [#621](https://github.com/ThrowTheSwitch/Ceedling/issues/621)
+
+For certain advanced testing scenarios test runners generated by Ceedling + Unity must have the same `#include` list as that of the test file itself from which a runner is gnerated. Previous versions of Ceedling did not provide the proper list of `#include` directives to runner generation. This has been fixed.
+
 ### Bug fixes for command line build tasks `files:header` and `files:support`
 
 Longstanding bugs produced duplicate and sometimes incorrect lists of header files. Similarly, support file lists were not properly expanded from globs. Both of these problems have been fixed. The `files:header` command line task has replaced the `files:include` task.
@@ -185,11 +191,11 @@ The existing feature has been improved with logging and validation as well as pr
 
 Issues [#806](https://github.com/ThrowTheSwitch/Ceedling/issues/806) + [#796](https://github.com/ThrowTheSwitch/Ceedling/issues/796)
 
-Preprocessing refers to expanding macros and other related code file text manipulations often needed in sophisticated projects before key test suite generation steps. Without (optional) preprocessing, generating test funners from test files and generating mocks from header files lead to all manner of build shenanigans.
+Preprocessing refers to expanding macros and other related code text manipulations often needed in sophisticated projects before key test suite generation steps. Without (optional) preprocessing in complicated code bases, generating test runners from test files and generating mocks from header files lead to all manner of build shenanigans.
 
-The preprocessing needed by Ceedling for sophisticated projects has always been a difficult feature to implement. The most significant reason is simply that there is no readily available cross-platform C code preprocessing tool that provides Ceedling everything it needs to do its job. Even gcc’s `cpp` preprocessor tool comes up short. Over time Ceedling’s attempt at preprocessing grew more brittle and complicated as community contribturs attempted to fix it or cause it to work properly with other new features.
+The preprocessing needed by Ceedling for sophisticated projects has always been a difficult feature to implement. The most significant reason is simply that there is no readily available cross-platform C code preprocessing tool that provides Ceedling everything it needs to accomplish this task. Even gcc’s `cpp` preprocessor tool comes up short. Over time Ceedling’s attempt at preprocessing grew more brittle and complicated as community contribturs attempted to fix it or cause it to work properly with other new features.
 
-This release of Ceedling stripped the feature back to basics and largely rewrote it within the context of the new build pipeline. Complicated regular expressions and Ruby-generated temporary files have been eliminated. Instead, Ceedling now blends two reports from gcc' `cpp` tool and complements this with additional context. In addition, preprocessing now occurs at the right moments in the overall build pipeline.
+This release of Ceedling stripped the feature back to basics and largely rewrote it. Complicated regular expressions and Ruby-generated temporary files have been eliminated. Instead, Ceedling now blends two reports from gcc’s `cpp` tool and complements this with additional context. In addition, preprocessing now occurs at the right moments in the build pipeline.
 
 While this new approach is not 100% foolproof, it is far more robust and far simpler than previous attempts. Other new Ceedling features should be able to address shortcomings in edge cases.
 
@@ -205,6 +211,15 @@ Documentation on Mixins and the new options for loading a project configuration
 
 Ceedling’s previous handling of defaults and configuration processing order certainly worked, but it was not as proper as it could be. To oversimplify, default values were applied in an ordering that caused complications for advanced plugins and advanced users. This has been rectified. Default settings are now processed after all user configurations and plugins.
 
+### `:project` ↳ `:use_test_preprocessor` has new configuration options
+
+`:project` ↳ `:use_test_preprocessor` is no longer a binary setting (true/false). What is preprocessed can be chosen with the options `:none`, `:tests`, `:mocks`, `:all`. 
+
+* `:none` maps to the previous false option (preprocessing disabled).
+* `:all` maps to the previous true option running preprpocessing for all mockable header files and test C files.
+* `:mocks` enables only preprocessing of header files that are to be mocked.
+* `:tests` enables only preprocessing of your test files.
+
 ### Plugin system improvements
 
 1. The plugin subsystem has incorporated logging to trace plugin activities at high verbosity levels.
@@ -314,13 +329,15 @@ In future revisions of Ceedling, smart rebuilds will be brought back (without re
 
 Note that release builds do retain a fair amount of smart rebuild capabilities. Release builds continue to rely on Rake (for now).
 
-### Preprocessor support for Unity’s `TEST_CASE()` and `TEST_RANGE()`
+### Preprocessor support for Unity’s parameterized test case macros `TEST_CASE()` and `TEST_RANGE()`
 
-Unity’s features `TEST_CASE()` and `TEST_RANGE()` continue to work but only when `:use_test_preprocessor` is disabled. The previous project configuration option `:use_preprocessor_directives` that preserved them when preprocessing is enabled is no longer recognized.
+Unity’s `TEST_CASE()` and `TEST_RANGE()` continue to work but only when `:project` ↳ `:use_test_preprocessor` is not enabled for test files. The previous project configuration option `:use_preprocessor_directives` that preserved these and other directive macros when preprocessing is enabled is no longer recognized.
 
 `TEST_CASE()` and `TEST_RANGE()` are macros that disappear when the preprocessor digests a test file. After preprocessing, they no longer exist in the test file that is compiled.
 
-In future revisions of Ceedling, support for `TEST_CASE()` and `TEST_RANGE()` when preprocessing is enabled will be brought back (very likely without a dedicated configuration option — hopefully, we’ll get it to just work™️).
+In future revisions of Ceedling, support for `TEST_CASE()` and `TEST_RANGE()` when preprocessing is enabled will be brought back (very likely without a dedicated configuration option — hopefully, we’ll get it to just work).
+
+Note: `:project` ↳ `:use_test_preprocessor` is no longer a binary setting (true/false). Mockable header file preprocessing can be enabled with `:mocks` while test files are left as is.
 
 ### Removed background task execution
 

examples/temp_sensor/project.yml

@@ -13,7 +13,7 @@
 
  # optional features. If you don't need them, keep them turned off for performance
  :use_mocks: TRUE
- :use_test_preprocessor: TRUE
+ :use_test_preprocessor: :all
  :use_backtrace: :none
 
  # tweak the way ceedling handles automatic tasks