From 66fbfa11cbbb55bbf8fd7b8dbb4f82540d4864ce Mon Sep 17 00:00:00 2001 From: Michael Karlesky Date: Tue, 27 Aug 2024 22:56:30 -0400 Subject: [PATCH] =?UTF-8?q?=E2=9C=A8=20Tool=20definitions=20cleanup=20&=20?= =?UTF-8?q?new=20shortcut?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - Removed problematic environment variable references in default tool definitions - Removed confusing inline Ruby evaluation from tool handling (its only use) - Expanded inline Ruby string expansion in tool definitions to run each time a tool is executed - Added missing exception handling and logging around shelling out to execute a tool command line --- docs/BreakingChanges.md | 10 +- docs/CeedlingPacket.md | 138 +++++++++++++++++---------- docs/Changelog.md | 29 +++++- lib/ceedling/configurator.rb | 65 ++++++++----- lib/ceedling/constants.rb | 1 - lib/ceedling/defaults.rb | 60 +++--------- lib/ceedling/exceptions.rb | 23 ++++- lib/ceedling/setupinator.rb | 8 +- lib/ceedling/tool_executor.rb | 54 +++++++---- lib/ceedling/tool_executor_helper.rb | 56 +++++------ lib/ceedling/tool_validator.rb | 5 +- plugins/gcov/config/defaults_gcov.rb | 17 ++-- 12 files changed, 272 insertions(+), 194 deletions(-) diff --git a/docs/BreakingChanges.md b/docs/BreakingChanges.md index 0ad40e4e..fbe74ad9 100644 --- a/docs/BreakingChanges.md +++ b/docs/BreakingChanges.md @@ -7,7 +7,7 @@ These breaking changes are complemented by two other documents: --- -# [1.0.0 pre-release] — 2024-07-22 +# [1.0.0 pre-release] — 2024-08-27 ## Explicit `:paths` ↳ `:include` entries in the project file @@ -181,6 +181,14 @@ In addition, a previously undocumented feature for merging a second configuratio Thorough documentation on Mixins and the new options for loading a project configuration can be found in _[CeedlingPacket](CeedlingPacket.md))_. +## Tool definition inline Ruby evaluation replacement removed (inline Ruby string expansion remains) + +Reaching back to the earliest days of Ceedling, tool definitions supported two slightly different string replacement options that executed at different points in a build’s lifetime. Yeah. It was maybe not great. + +Support for `{...}` Ruby evaluation in tool definitions has been removed. + +Support for `#{...}` Ruby string expansion in tool definitions remains and is now evaluated each time a tool is executed during a build. + ## Command Hooks plugin configuration change In previous versions of Ceedling, the Command Hooks plugin associated tools and hooks by a naming convention within the top-level `:tools` section of your project configuration. This required some semi-ugly tool names and could lead to a rather unwieldy `:tools` list. Further, this convention also limited a hook to an association with only a single tool. diff --git a/docs/CeedlingPacket.md b/docs/CeedlingPacket.md index a6eee929..6854b6a6 100644 --- a/docs/CeedlingPacket.md +++ b/docs/CeedlingPacket.md @@ -261,11 +261,15 @@ built-in plugins come with Ceedling. ## What’s with This Name? -Glad you asked. Ceedling is tailored for unit tested C projects -and is built upon Rake (a Make replacement implemented in the Ruby -scripting language). So, we've got C, our Rake, and the fertile -soil of a build environment in which to grow and tend your project -and its unit tests. Ta da - _Ceedling_. +Glad you asked. Ceedling is tailored for unit tested C projects and is built +upon Rake, a Make replacement implemented in the Ruby scripting language. + +So, we've got C, our Rake, and the fertile soil of a build environment in which +to grow and tend your project and its unit tests. Ta da — _Ceedling_. + +Incidentally, though Rake was the backbone of the earliest versions of +Ceedling, it is now being phased out incrementally in successive releases +of this tool. The name Ceedling is not going away, however! ## What Do You Mean “Tailored for unit tested C projects”? @@ -2403,14 +2407,19 @@ for this. A few highlights from that reference page: Ceedling is able to execute inline Ruby string substitution code within the entries of certain project file configuration elements. -This evaluation occurs when the project file is loaded and processed into a -data structure for use by the Ceedling application. +In some cases, this evaluation may occurs when elements of the project +configuration are loaded and processed into a data structure for use by the +Ceedling application (e.g. path handling). In other cases, this evaluation +occurs each time a project configuration element is referenced (e.g. tools). -_Note:_ One good option for validating and troubleshooting inline Ruby string -exapnsion is use of `ceedling dumpconfig` at the command line. This application -command causes your project configuration to be processed and written to a -YAML file with any inline Ruby string expansions, well, expanded along with -defaults set, plugin actions applied, etc. +_Notes:_ +* One good option for validating and troubleshooting inline Ruby string + exapnsion is use of `ceedling dumpconfig` at the command line. This application + command causes your project configuration to be processed and written to a + YAML file with any inline Ruby string expansions, well, expanded along with + defaults set, plugin actions applied, etc. +* A commonly needed expansion is that of referencing an environment variable. + Inline Ruby string expansion supports this. See the example below. #### Ruby string expansion syntax @@ -4255,7 +4264,7 @@ A few items before we dive in: 1. Sometimes Ceedling’s built-in tools are _nearly_ what you need but not quite. If you only need to add some arguments to all uses of tool's command line, Ceedling offers a shortcut to do so. See the - [final section of the `:tools`][tool-args-shortcut] documentation for + [final section of the `:tools`][tool-definition-shortcuts] documentation for details. 1. If you need fine-grained control of the arguments Ceedling uses in the build steps for test executables, see the documentation for [`:flags`][flags]. @@ -4268,7 +4277,7 @@ A few items before we dive in: the shortcut in (1) might be the simplest option. [flags]: #flags-configure-preprocessing-compilation--linking-command-line-flags -[tool-args-shortcut]: #ceedling-tool-arguments-addition-shortcut +[tool-definition-shortcuts]: #ceedling-tool-modification-shortcuts ### Ceedling tools for test suite builds @@ -4478,29 +4487,15 @@ require that some number of its arguments or even the executable itself change for each run. Consequently, every tool’s argument list and executable field possess two means for substitution at runtime. -Ceedling provides two kinds of inline Ruby execution and a notation for -populating tool elements with dynamically gathered values within the build -environment. - -##### Tool element runtime substitution: Inline Ruby execution +Ceedling provides inline Ruby string expansion and a notation for populating +tool elements with dynamically gathered values within the build environment. -Specifically for tool configuration elements, Ceedling provides two types of -inline Ruby execution. +##### Tool element runtime substitution: Inline Ruby string expansion -1. `"#{...}"`: This notation is that of the beloved - [inline Ruby string expansion][inline-ruby-string-expansion] available in - a variety of configuration file sections. This string expansion occurs once - at startup. - -1. `{...}`: This notation causes inline Ruby execution similarly to the - preceding except that the substitution occurs each time the tool is executed. - - Why might you need this? Say you have a collection of paths on disk and some - of those paths include spaces. Further suppose that a single tool that must - use those paths requires those spaces to be escaped, but all other uses of - those paths requires the paths to remain unchanged. You could use this - Ceedling feature to insert Ruby code that iterates those paths and escapes - those spaces in the array as used by the tool of this example. +`"#{...}"`: This notation is that of the beloved +[inline Ruby string expansion][inline-ruby-string-expansion] available in a +variety of configuration file sections. This string expansion occurs each +time a tool configuration is executed during a build. ##### Tool element runtime substitution: Notational substitution @@ -4636,38 +4631,75 @@ Notes on test fixture tooling example: 1. We’re using `$stderr` redirection to allow us to capture simulator error messages to `$stdout` for display at the run's conclusion. -### Ceedling tool arguments addition shortcut +### Ceedling tool modification shortcuts Sometimes Ceedling’s default tool defininitions are _this close_ to being just -what you need. But, darn, you need one extra argument on the command line, and -you'd love to not override an entire tool definition to tweak it. +what you need. But, darn, you need one extra argument on the command line, or +you just need to hack the tool executable. You’d love to get away without +overriding an entire tool definition just in order to tweak it. + +We got you. + +#### Ceedling tool executable replacement + +Sometimes you need to do some sneaky stuff. We get it. This feature lets you +replace the executable of a tool definition — including an internal default — +with your own. + +To use this shortcut, simply add a configuration section to your project file at +the top-level, `:tools_` ↳ `:executable`. Of course, you can +combine this with the following modification option in a single block for the +tool. Executable replacement can make use of +[inline Ruby string expansion][inline-ruby-string-expansion]. + +See the list of tool names at the beginning of the `:tools` documentation to +identify the named options. Plugins can also include their own tool definitions +that can be modified with this same option. + +This example YAML... + +```yaml +:tools_test_compiler: + :executable: foo +``` + +... will produce the following: + +```shell + > foo +``` + +#### Ceedling tool arguments addition shortcut + +Now, this little feature only allows you to add arguments to the end of a tool +command line. Not the beginning. And, you can’t remove arguments with this +option. -We got you. Now, this little feature only allows you to add arguments to the -end of a tool command line. Not the beginning. And, you can’t remove arguments -with this hack. +Further, this little feature is a blanket application across all uses of a tool. +If you need fine-grained control of command line flags in build steps per test +executable, please see the [`:flags` configuration documentation][flags]. -Further, this little feature is a blanket application across all uses of a -tool. If you need fine-grained control of command line flags in build steps per -test executable, please see the [`:flags` configuration documentation][flags]. +To use this shortcut, simply add a configuration section to your project file at +the top-level, `:tools_` ↳ `:arguments`. Of course, you can +combine this with the preceding modification option in a single block for the +tool. -To use this shortcut, simply add a configuration section to your project file -at the top-level, `:tools_` ↳ `:arguments`. See the list of -tool names at the beginning of the `:tools` documentation to identify the named -options. Plugins can also include their own tool definitions that can be -modified with this same hack. +See the list of tool names at the beginning of the `:tools` documentation to +identify the named options. Plugins can also include their own tool definitions +that can be modified with this same hack. -This example YAML: +This example YAML... ```yaml :tools_test_compiler: :arguments: - - --flag # Add `--flag` to the end of all test C file compilation + - --flag # Add `--flag` to the end of all test C file compilation ``` -...will produce this command line: +... will produce the following (for the default executable): ```shell - > gcc --flag + > gcc --flag ``` ## `:plugins` Ceedling extensions diff --git a/docs/Changelog.md b/docs/Changelog.md index 359839fb..88c9d8d8 100644 --- a/docs/Changelog.md +++ b/docs/Changelog.md @@ -9,7 +9,7 @@ This changelog is complemented by two other documents: --- -# [1.0.0 pre-release] — 2024-08-12 +# [1.0.0 pre-release] — 2024-08-27 ## 🌟 Added @@ -145,7 +145,19 @@ The application commands `ceedling new` and `ceedling upgrade` at the command li If the information is unavailable such as in local development, the SHA is omitted. -This source for this string is intended to be generated and captured in the Gem at the time of an automated build in CI. +This source for this string is generated and captured in the Gem at the time of Ceedling’s automated build in CI. + +### Tool definition modification shortcuts expanded for `:executable` + +A shortcut for adding arguments to an existing tool defition already existed. The handling for this shortcut has been expanded to allow `:executable` to be redefined. + +```yaml +:tools_test_compiler: + :executable: foo # Shell out for `foo` instead of `gcc` + :arguments: # Existing functionality + - --flag1 # Add the following at the end of existing list of command line arguments + - --flag2 +``` ## 💪 Fixed @@ -317,6 +329,12 @@ In previous versions of Ceedling, the Command Hooks plugin associated tools and Hooks are now enabled within a top-level `:command_hooks` section in your project configuration. Each hook key in this configuration block can now support one or more tools organized beneath it. As such, each hook can execute one or more tools. +### Tool definition inline Ruby string expansion now happens at each execution + +Reaching back to the earliest days of Ceedling, tool definitions supported two slightly different string replacement options that executed at different points in a build’s lifetime. Yeah. It was maybe not great. This has been simplfied. + +Only support for `#{...}` Ruby string expansion in tool definitions remains. Any such expansions are now evaluated each time a tool is executed during a build. + ## 👋 Removed ### `verbosity` and `log` command line tasks have been replaced with command line switches @@ -396,3 +414,10 @@ The Gcov plugin’s `:abort_on_uncovered` option plus the related `:uncovered_ig ### Undocumented environment variable `CEEDLING_USER_PROJECT_FILE` support removed A previously undocumented feature for merging a second configuration via environment variable `CEEDLING_USER_PROJECT_FILE` has been removed. This feature has been superseded by the new Mixins functionality. + +### Tool definition inline Ruby evaluation replacement removed (inline Ruby string expansion remains) + +Reaching back to the earliest days of Ceedling, tool definitions supported two slightly different string replacement options that executed at different points in a build’s lifetime. Yeah. It was maybe not great. + +Support for `{...}` Ruby evaluation in tool definitions has been removed. Support for `#{...}` Ruby string expansion in tool definitions remains. + diff --git a/lib/ceedling/configurator.rb b/lib/ceedling/configurator.rb index b4a4a077..9f57246e 100644 --- a/lib/ceedling/configurator.rb +++ b/lib/ceedling/configurator.rb @@ -306,7 +306,6 @@ def get_cmock_config # Process our tools # - :tools entries # - Insert missing names for - # - Handle inline Ruby string substitution # - Handle needed defaults # - Configure test runner from backtrace configuration def populate_tools_config(config) @@ -323,11 +322,6 @@ def populate_tools_config(config) # Populate name if not given tool[:name] = name.to_s if (tool[:name].nil?) - # Handle inline Ruby string substitution in executable - if (tool[:executable] =~ RUBY_STRING_REPLACEMENT_PATTERN) - tool[:executable].replace(@system_wrapper.module_eval(tool[:executable])) - end - # Populate $stderr redirect option tool[:stderr_redirect] = StdErrRedirect::NONE if (tool[:stderr_redirect].nil?) @@ -337,29 +331,56 @@ def populate_tools_config(config) end - # Smoosh in extra arguments specified at top-level of config. - # This is useful for tweaking arguments for tools (where argument order does not matter). - # Arguments are squirted in at *end* of list. - def populate_tools_supplemental_arguments(config) - msg = @reportinator.generate_progress( 'Processing tool definition supplemental arguments' ) + # Process any tool definition shortcuts + # - Append extra arguments + # - Redefine executable + # + # :tools_ + # :arguments: [...] + # :executable: '...' + def populate_tools_shortcuts(config) + msg = @reportinator.generate_progress( 'Processing tool definition shortcuts' ) @loginator.log( msg, Verbosity::OBNOXIOUS ) prefix = 'tools_' - config[:tools].each do |key, tool| - name = key.to_s() - - # Supplemental tool definition - supplemental = config[(prefix + name).to_sym] - - if (not supplemental.nil?) - args_to_add = supplemental[:arguments] + config[:tools].each do |name, tool| + # Lookup shortcut tool definition (:tools_) + shortcut = (prefix + name.to_s).to_sym + + # Logging message to be built up + msg = '' + + # Try to lookup the executable from user config + executable, _ = @config_walkinator.fetch_value(shortcut, :executable, + hash:config + ) + + # Try to lookup arguments from user config + args_to_add, _ = @config_walkinator.fetch_value(shortcut, :arguments, + hash:config, + default: [] + ) + + # If either tool definition modification is happening, start building the logging message + if !args_to_add.empty? or !executable.nil? + msg += " > #{name}\n" + end - msg = " > #{name}: Arguments " + args_to_add.map{|arg| "\"#{arg}\""}.join( ', ' ) - @loginator.log( msg, Verbosity::DEBUG ) + # Log the executable and redefine the tool config + if !executable.nil? + msg += " executable: \"#{executable}\"\n" + tool[:executable] = executable + end - # Adding and flattening is not a good idea -- might over-flatten if array nesting in tool args + # Log the arguments and add to the tool config + if !args_to_add.empty? + msg += " arguments: " + args_to_add.map{|arg| "\"#{arg}\""}.join( ', ' ) + "\n" + puts(tool[:arguments]) tool[:arguments].concat( args_to_add ) end + + # Log + @loginator.log( msg, Verbosity::DEBUG ) if !msg.empty? end end diff --git a/lib/ceedling/constants.rb b/lib/ceedling/constants.rb index 8f505176..973e5afc 100644 --- a/lib/ceedling/constants.rb +++ b/lib/ceedling/constants.rb @@ -133,7 +133,6 @@ class StdErrRedirect # Match presence of any glob pattern characters GLOB_PATTERN = /[\*\?\{\}\[\]]/ RUBY_STRING_REPLACEMENT_PATTERN = /#\{.+\}/ -RUBY_EVAL_REPLACEMENT_PATTERN = /^\{(.+)\}$/ TOOL_EXECUTOR_ARGUMENT_REPLACEMENT_PATTERN = /(\$\{(\d+)\})/ TEST_STDOUT_STATISTICS_PATTERN = /\n-+\s*(\d+)\s+Tests\s+(\d+)\s+Failures\s+(\d+)\s+Ignored\s+(OK|FAIL)\s*/i diff --git a/lib/ceedling/defaults.rb b/lib/ceedling/defaults.rb index e7b1a71b..5f4663fa 100644 --- a/lib/ceedling/defaults.rb +++ b/lib/ceedling/defaults.rb @@ -16,16 +16,14 @@ CEEDLING_PLUGINS = [] unless defined? CEEDLING_PLUGINS DEFAULT_TEST_COMPILER_TOOL = { - :executable => ENV['TEST_CC'].nil? ? FilePathUtils.os_executable_ext('gcc').freeze : ENV['TEST_CC'], + :executable => FilePathUtils.os_executable_ext('gcc').freeze, :name => 'default_test_compiler'.freeze, :optional => false.freeze, :arguments => [ - ENV['CPPFLAGS'].nil? ? "" : ENV['CPPFLAGS'].split, "-I\"${5}\"".freeze, # Per-test executable search paths "-D\"${6}\"".freeze, # Per-test executable defines "-DGNU_COMPILER".freeze, # OSX clang "-g".freeze, - ENV['CFLAGS'].nil? ? "" : ENV['CFLAGS'].split, "-c \"${1}\"".freeze, "-o \"${2}\"".freeze, # gcc's list file output options are complex; no use of ${3} parameter in default config @@ -35,11 +33,10 @@ } DEFAULT_TEST_ASSEMBLER_TOOL = { - :executable => ENV['TEST_AS'].nil? ? FilePathUtils.os_executable_ext('as').freeze : ENV['TEST_AS'], + :executable => FilePathUtils.os_executable_ext('as').freeze, :name => 'default_test_assembler'.freeze, :optional => false.freeze, :arguments => [ - ENV['TEST_ASFLAGS'].nil? ? "" : ENV['TEST_ASFLAGS'].split, "-I\"${3}\"".freeze, # Search paths # Any defines (${4}) are not included since GNU assembler ignores them "\"${1}\"".freeze, @@ -48,18 +45,15 @@ } DEFAULT_TEST_LINKER_TOOL = { - :executable => ENV['TEST_CCLD'].nil? ? FilePathUtils.os_executable_ext('gcc').freeze : ENV['TEST_CCLD'], + :executable => FilePathUtils.os_executable_ext('gcc').freeze, :name => 'default_test_linker'.freeze, :optional => false.freeze, :arguments => [ - ENV['TEST_CFLAGS'].nil? ? "" : ENV['TEST_CFLAGS'].split, - ENV['TEST_LDFLAGS'].nil? ? "" : ENV['TEST_LDFLAGS'].split, "${1}".freeze, "${5}".freeze, "-o \"${2}\"".freeze, "".freeze, "${4}".freeze, - ENV['TEST_LDLIBS'].nil? ? "" : ENV['TEST_LDLIBS'].split ].freeze } @@ -80,11 +74,10 @@ } DEFAULT_TEST_SHALLOW_INCLUDES_PREPROCESSOR_TOOL = { - :executable => ENV['TEST_CC'].nil? ? FilePathUtils.os_executable_ext('gcc').freeze : ENV['TEST_CC'], + :executable => FilePathUtils.os_executable_ext('gcc').freeze, :name => 'default_test_shallow_includes_preprocessor'.freeze, :optional => false.freeze, :arguments => [ - ENV['TEST_CPPFLAGS'].nil? ? "" : ENV['TEST_CPPFLAGS'].split, '-E'.freeze, # Run only through preprocessor stage with its output '-MM'.freeze, # Output make rule + suppress header files found in system header directories '-MG'.freeze, # Assume missing header files are generated files (do not discard) @@ -98,11 +91,10 @@ } DEFAULT_TEST_NESTED_INCLUDES_PREPROCESSOR_TOOL = { - :executable => ENV['TEST_CC'].nil? ? FilePathUtils.os_executable_ext('gcc').freeze : ENV['TEST_CC'], + :executable => FilePathUtils.os_executable_ext('gcc').freeze, :name => 'default_test_nested_includes_preprocessor'.freeze, :optional => false.freeze, :arguments => [ - ENV['TEST_CPPFLAGS'].nil? ? "" : ENV['TEST_CPPFLAGS'].split, '-E'.freeze, # Run only through preprocessor stage with its output '-MM'.freeze, # Output make rule + suppress header files found in system header directories '-MG'.freeze, # Assume missing header files are generated files (do not discard) @@ -117,11 +109,10 @@ } DEFAULT_TEST_FILE_PREPROCESSOR_TOOL = { - :executable => ENV['TEST_CC'].nil? ? FilePathUtils.os_executable_ext('gcc').freeze : ENV['TEST_CC'], + :executable => FilePathUtils.os_executable_ext('gcc').freeze, :name => 'default_test_file_preprocessor'.freeze, :optional => false.freeze, :arguments => [ - ENV['TEST_CPPFLAGS'].nil? ? "" : ENV['TEST_CPPFLAGS'].split, '-E'.freeze, "-I\"${4}\"".freeze, # Per-test executable search paths "-D\"${3}\"".freeze, # Per-test executable defines @@ -141,11 +132,10 @@ end DEFAULT_TEST_DEPENDENCIES_GENERATOR_TOOL = { - :executable => ENV['TEST_CC'].nil? ? FilePathUtils.os_executable_ext('gcc').freeze : ENV['TEST_CC'], + :executable => FilePathUtils.os_executable_ext('gcc').freeze, :name => 'default_test_dependencies_generator'.freeze, :optional => false.freeze, :arguments => [ - ENV['TEST_CPPFLAGS'].nil? ? "" : ENV['TEST_CPPFLAGS'].split, '-E'.freeze, "-I\"${5}\"".freeze, # Per-test executable search paths "-D\"${4}\"".freeze, # Per-test executable defines @@ -162,11 +152,10 @@ } DEFAULT_RELEASE_DEPENDENCIES_GENERATOR_TOOL = { - :executable => ENV['RELEASE_CC'].nil? ? FilePathUtils.os_executable_ext('gcc').freeze : ENV['RELEASE_CC'], + :executable => FilePathUtils.os_executable_ext('gcc').freeze, :name => 'default_release_dependencies_generator'.freeze, :optional => false.freeze, :arguments => [ - ENV['RELEASE_CPPFLAGS'].nil? ? "" : ENV['RELEASE_CPPFLAGS'].split, '-E'.freeze, {"-I\"$\"" => 'COLLECTION_PATHS_SOURCE_INCLUDE_VENDOR'}.freeze, {"-I\"$\"" => 'COLLECTION_PATHS_RELEASE_TOOLCHAIN_INCLUDE'}.freeze, @@ -185,15 +174,13 @@ } DEFAULT_RELEASE_COMPILER_TOOL = { - :executable => ENV['RELEASE_CC'].nil? ? FilePathUtils.os_executable_ext('gcc').freeze : ENV['RELEASE_CC'], + :executable => FilePathUtils.os_executable_ext('gcc').freeze, :name => 'default_release_compiler'.freeze, :optional => false.freeze, :arguments => [ - ENV['RELEASE_CPPFLAGS'].nil? ? "" : ENV['RELEASE_CPPFLAGS'].split, "-I\"${5}\"".freeze, # Search paths "-D\"${6}\"".freeze, # Defines "-DGNU_COMPILER".freeze, - ENV['RELEASE_CFLAGS'].nil? ? "" : ENV['RELEASE_CFLAGS'].split, "-c \"${1}\"".freeze, "-o \"${2}\"".freeze, # gcc's list file output options are complex; no use of ${3} parameter in default config @@ -203,11 +190,10 @@ } DEFAULT_RELEASE_ASSEMBLER_TOOL = { - :executable => ENV['RELEASE_AS'].nil? ? FilePathUtils.os_executable_ext('as').freeze : ENV['RELEASE_AS'], + :executable => FilePathUtils.os_executable_ext('as').freeze, :name => 'default_release_assembler'.freeze, :optional => false.freeze, :arguments => [ - ENV['RELEASE_ASFLAGS'].nil? ? "" : ENV['RELEASE_ASFLAGS'].split, "-I\"${3}\"".freeze, # Search paths "-D\"${4}\"".freeze, # Defines (FYI--allowed with GNU assembler but ignored) "\"${1}\"".freeze, @@ -216,23 +202,20 @@ } DEFAULT_RELEASE_LINKER_TOOL = { - :executable => ENV['RELEASE_CCLD'].nil? ? FilePathUtils.os_executable_ext('gcc').freeze : ENV['RELEASE_CCLD'], + :executable => FilePathUtils.os_executable_ext('gcc').freeze, :name => 'default_release_linker'.freeze, :optional => false.freeze, :arguments => [ - ENV['RELEASE_CFLAGS'].nil? ? "" : ENV['RELEASE_CFLAGS'].split, - ENV['RELEASE_LDFLAGS'].nil? ? "" : ENV['RELEASE_LDFLAGS'].split, "\"${1}\"".freeze, "${5}".freeze, "-o \"${2}\"".freeze, "".freeze, "${4}".freeze, - ENV['RELEASE_LDLIBS'].nil? ? "" : ENV['RELEASE_LDLIBS'].split ].freeze } DEFAULT_TEST_BACKTRACE_GDB_TOOL = { - :executable => ENV['GDB'].nil? ? FilePathUtils.os_executable_ext('gdb').freeze : ENV['GDB'], + :executable => FilePathUtils.os_executable_ext('gdb').freeze, :name => 'default_test_backtrace_gdb'.freeze, :optional => false.freeze, :arguments => [ @@ -415,25 +398,6 @@ # All tools populated while building up config / defaults structure :tools => {}, - - # Empty argument lists for default tools - # Note: These can be overridden in project file to add arguments totally redefining tools - :test_compiler => { :arguments => [] }, - :test_assembler => { :arguments => [] }, - :test_linker => { :arguments => [] }, - :test_fixture => { - :arguments => [], - :link_objects => [], # compiled object files to always be linked in (e.g. cmock.o if using mocks) - }, - :test_backtrace_gdb => { :arguments => [] }, - :test_includes_preprocessor => { :arguments => [] }, - :test_file_preprocessor => { :arguments => [] }, - :test_file_preprocessor_directives => { :arguments => [] }, - :test_dependencies_generator => { :arguments => [] }, - :release_compiler => { :arguments => [] }, - :release_linker => { :arguments => [] }, - :release_assembler => { :arguments => [] }, - :release_dependencies_generator => { :arguments => [] } }.freeze diff --git a/lib/ceedling/exceptions.rb b/lib/ceedling/exceptions.rb index fec507b6..3c34f8a9 100644 --- a/lib/ceedling/exceptions.rb +++ b/lib/ceedling/exceptions.rb @@ -7,21 +7,34 @@ require 'ceedling/constants' + class CeedlingException < RuntimeError # Nothing at the moment end + class ShellExecutionException < CeedlingException + attr_reader :shell_result - def initialize(shell_result:, name:) + + def initialize(shell_result:{}, name:, message:'') @shell_result = shell_result - message = name + " terminated with exit code [#{shell_result[:exit_code]}]" + # If shell results exist... + if !shell_result.empty? + message = "Tool #{name} terminated with exit code [#{shell_result[:exit_code]}]" + + if !shell_result[:output].empty? + message += " and output >> \"#{shell_result[:output].strip()}\"" + end + + super( message ) - if !shell_result[:output].empty? - message += " and output >> \"#{shell_result[:output].strip()}\"" + # Otherwise, just report the provided message + else + message = "Tool #{name} encountered an error:: #{message}" + super( message ) end - super( message ) end end diff --git a/lib/ceedling/setupinator.rb b/lib/ceedling/setupinator.rb index 4b62b747..321ab861 100644 --- a/lib/ceedling/setupinator.rb +++ b/lib/ceedling/setupinator.rb @@ -135,9 +135,13 @@ def do_setup( app_cfg ) @configurator.eval_defines( config_hash ) @configurator.standardize_paths( config_hash ) - # Fill out any missing tool config value / supplement arguments + # Fill out any missing tool config value @configurator.populate_tools_config( config_hash ) - @configurator.populate_tools_supplemental_arguments( config_hash ) + + # From any tool definition shortcuts: + # - Redefine executable if set + # - Add arguments from tool definition shortcuts if set + @configurator.populate_tools_shortcuts( config_hash ) # Configure test runner build & runtime options @test_runner_manager.configure_build_options( config_hash ) diff --git a/lib/ceedling/tool_executor.rb b/lib/ceedling/tool_executor.rb index cbfeb90f..7c428d23 100644 --- a/lib/ceedling/tool_executor.rb +++ b/lib/ceedling/tool_executor.rb @@ -37,8 +37,12 @@ def build_command_line(tool_config, extra_params, *args) build_arguments(tool_config[:name], tool_config[:arguments], *args), ].reject{|s| s.nil? || s.empty?}.join(' ').strip + # Log command as is @loginator.log( "Command: #{command}", Verbosity::DEBUG ) + # Update executable after any expansion + command[:executable] = executable + return command end @@ -59,27 +63,32 @@ def exec(command, args=[]) shell_result = {} - time = Benchmark.realtime do - shell_result = @system_wrapper.shell_capture3( command:command_line, boom:options[:boom] ) - end - shell_result[:time] = time + # Wrap system level tool execution in exception handling + begin + time = Benchmark.realtime do + shell_result = @system_wrapper.shell_capture3( command:command_line, boom:options[:boom] ) + end + shell_result[:time] = time + + # Ultimately, re-raise the exception as ShellExecutionException populated with the exception message + rescue => error + raise ShellExecutionException.new( name:pretty_tool_name( command ), message: error.message ) + + # Be sure to log what we can + ensure + # Scrub the string for illegal output + unless shell_result[:output].nil? + shell_result[:output] = shell_result[:output].scrub if "".respond_to?(:scrub) + shell_result[:output].gsub!(/\033\[\d\dm/,'') + end - # Scrub the string for illegal output - unless shell_result[:output].nil? - shell_result[:output] = shell_result[:output].scrub if "".respond_to?(:scrub) - shell_result[:output].gsub!(/\033\[\d\dm/,'') + @tool_executor_helper.log_results( command_line, shell_result ) end - @tool_executor_helper.log_results( command_line, shell_result ) - # Go boom if exit code is not 0 and that code means a fatal error # (Sometimes we don't want a non-0 exit code to cause an exception as the exit code may not mean a build-ending failure) if ((shell_result[:exit_code] != 0) and options[:boom]) - raise ShellExecutionException.new( - shell_result: shell_result, - # Titleize the command's name -- each word is capitalized and any underscores replaced with spaces - name: "'#{command[:name].split(/ |\_/).map(&:capitalize).join(" ")}' " + "(#{command[:executable]})" - ) + raise ShellExecutionException.new( shell_result:shell_result, name:pretty_tool_name( command ) ) end return shell_result @@ -141,11 +150,6 @@ def expandify_element(tool_name, element, *args) element.sub!(/\\\$/, '$') element.strip! - # handle inline ruby execution - if (element =~ RUBY_EVAL_REPLACEMENT_PATTERN) - element.replace(eval($1)) - end - build_string = '' # handle array or anything else passed into method to be expanded in place of replacement operators @@ -219,4 +223,14 @@ def dehashify_argument_elements(tool_name, hash) return build_string.strip end + def pretty_tool_name(command) + # Titleize command's name -- each word capitalized plus underscores replaced with spaces + name = "#{command[:name].split(/ |\_/).map(&:capitalize).join(" ")}" + + executable = command[:executable].empty? ? '' : command[:executable] + + # 'Name' (executable) + return "'#{name}' " + "(#{executable})" + end + end diff --git a/lib/ceedling/tool_executor_helper.rb b/lib/ceedling/tool_executor_helper.rb index 3c30cd55..0d17d412 100644 --- a/lib/ceedling/tool_executor_helper.rb +++ b/lib/ceedling/tool_executor_helper.rb @@ -69,33 +69,35 @@ def log_results(command_str, shell_result) output = "> Shell executed command:\n" output += "`#{command_str}`\n" - # Detailed debug logging - if @verbosinator.should_output?( Verbosity::DEBUG ) - output += "> With $stdout: " - output += shell_result[:stdout].empty? ? "\n" : "\n#{shell_result[:stdout].strip()}\n" - - output += "> With $stderr: " - output += shell_result[:stderr].empty? ? "\n" : "\n#{shell_result[:stderr].strip()}\n" - - output += "> And terminated with status: #{shell_result[:status]}\n" - - @loginator.log( '', Verbosity::DEBUG ) - @loginator.log( output, Verbosity::DEBUG ) - @loginator.log( '', Verbosity::DEBUG ) - - return # Bail out - end - - # Slightly less verbose obnoxious logging - if !shell_result[:output].empty? - output += "> Produced output: " - output += shell_result[:output].strip().empty? ? "\n" : "\n#{shell_result[:output].strip()}\n" - end - - if !shell_result[:exit_code].nil? - output += "> And terminated with exit code: [#{shell_result[:exit_code]}]\n" - else - output += "> And exited prematurely\n" + if !shell_result.empty? + # Detailed debug logging + if @verbosinator.should_output?( Verbosity::DEBUG ) + output += "> With $stdout: " + output += shell_result[:stdout].empty? ? "\n" : "\n#{shell_result[:stdout].strip()}\n" + + output += "> With $stderr: " + output += shell_result[:stderr].empty? ? "\n" : "\n#{shell_result[:stderr].strip()}\n" + + output += "> And terminated with status: #{shell_result[:status]}\n" + + @loginator.log( '', Verbosity::DEBUG ) + @loginator.log( output, Verbosity::DEBUG ) + @loginator.log( '', Verbosity::DEBUG ) + + return # Bail out + end + + # Slightly less verbose obnoxious logging + if !shell_result[:output].empty? + output += "> Produced output: " + output += shell_result[:output].strip().empty? ? "\n" : "\n#{shell_result[:output].strip()}\n" + end + + if !shell_result[:exit_code].nil? + output += "> And terminated with exit code: [#{shell_result[:exit_code]}]\n" + else + output += "> And exited prematurely\n" + end end @loginator.log( '', Verbosity::OBNOXIOUS ) diff --git a/lib/ceedling/tool_validator.rb b/lib/ceedling/tool_validator.rb index fe11fb46..30c567fd 100644 --- a/lib/ceedling/tool_validator.rb +++ b/lib/ceedling/tool_validator.rb @@ -41,7 +41,7 @@ def validate_executable(tool:, name:, extension:, respect_optional:, boom:) # Handle a missing :executable if (executable.nil? or executable.empty?) - error = "#{name} is missing :executable in its configuration." + error = "Tool #{name} is missing :executable in its configuration." if !boom @loginator.log( error, Verbosity::ERRORS ) return false @@ -53,9 +53,10 @@ def validate_executable(tool:, name:, extension:, respect_optional:, boom:) # If tool is optional and we're respecting that, don't bother to check if executable is legit return true if tool[:optional] and respect_optional - # Skip everything if we've got an argument replacement pattern in :executable + # Skip everything if we've got an argument replacement pattern or Ruby string replacement in :executable # (Allow executable to be validated by shell at run time) return true if (executable =~ TOOL_EXECUTOR_ARGUMENT_REPLACEMENT_PATTERN) + return true if (executable =~ RUBY_STRING_REPLACEMENT_PATTERN) # Extract the executable (including optional filepath) apart from any additional arguments # Be mindful of legal quote enclosures (e.g. `"Code Cruncher" foo bar`) diff --git a/plugins/gcov/config/defaults_gcov.rb b/plugins/gcov/config/defaults_gcov.rb index 8107bbbe..5b5d8ab0 100644 --- a/plugins/gcov/config/defaults_gcov.rb +++ b/plugins/gcov/config/defaults_gcov.rb @@ -6,19 +6,17 @@ # ========================================================================= DEFAULT_GCOV_COMPILER_TOOL = { - :executable => ENV['GCOV_CC'].nil? ? FilePathUtils.os_executable_ext('gcc').freeze : ENV['GCOV_CC'], + :executable => FilePathUtils.os_executable_ext('gcc').freeze, :name => 'default_gcov_compiler'.freeze, :optional => false.freeze, :arguments => [ "-g".freeze, "-fprofile-arcs".freeze, "-ftest-coverage".freeze, - ENV['GCOV_CPPFLAGS'].nil? ? "" : ENV['GCOV_CPPFLAGS'].split, "-I\"${5}\"".freeze, # Per-test executable search paths "-D\"${6}\"".freeze, # Per-test executable defines "-DGCOV_COMPILER".freeze, "-DCODE_COVERAGE".freeze, - ENV['GCOV_CFLAGS'].nil? ? "" : ENV['GCOV_CFLAGS'].split, "-c \"${1}\"".freeze, "-o \"${2}\"".freeze, # gcc's list file output options are complex; no use of ${3} parameter in default config @@ -28,20 +26,17 @@ } DEFAULT_GCOV_LINKER_TOOL = { - :executable => ENV['GCOV_CCLD'].nil? ? FilePathUtils.os_executable_ext('gcc').freeze : ENV['GCOV_CCLD'], + :executable => FilePathUtils.os_executable_ext('gcc').freeze, :name => 'default_gcov_linker'.freeze, :optional => false.freeze, :arguments => [ "-g".freeze, "-fprofile-arcs".freeze, "-ftest-coverage".freeze, - ENV['GCOV_CFLAGS'].nil? ? "" : ENV['GCOV_CFLAGS'].split, - ENV['GCOV_LDFLAGS'].nil? ? "" : ENV['GCOV_LDFLAGS'].split, "${1}".freeze, "${5}".freeze, "-o \"${2}\"".freeze, "${4}".freeze, - ENV['GCOV_LDLIBS'].nil? ? "" : ENV['GCOV_LDLIBS'].split ].freeze } @@ -54,7 +49,7 @@ # Produce summaries printed to console DEFAULT_GCOV_SUMMARY_TOOL = { - :executable => ENV['GCOV_SUMMARY'].nil? ? FilePathUtils.os_executable_ext('gcov').freeze : ENV['GCOV_SUMMARY'], + :executable => FilePathUtils.os_executable_ext('gcov').freeze, :name => 'default_gcov_summary'.freeze, :optional => true.freeze, :arguments => [ @@ -68,7 +63,7 @@ # Produce .gcov files (used in conjunction with ReportGenerator) DEFAULT_GCOV_REPORT_TOOL = { - :executable => ENV['GCOV_REPORT'].nil? ? FilePathUtils.os_executable_ext('gcov').freeze : ENV['GCOV_REPORT'], + :executable => FilePathUtils.os_executable_ext('gcov').freeze, :name => 'default_gcov_report'.freeze, :optional => true.freeze, :arguments => [ @@ -83,7 +78,7 @@ # Produce reports with `gcovr` DEFAULT_GCOV_GCOVR_REPORT_TOOL = { # No extension handling -- `gcovr` is generally an extensionless Python script - :executable => ENV['GCOVR'].nil? ? 'gcovr'.freeze : ENV['GCOVR'], + :executable => 'gcovr'.freeze, :name => 'default_gcov_gcovr_report'.freeze, :optional => true.freeze, :arguments => [ @@ -93,7 +88,7 @@ # Produce reports with `reportgenerator` DEFAULT_GCOV_REPORTGENERATOR_REPORT_TOOL = { - :executable => ENV['REPORTGENERATOR'].nil? ? FilePathUtils.os_executable_ext('reportgenerator').freeze : ENV['REPORTGENERATOR'], + :executable => FilePathUtils.os_executable_ext('reportgenerator').freeze, :name => 'default_gcov_reportgenerator_report'.freeze, :optional => true.freeze, :arguments => [