Merge pull request #11 from CDLUC3/issue6-ssm_root_path_as_list

Issue6 ssm root path as list

README.md

@@ -10,9 +10,10 @@ Intended for use by CDL UC3 services.  We rely on EC2 instance profiles to provi
 
 ### Parameters
 
-- `ssm_root_path`: prefix to apply to all parameter name lookups.  This must be
-  a fully qualified parameter path, i.e. it must start with a forward slash
-  ('/').  Defaults to value of environment var `SSM_ROOT_PATH` if defined.
+- `ssm_root_path`: colon separated list of path prefixes to apply to all
+  parameter name lookups.  Each path prefix in `ssm_root_path` must be a fully
+  qualified parameter path, i.e. it must start with a forward slash ('/').
+  Defaults to value of environment var `SSM_ROOT_PATH` if defined.
 
 - `region`: AWS region in which to perform the SSM lookup.  Defaults to value
   of environment var `AWS_REGION` if defined, or failing that, to `us-west-2`. 
@@ -25,7 +26,7 @@ Intended for use by CDL UC3 services.  We rely on EC2 instance profiles to provi
 - `ssm_skip_resolution`: boolean flag.   When set, no SSM ParameterStore
   lookups will occur.  Key lookups fall back to local environment lookups or to
   defined default values.  Defaults to value of environment var
-  `SSM_SKIP_RESOLUTION` if defined.
+  `SSM_SKIP_RESOLUTION`, or to 'false' if `SSM_SKIP_RESOLUTION` is not defined.
 
 
 ### Instantiation 
@@ -43,15 +44,15 @@ Explicit parameter declaration.  All unqualified lookup keys will have the
 
 ```ruby
 myResolver = Uc3Ssm::ConfigResolver.new(
-  ssm_root_path: "/my/root/path"
+  ssm_root_path: "/my/root/path:/my/other/root/path"
   region: "us-west-2",
 )
 ```
 
 Implicit parameter declaration using environment vars.
 
 ```ruby
-ENV['SSM_ROOT_PATH'] = '/my/other/root/path'
+ENV['SSM_ROOT_PATH'] = '/my/root/path:/my/other/root/path'
 ENV['AWS_REGION'] = 'us-west-2'
 myResolver = Uc3Ssm::ConfigResolver.new()
 ```
@@ -65,8 +66,12 @@ perform a simple lookup for a single ssm parameter.
 
 When `key` is prefixed be a forward slash (e.g. `/cleverman` or
 `/price/tea/china`), it is considered to be a fully qualified parameter name
-and is passed 'as is' to SSM.  If not so prefixed, then `ssm_root_path` is
-prepended to `key` to form a fully qualified parameter name.
+and is passed 'as is' to SSM.
+
+If `key` is not fully qaulified, then each path prefix in `ssm_root_path` is
+prepended to `key` to form a fully qualified parameter name.  A lookup is
+performed for each such fully qualified parameter name in order until a value
+is found or all lookups fail.
 
 NOTE: if `ssm_root_path` is not defined, and `key` is unqualified (no forward
 slash prefix), an exception is thrown.
@@ -75,12 +80,15 @@ slash prefix), an exception is thrown.
 Example:
 
 ```ruby
-myResolver = Uc3Ssm::ConfigResolver.new(ssm_root_path: "/my/root/path")
+myResolver = Uc3Ssm::ConfigResolver.new(
+  ssm_root_path: "/my/root/path:/my/other/root/path"
+)
 myResolver.parameter_for_key('/cheese/blue')
 # returns value for parameter name '/cheese/blue'
 
 myResolver.parameter_for_key('blee')
-# returns value for parameter name '/my/root/path/blee'
+# performs a lookup for parameter name '/my/root/path/blee'.`
+# if this is not found, performs a lookup for '/my/other/root/path/blee'.
 
 myDefaultResolver = Uc3Ssm::ConfigResolver.new()
 myDefaultResolver.parameter_for_key('blee')
@@ -102,22 +110,25 @@ client_secret = ssm.parameter_for_key('client_secret') || ''
 perform a lookup for all parameters prefixed by `options['path']`.
 
 As with `myResolver.parameter_for_key(key)`, when `options[path]` is not fully
-qualified, `ssm_root_path` is prepended to `path` to form a fully qualified
-parameter path.  If `options['path'] is not specified, then the search is done
-with `ssm_root_path.
+qualified, each path prefix in `ssm_root_path` is prepended to `path` to form a
+fully qualified parameter path.  If `options['path'] is not specified, then the
+search is done for each path prefix in `ssm_root_path`.  Returns a list of
+parameters which is the union of all searches made.
 
 All other keys in `options` are passed to `Aws::SSM::Client.get_parameters_by_path`.
 See https://docs.aws.amazon.com/sdk-for-ruby/v2/api/Aws/SSM/Client.html#get_parameters_by_path-instance_method
 
 Example:
 
 ```ruby
-myResolver = Uc3Ssm::ConfigResolver.new(ssm_root_path: "/my/base/path")
+myResolver = Uc3Ssm::ConfigResolver.new(
+  ssm_root_path: "/my/base/path:/my/other/path"
+)
 myResolver.parameter_for_key(path: 'args')
-# returns values for all parameter names directly under "/my/base/path/args"
+# returns values for all parameter names directly under both `/my/base/path/args` and `/my/other/path`.`
 
 myResolver.parameter_for_key(path: 'args', resursive: true)
-# returns values for all parameter names recursively starting with "/my/base/path/args"
+# returns values for all parameter names recursively under both `/my/base/path/args` and `/my/other/path`.
 ```
 
 

lib/uc3-ssm.rb

@@ -22,20 +22,28 @@ class ConfigResolver
     #          Not needed if AWS_REGION is configured.
     # ssm_root_path - prefix to apply to all key lookups.
     #                 This allows the same config to be used in prod and non prod envs.
+    #                 Can be a list of path strings separated by ':', in which case the
+    #                 we search for keys under each path sequentially, returning the value
+    #                 of the first matching key found.  Example:
+    #                   ssm_root_path: '/prog/srvc/subsrvc/env:/prod/srvc/subsrvc/default'
     # rubocop:disable Metrics/MethodLength, Metrics/AbcSize
     def initialize(**options)
+      # see issue #9 - @regex should not be a user definable option
       dflt_regex = '^(.*)\\{!(ENV|SSM):\\s*([^\\}!]*)(!DEFAULT:\\s([^\\}]*))?\\}(.*)$'
-      dflt_ssm_root_path = ENV['SSM_ROOT_PATH'] || ''
-      dflt_region = ENV['AWS_REGION'] || 'us-west-2'
+      @regex = options.fetch(:regex, dflt_regex)
+
+      # see issue #10 - @ssm_skip_resolution only settable as ENV var
       @ssm_skip_resolution = ENV.key?('SSM_SKIP_RESOLUTION')
+      # dflt_ssm_skip_resolution = ENV['SSM_SKIP_RESOLUTION'] || false
+      # @ssm_skip_resolution = options.fetch(:ssm_skip_resolution, dflt_ssm_skip_resolution)
 
-      @logger = options.fetch(:logger, Logger.new(STDOUT))
+      dflt_region = ENV['AWS_REGION'] || 'us-west-2'
+      dflt_ssm_root_path = ENV['SSM_ROOT_PATH'] || ''
 
       @region = options.fetch(:region, dflt_region)
-      @regex = options.fetch(:regex, dflt_regex)
       @ssm_root_path = sanitize_root_path(options.fetch(:ssm_root_path, dflt_ssm_root_path))
       @def_value = options.fetch(:def_value, '')
-
+      @logger = options.fetch(:logger, Logger.new(STDOUT))
       @client = Aws::SSM::Client.new(region: @region) unless @ssm_skip_resolution
     rescue Aws::Errors::MissingRegionError
       raise ConfigResolverError, 'No AWS region defined. Either set ENV["AWS_REGION"] or pass in `region: [region]`'
@@ -69,31 +77,54 @@ def resolve_hash_values(hash:, resolve_key: nil, return_key: nil)
     # Retrieve all key+values for a path (using the ssm_root_path if none is specified)
     # See https://docs.aws.amazon.com/sdk-for-ruby/v2/api/Aws/SSM/Client.html#get_parameters_by_path-instance_method
     # details on available `options`
+    # rubocop:disable Metrics/MethodLength, Metrics/AbcSize, Metrics/CyclomaticComplexity
     def parameters_for_path(**options)
       return [] if @ssm_skip_resolution
 
-      options[:path] = options[:path].nil? ? @ssm_root_path : sanitize_parameter_key(options[:path])
-      resp = @client.get_parameters_by_path(options)
-      !resp.nil? && resp.parameters.any? ? resp.parameters : []
+      param_list = []
+      path_list = options[:path].nil? ? @ssm_root_path : sanitize_parameter_key(options[:path])
+      path_list.each do |root_path|
+        begin
+          options[:path] = root_path
+          resp = @client.get_parameters_by_path(options)
+          param_list += resp.parameters if !resp.nil? && resp.parameters.any?
+        rescue Aws::SSM::Errors::ParameterNotFound
+          @logger.debug "ParameterNotFound for path '#{root_path}' in parameters_by_path"
+          next
+        end
+      end
+
+      param_list
     rescue Aws::Errors::MissingCredentialsError
       raise ConfigResolverError, 'No AWS credentials available. Make sure the server has access to the aws-sdk'
     end
+    # rubocop:enable Metrics/MethodLength, Metrics/AbcSize, Metrics/CyclomaticComplexity
 
     # Retrieve a value for a single key
     def parameter_for_key(key)
-      key = sanitize_parameter_key(key)
-      retrieve_ssm_value(key)
+      return key if @ssm_skip_resolution
+
+      keylist = sanitize_parameter_key(key)
+      keylist.each do |k|
+        val = retrieve_ssm_value(k)
+        return val unless val.nil?
+      end
     end
 
     private
 
-    # Ensure root_path starts and ends with '/'
+    # Split root_path string into an array of root_paths.
+    # Ensure each root_path starts and ends with '/'
     def sanitize_root_path(root_path)
-      return root_path if root_path.empty?
+      return [] if root_path.empty?
 
-      raise ConfigResolverError, 'ssm_root_path must start with forward slash' unless root_path.start_with?('/')
+      root_path_list = []
+      root_path.split(':').each do |path|
+        raise ConfigResolverError, 'ssm_root_path must start with forward slash' unless path.start_with?('/')
 
-      root_path.end_with?('/') ? root_path : root_path + '/'
+        root_path_list.push(path.end_with?('/') ? path : path + '/')
+      end
+      root_path_list
     end
 
     def return_hash(hash, return_key = nil)
@@ -159,14 +190,13 @@ def lookup_env(key, defval = nil)
       raise ConfigResolverError, "Environment variable #{key} not found, no default provided"
     end
 
-    # rubocop:disable Metrics/MethodLength
     def lookup_ssm(key, defval = nil)
-      key = sanitize_parameter_key(key)
-      begin
-        val = retrieve_ssm_value(key)
+      return key if @ssm_skip_resolution
+
+      keylist = sanitize_parameter_key(key)
+      keylist.each do |k|
+        val = retrieve_ssm_value(k)
         return val unless val.nil?
-      rescue ConfigResolverError
-        @logger.warn "SSM key #{key} not found"
       end
       return defval unless defval.nil?
 
@@ -175,24 +205,35 @@ def lookup_ssm(key, defval = nil)
 
       raise ConfigResolverError, "SSM key #{key} not found, no default provided"
     end
-    # rubocop:enable Metrics/MethodLength
 
-    # Prepend ssm_root_path to `key` to make fully qualified parameter name
+    # Return an array of fully qualified parameter names. For each root_path in
+    # @ssm_root_path prepend root_path to `key` to make fully qualified
+    # parameter name.
     def sanitize_parameter_key(key)
       key_missing_msg = 'SSM paramter name not valid.  Must be a non-empty string.'
       raise ConfigResolverError, key_missing_msg.to_s if key.nil? || key.empty?
 
+      return [key] if key.start_with?('/')
+
       key_not_qualified_msg = 'SSM parameter name is not fully qualified and no ssm_root_path defined.'
-      raise ConfigResolverError, key_not_qualified_msg.to_s if !key.start_with?('/') && @ssm_root_path.empty?
+      raise ConfigResolverError, key_not_qualified_msg.to_s if @ssm_root_path.empty?
+
+      keylist = []
+      @ssm_root_path.each do |root_path|
+        keylist.push("#{root_path}#{key}")
+      end
 
-      "#{@ssm_root_path}#{key}".strip
+      keylist
     end
 
     # Attempt to retrieve the value from AWS SSM
     def retrieve_ssm_value(key)
       return key if @ssm_skip_resolution
 
       @client.get_parameter(name: key)[:parameter][:value]
+    rescue Aws::SSM::Errors::ParameterNotFound
+      @logger.debug "ParameterNotFound for key '#{key}' in retrieve_ssm_value"
+      nil
     rescue Aws::Errors::MissingCredentialsError
       raise ConfigResolverError, 'No AWS credentials available. Make sure the server has access to the aws-sdk'
     rescue StandardError => e

lib/uc3-ssm/version.rb

@@ -1,5 +1,5 @@
 # frozen_string_literal: true
 
 module Uc3Ssm
-  VERSION = '0.2.0'
+  VERSION = '0.3.0'
 end

spec/test/initialize_resolver_object_spec.rb

@@ -4,69 +4,88 @@
 require 'aws-sdk-ssm'
 
 # rubocop:disable Metrics/BlockLength
-RSpec.describe 'initialize_resolver_object_tests', type: :feature do
+RSpec.describe 'Test resolver object initialization. ', type: :feature do
 
-  context 'new instance creation' do
-    describe 'ConfigResolver.new with no options' do
+  context 'ConfigResolver.new' do
+    describe 'with no user provided options' do
       myResolver = Uc3Ssm::ConfigResolver.new
-      it 'sets @region to default' do
+      it 'sets @region to default.' do
         expect(myResolver.instance_variable_get(:@region)).to eq('us-west-2')
       end
-      it 'sets @ssm_root_path to default' do
-        expect(myResolver.instance_variable_get(:@ssm_root_path)).to eq('')
+      it 'sets @ssm_root_path to empty array.' do
+        expect(myResolver.instance_variable_get(:@ssm_root_path)).to eq([])
       end
-      it 'sets @def_value to default' do
+      it 'sets @def_value to empty string.' do
         expect(myResolver.instance_variable_get(:@def_value)).to eq('')
       end
-      it 'sets @ssm_skip_resolution to false' do
+      it 'sets @ssm_skip_resolution to false.' do
         expect(myResolver.instance_variable_get(:@ssm_skip_resolution)).to be false
       end
-      it 'sets @client to AWS SSM Client object' do
+      it 'sets @client to AWS SSM Client object.' do
         expect(myResolver.instance_variable_get(:@client)).to be_instance_of(Aws::SSM::Client)
       end
     end
 
-    describe 'ConfigResolver.new with options' do
+    describe 'with user provided options' do
       myResolver = Uc3Ssm::ConfigResolver.new(
         region: 'us-east-1',
         ssm_root_path: '/root/path/',
-        def_value: 'NOT_APPLICABLE'
+        def_value: 'NOT_APPLICABLE',
+        # see issue #10 - @ssm_skip_resolution only settable as ENV var
+        ssm_skip_resolution: true
       )
-      it 'sets @region' do
+      it 'sets @region.' do
         expect(myResolver.instance_variable_get(:@region)).to eq('us-east-1')
       end
-      it 'sets @ssm_root_path' do
-        expect(myResolver.instance_variable_get(:@ssm_root_path)).to eq('/root/path/')
+      it 'sets @ssm_root_path as an array.' do
+        expect(myResolver.instance_variable_get(:@ssm_root_path)).to eq(['/root/path/'])
       end
-      it 'sets @def_value' do
+      it 'sets @def_value.' do
         expect(myResolver.instance_variable_get(:@def_value)).to eq('NOT_APPLICABLE')
       end
+      # see issue #10 - @ssm_skip_resolution only settable as ENV var
+      #it 'sets @ssm_skip_resolution to true.' do
+      #  expect(myResolver.instance_variable_get(:@ssm_skip_resolution)).to be true
+      #end
     end
 
-    describe 'ConfigResolver.new with ENV vars' do
+    describe 'where ssm_root_path is list of colon separated paths' do
+      myResolver = Uc3Ssm::ConfigResolver.new(
+        ssm_root_path: '/root/path/:/no/trailing/slash',
+      )
+      it '@ssm_root_path is array with 2 paths.' do
+        expect(myResolver.instance_variable_get(:@ssm_root_path).length).to eq(2)
+      end
+      it 'appends trailing slash to each path in @ssm_root_path.' do
+        expect(myResolver.instance_variable_get(:@ssm_root_path)).to eq(['/root/path/', '/no/trailing/slash/'])
+      end
+    end
+
+    describe 'when ssm_root_path does not start with forward slash.' do
+      it 'raises exception.' do
+        expect {
+          Uc3Ssm::ConfigResolver.new(ssm_root_path: 'no/starting/slash/')
+        }.to raise_exception(Uc3Ssm::ConfigResolverError)
+      end
+    end
+
+    describe 'with options provided by ENV vars' do
       ENV['AWS_REGION'] = 'eu-east-3'
-      ENV['SSM_ROOT_PATH'] = '/root/path/no/trailing/slash'
+      ENV['SSM_ROOT_PATH'] = '/root/path'
       ENV['SSM_SKIP_RESOLUTION'] = 'Y'
       myResolver = Uc3Ssm::ConfigResolver.new
-      it 'sets @region' do
+      it 'sets @region.' do
         expect(myResolver.instance_variable_get(:@region)).to eq('eu-east-3')
       end
-      it '@ssm_root_path has trailing slash' do
-        expect(myResolver.instance_variable_get(:@ssm_root_path)).to eq('/root/path/no/trailing/slash/')
+      it 'sets @ssm_root_path.' do
+        expect(myResolver.instance_variable_get(:@ssm_root_path)).to eq(['/root/path/'])
       end
-      it 'sets @ssm_skip_resolution to true' do
+      it 'sets @ssm_skip_resolution.' do
         expect(myResolver.instance_variable_get(:@ssm_skip_resolution)).to be true
       end
-      it 'sets @client to AWS SSM Client object' do
+      it 'does not set @client because @ssm_skip_resolution is true.' do
         expect(myResolver.instance_variable_get(:@client)).to be nil
       end
     end
-
-    describe 'ConfigResolver.new with bad input' do
-      ENV['SSM_ROOT_PATH'] = 'no/starting/slash/'
-      it '@ssm_root_path raises exception' do
-        expect {badResolver = Uc3Ssm::ConfigResolver.new}.to raise_exception(Uc3Ssm::ConfigResolverError)
-      end
-    end
   end
 end