Credhub interpolation support in CF app manifests

[gberche-orange]

Is your feature request related to a problem? Please describe.

• As a template author
• In order to use credhub-generated passwords in Cf apps pushed by the Cf app support
• I need COA to invoke the credhub interpolate command on the manifest.yml (following the spruce processing)

Describe the solution you'd like

• given credhub is populated with a secret

$ credhub get -n /bosh-master/coa-cf-mysql-broker/broker-auth
id: 4846e5b3-...
name: /bosh-master/coa-cf-mysql-broker/broker-auth
type: user
value:
  password: randonuserpassword
  password_hash: redacted
  username: randonusername
version_created_at: "2019-09-25T09:26:02Z"

• given a coab-depls/cf-apps-deployments/coa-cf-mysql-broker/template/coa-cf-mysql-broker_manifest-tpl.yml file with content

[...]
# interpolated by spruce using vars file
     pipeline.maxExecutionDurationSeconds: (( grab secrets.coa-cf-mysql-broker.pipeline.maxExecutionDurationSeconds ))
# interpolated by credhub, in this example a credential of type user with fields 
     spring.security.user.name: "(( /coa-cf-mysql-broker.broker-auth.username))"
     spring.security.user.password: "(( /coa-cf-mysql-broker.broker-auth.password))"

• given a coab-depls/cf-apps-deployments/coa-cf-mysql-broker/secrets/secrets.yml file with content

    pipeline:
      maxExecutionDurationSeconds: 3600

• then the resulting coa-cf-mysql-broker_manifest.yml file content is

[...]
# interpolated by spruce using vars file
     pipeline.maxExecutionDurationSeconds: 3600
# interpolated by credhub, in this example a credential of type user with fields 
     spring.security.user.name: "randonusername"
     spring.security.user.password: "randonuserpassword"

Note that this proposal currently conflicts with CF CLI interpolation proposal at #188

Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.

Additional context

See https://github.com/cloudfoundry-incubator/credhub-cli

$ credhub interpolate --help
Usage:
  credhub [OPTIONS] interpolate [interpolate-OPTIONS]

Fill a template with values returned from CredHub.

Uses double-paren placeholders in the style of the bosh cli. Example:

---
something-stored-in-credhub: ((path/to/var))
something-else: static value

In the above example, the whole value of the cred will be inserted.
For instance, if path/to/var is of type ssh, the output will have all the credential's fields, like this:

---
something-stored-in-credhub:
private_key: fake-private-key
public_key: fake-public-key
public_key_fingerprint: fake-fingerprint
some-other-key: static value

If you want just the password value, you'd need to use ((path/to/var.public_key)),
which would only have the specified field, like this:

---
something-stored-in-credhub: fake-public-key
something-else: static value

If the prefix flag is provided, the given prefix will be prepended
to any credentials that do not start with the '/' character.
Example:

---
something: ((/env-specific-path/path/to/var))
same-thing: ((path/to/var))

When this example is used with the prefix flag 'env-specific-path', they will be evaluated to the same thing.

Application Options:
      --version           Version of CLI and targeted CredHub API
      --token             Return your current CredHub authentication token

Help Options:
  -h, --help              Show this help message

[interpolate command options]
      -f, --file=         Path to the file to interpolate
      -p, --prefix=       Prefix to be applied to credential paths. Will not be applied to paths that start with '/'
      -s, --skip-missing  allow skipping missing params

[gberche-orange]

Alternative options for creating credhub credentials for a COA-managed CF deployment:

• support a 1st class credhub-vars.yml file matching the bosh variables format such as the following and which COA would invoke using Credhub CLI
  • given

variables:
- name: admin_password
  type: password
- name: postgres_password
  type: password
- name: default_ca
  type: certificate
  options:
    is_ca: true
    common_name: bosh-ca
- name: director_ssl
  type: certificate
  options:
    ca: default_ca
    common_name: ((internal_ip))
    alternative_names: [((internal_ip))]

• then a credhub cli command similar to the following is made (this call is idempotent, i.e. can be safely executed multiple times, it has the safe side effect and returns the same response)

$ credhub curl -X POST -p /api/v1/data -d='{"name": "/some-password-name","type": "password"}'
{
  "id": "5d190413-392b-47c9-a0e1-a5240a1ac757",
  "name": "/some-password-name",
  "type": "password",
  "value": "l63zFn1h56PRg6ZWxaXuEDQZQLjwho",
  "version_created_at": "2019-10-16T12:10:46Z"
}

• and a credhub fingerprint (for the created credential) is saved for operators inspection in git
• pre-cf-push.sh invokes credhub cli to generate the credentials
  • prereq to refine the exported environment variables to support credhub login
• wait for a CF CLI plugin/1st class support for credhub

$ credhub login --help
Usage:
  credhub [OPTIONS] login [login-OPTIONS]

Authenticate with CredHub. UAA password and client credential grants are supported. If client credentials exist in the environment, authentication will be performed automatically without the need to
explicitly call this command.

Application Options:
      --version                  Version of CLI and targeted CredHub API
      --token                    Return your current CredHub authentication token

Help Options:
  -h, --help                     Show this help message

[login command options]
      -u, --username=            Authentication username
      -p, --password=            Authentication password
          --client-name=         Client name for UAA client grant [$CREDHUB_CLIENT]
          --client-secret=       Client secret for UAA client grant [$CREDHUB_SECRET]
      -s, --server=              URI of API server to target [$CREDHUB_SERVER]
          --ca-cert=             Trusted CA for API and UAA TLS connections [$CREDHUB_CA_CERT]
          --skip-tls-validation  Skip certificate validation of the API endpoint. Not recommended!
          --sso                  Prompt for a one-time passcode to login
          --sso-passcode=        One-time passcode

$ credhub api --help  
Usage:
  credhub [OPTIONS] api [api-OPTIONS] [SERVER]

Get or set the CredHub API target where commands are sent. The api command without any flags will return the current target. If --ca-cert or --skip-tls-validation are provided, these preferences will be
cached for future requests.

Application Options:
      --version                  Version of CLI and targeted CredHub API
      --token                    Return your current CredHub authentication token

Help Options:
  -h, --help                     Show this help message

[api command options]
      -s, --server=              URI of API server to target [$CREDHUB_SERVER]
          --ca-cert=             Trusted CA for API and UAA TLS connections. Multiple flags may be provided. [$CREDHUB_CA_CERT]
          --skip-tls-validation  Skip certificate validation of the API endpoint. Not recommended!

[api command arguments]
  SERVER:                        URI of API server to target



$ credhub generate --help
Usage:
  credhub [OPTIONS] generate [generate-OPTIONS]

Set a credential with generated value(s). A type must be specified when generating a credential. The provided flags are used to set parameters for the credential that is generated, e.g. a certificate
credential may use --common-name, --duration and --self-sign to generate an appropriate value. Supported credential types are prefixed in the flag description.

Application Options:
      --version                Version of CLI and targeted CredHub API
      --token                  Return your current CredHub authentication token

Help Options:
  -h, --help                   Show this help message

[generate command options]
      -n, --name=              Name of the credential to generate
      -t, --type=              Sets the credential type to generate. Valid types include 'password', 'user', 'certificate', 'ssh' and 'rsa'.
      -O, --no-overwrite       Credential is not modified if stored value already exists
      -j, --output-json        Return response in JSON format
      -z, --username=          [User] Sets the username value of the credential
      -l, --length=            [Password, User] Length of the generated value (Default: 30)
      -S, --include-special    [Password, User] Include special characters in the generated value
      -N, --exclude-number     [Password, User] Exclude number characters from the generated value
      -U, --exclude-upper      [Password, User] Exclude upper alpha characters from the generated value
      -L, --exclude-lower      [Password, User] Exclude lower alpha characters from the generated value
      -m, --ssh-comment=       [SSH] Comment appended to public key to help identify in environment
      -k, --key-length=        [Certificate, SSH, RSA] Bit length of the generated key (Default: 2048)
      -d, --duration=          [Certificate] Valid duration (in days) of the generated certificate (Default: 365)
      -c, --common-name=       [Certificate] Common name of the generated certificate
      -o, --organization=      [Certificate] Organization of the generated certificate
      -u, --organization-unit= [Certificate] Organization unit of the generated certificate
      -i, --locality=          [Certificate] Locality/city of the generated certificate
      -s, --state=             [Certificate] State/province of the generated certificate
      -y, --country=           [Certificate] Country of the generated certificate
      -a, --alternative-name=  [Certificate] A subject alternative name of the generated certificate (may be specified multiple times)
      -g, --key-usage=         [Certificate] Key Usage extensions for the generated certificate (may be specified multiple times)
      -e, --ext-key-usage=     [Certificate] Extended Key Usage extensions for the generated certificate (may be specified multiple times)
          --ca=                [Certificate] Name of CA used to sign the generated certificate
          --is-ca              [Certificate] The generated certificate is a certificate authority
          --self-sign          [Certificate] The generated certificate will be self-signed