Updated Splunk Quick Start #44
Conversation
initial commit of user_data.sh -- testing viability of using external shell scripts for user data instead of doing everything in-line with the template.
missed updating the creationpolicy timeout on the previous commit. fixing that, and lowering the timeout to 10 minutes instead of 15.
10 minutes was not enough for the CM to boot before timeout. changed back to 15. changed systemctl commands to use splunk binary moved variables up in user data removed the cloud init files for splunk ansible before ansible is ran
…ked inside the if statement
without something in the functions, bash will error out on a syntax error. added echo statements as placeholders.
Prior to this change, when the script was ran without any arguments, it would still try to execute the base install of Splunk. This change was made so that if someone executes the script to see usage, it won't try to execute the install before showing the usage.
previously, it wrote to /etc/splunk/splunk.license and then the user_data script moved it. that seems unnecessary, so I'm having cloudformation write directly to the appropriate directory.
the minimum number of indexers in an AZ must be at least as large as the replication factor. since we're using RF = 2, and a default minimum of 2 AZ, there must be at least 4 indexers in total - 2 in each AZ.
added sleep in the deployer after restart. this allows for splunk to be fully restarted before issuing splunk cli commands that would fail because the REST endpoint wasn't yet responding.
converted the JSON templates to YAML, which is now required for the quickstart repo.
unfinished docs, but commiting so that they may be shared.
|
@sshvans - the .metadata file has been added and PR updated. |
|
/do-e2e-tests |
|
I'm creating a backend CI pipeline for this project. This is a one-time procedure but will add a delay of approximately 5 minutes. |
|
/do-e2e-tests |
|
@billbartlett Seems like there are still merge conflicts. |
This reverts commit ba54d82.
… .yaml, the corresponding template URL was not updated.
successfully tested taskcat on us-west-2 with these settings, where bracketed sample values were substituted with real data.
|
/do-e2e-tests |
.taskcat.yml
Outdated
| WebClientLocation: 72.21.196.66/32 | ||
| AvailabilityZones: us-west-2a,us-west-2b | ||
| KeyName: [your-aws-keypair] | ||
| QSS3BucketName: [quickstart-bucket-name] |
There was a problem hiding this comment.
the QSS3BucketName will need to be a bucket that contains the quickstart code. isn't $[taskcat_autobucket] used for random bucket names?
There was a problem hiding this comment.
For testing, $[taskcat_autobucket] is used by CI to create a temp bucket, upload the repo content and test from that bucket. In the template, the parameter default value needs to be quickstart bucket name, as thats where the templates are staged for customers to deploy.
| SplunkLicenseBucket: [splunk-license-bucket-name] | ||
| SplunkLicensePath: [splunk-license-file] |
There was a problem hiding this comment.
Do we need splunk license file for the testing? If yes, can you provide a test file via email, for our CI, so that our CI platform can test the QS automatically?
|
@billbartlett Let me know when you have resolved the comments above and I'll give it a test run. |
…ack in the template
from wget with a public bucket requirement to authenticated s3 via metadata & cfn-init also updated EBS volume types from gp2 to gp3
| S3AccessCreds: | ||
| type: S3 | ||
| accessKeyId: !Ref 'CfnKeys' | ||
| secretKey: !GetAtt 'CfnKeys.SecretAccessKey' | ||
| buckets: | ||
| - !Ref 'QSS3BucketName' |
There was a problem hiding this comment.
Why not use IAM role? Instead of creating user and using secret access key? That way you don't need to worry about rotating keys. Its much secure.
Issue #, if available:
Description of changes: There are a multitude of changes, and the primary driver for updating is to include Splunk's SmartStore functionality.
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.