Skip to content
This repository has been archived by the owner on Oct 4, 2024. It is now read-only.

Doc edits #45

Open
wants to merge 47 commits into
base: doc-edits
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
47 commits
Select commit Hold shift + click to select a range
8af4a53
add templates for smartstore
Aug 14, 2019
c52e00f
added smartstore support
theckm May 6, 2020
31f41c1
Update splunk-enterprise-master-ss.template
May 7, 2020
672a208
Create user_data.sh
May 13, 2020
5e09070
Update user_data.sh
May 13, 2020
3b5875b
updated for further testing using user_data.sh
billbartlett May 20, 2020
a385884
changed timeout for CM wait condition
billbartlett May 20, 2020
3dc459f
decrease cm timeout condition to 15 minutes
billbartlett May 20, 2020
f00266d
changed timeout for CM to 10 minutes.
billbartlett May 21, 2020
efac84e
housekeeping
billbartlett May 21, 2020
1ee5b78
fixed location of the user_data.sh download. it was inadvertently lo…
billbartlett May 21, 2020
884087a
added time to ansible start script to see how long the installation t…
billbartlett May 22, 2020
92cb531
fixed some syntax issues and added echo statements to the functions.
billbartlett May 22, 2020
3b3dc28
add splunk cluster master to user_data.sh process
billbartlett May 27, 2020
01e6913
updated to add splunk cluster manager to user_data script
billbartlett Jun 8, 2020
c151a01
adding license by default for testing
billbartlett Jul 9, 2020
989e85c
add 0.0.0.0/0 as access CIDR by default for easier testing
billbartlett Jul 9, 2020
c92fe87
.
billbartlett Jul 9, 2020
9641ec4
moved all of the code that runs on every machine to a 'base' function.
billbartlett Jul 13, 2020
b91040e
changed location that splunk license installs to
billbartlett Jul 13, 2020
555c1ac
changed minimum indexer count to 4 instead of 3
billbartlett Jul 16, 2020
56cdbf7
migrated search head cluster logic to user_data.sh
billbartlett Jul 16, 2020
5ffa5bd
updated functionality for clustered search heads
billbartlett Jul 23, 2020
0913c16
migrated indexer cluster to user_data.sh
billbartlett Jul 23, 2020
489450d
added sleep to ensure splunkd is fully up before editing cluster config
billbartlett Jul 23, 2020
e441d9a
updated instance types
billbartlett Jul 23, 2020
201f4a4
clean up instance type selection, and add nvme drive setup code for i…
billbartlett Sep 14, 2020
7ec5120
Update user_data.sh
billbartlett Oct 13, 2020
1d06a41
initial commit of YAML conversion
billbartlett Oct 13, 2020
ba45368
initial adoc commit
Jan 22, 2021
a8c2975
removing/renaming old templates, updating some docs
Feb 12, 2021
c3793e4
updating docs and associated images
Feb 25, 2021
f4c4a60
cleanup and rename of templates
Feb 25, 2021
b396f1e
cleaning up docs
Mar 9, 2021
311ae3d
updated readme to highlight a splunk license is now required for quic…
Apr 20, 2021
e0e87ae
updated quickstart cloudformation to utilize smartstore, as well as
billbartlett Apr 4, 2022
898d9c8
fix default parameter values
billbartlett Apr 7, 2022
73248c8
fix default values
billbartlett Apr 26, 2022
a64b47a
remove everything code-related
billbartlett Apr 28, 2022
defa08d
move documentation to deployment_guide subfolder
billbartlett Apr 28, 2022
8175690
update adocs
billbartlett May 10, 2022
e459a9b
added _deployment_guide.adoc
billbartlett May 20, 2022
2f55069
added boilerplate
billbartlett May 20, 2022
d389f42
updated boilerplate submodule to point to aws-ia instead of
billbartlett May 20, 2022
01b66c4
remove boilerplate
billbartlett May 20, 2022
25e1883
re-add boilerplate
billbartlett May 20, 2022
a179031
updated some of the documentation to align with new formatting.
billbartlett May 20, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 3 additions & 4 deletions .gitmodules
Original file line number Diff line number Diff line change
@@ -1,4 +1,3 @@
[submodule "submodules/quickstart-aws-vpc"]
path = submodules/quickstart-aws-vpc
url = ../../aws-quickstart/quickstart-aws-vpc.git
branch = master
[submodule "docs/boilerplate"]
path = docs/boilerplate
url = https://github.com/aws-ia/aws-ia-documentation-base-common.git
74 changes: 37 additions & 37 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,37 +1,37 @@
# Splunk Enterprise on AWS - Quick Start

Source code associated with [Splunk Enterprise AWS Quick Start](https://fwd.aws/r7QNJ)

## Usage

Use these templates to deploy a highly available Splunk Enterprise environment across multiple AZs (2 or 3) in a given AWS region.
AZ-aware indexer clustering is enabled for horizontal scaling and to guarantee data is replicated in every AZ.
AZ-aware Search head clustering (3 nodes by default) can also be enabled for horizontal scaling and to guarantee data is available for search in every AZ.

View the accompanying [deployment guide](https://fwd.aws/bGBmy) for everything you need to get started. Refer to 'Deployment Steps' section for a step-by-step walkthrough on how to use these templates in AWS console.

### Prerequisites

Before getting started with the template configuration, you will need to make your Splunk Enterprise license privately accessible for CloudFormation template deployment via S3 download. The following steps will guide you through that process. *(Note: This step is not required, and you can upload your license from the Splunk web interface. It is, however, required that you have a non-trial Splunk Enterprise license to fully utilize the deployment our template creates. If you don't already have a Splunk Enterprise license, you can obtain one by contacting sales@splunk.com.)*

1. From the AWS Console, select "S3" under the "Storage" heading, or by simply typing "S3" into the search bar.
2. You can either select an existing private bucket to upload to, or create a new one. If you select an existing bucket, make sure its access policy does not grant public access. By default, all the S3 resources are private, so only the AWS account that created the resources can access them. For this exercise, I'm outlining how to create a new bucket.
3. Click "create bucket"
4. Name your bucket, and select your region. In this example, I will use "bbartlett-splunk-config". Your bucket name must be unique, and you should select the same region where you plan on deploying Splunk. <br><br> ![new bucket example](https://s3-us-west-2.amazonaws.com/splk-bbartlett/splunk_newbucket.png) <br><br>
5. Once you've created your bucket, select your new bucket from the list of buckets.
6. Click "Upload" on the upper left of the page
7. Click "Add Files"
8. Select your license file.
9. Click "Start Upload" on the lower right of the page.
10. Once the license has finished uploading, you'll need the bucket name and the filename to use with the CloudFormation template.

## License

This project is licensed under Apache License 2.0 - see [LICENSE.txt](./LICENSE.txt) file for details

## Help

If you have any problems or general questions, please file an issue in the parent repository:
https://github.com/aws-quickstart/quickstart-splunk-enterprise/issues


# Splunk Enterprise on AWS - Quick Start
Source code associated with [Splunk Enterprise AWS Quick Start](https://fwd.aws/r7QNJ)
## Usage
Use these templates to deploy a highly available Splunk Enterprise environment across multiple AZs (2 or 3) in a given AWS region.
AZ-aware indexer clustering is enabled for horizontal scaling and to guarantee data is replicated in every AZ.
AZ-aware Search head clustering (3 nodes by default) can also be enabled for horizontal scaling and to guarantee data is available for search in every AZ.
View the accompanying [deployment guide](https://fwd.aws/bGBmy) for everything you need to get started. Refer to 'Deployment Steps' section for a step-by-step walkthrough on how to use these templates in AWS console.
### Prerequisites
Before getting started with the template configuration, you will need to make your Splunk Enterprise license privately accessible for CloudFormation template deployment via S3 download. The following steps will guide you through that process. *(Note: This step is required. A non-trial Splunk Enterprise license is required to allow our template to configure the Splunk deployment. If you don't already have a Splunk Enterprise license, you can obtain one by contacting sales@splunk.com.)*
1. From the AWS Console, select "S3" under the "Storage" heading, or by simply typing "S3" into the search bar.
2. You can either select an existing private bucket to upload to, or create a new one. If you select an existing bucket, make sure its access policy does not grant public access. By default, all the S3 resources are private, so only the AWS account that created the resources can access them. For this exercise, I'm outlining how to create a new bucket.
3. Click "create bucket"
4. Name your bucket, and select your region. In this example, I will use "bbartlett-splunk-config". Your bucket name must be unique, and you should select the same region where you plan on deploying Splunk. <br><br> ![new bucket example](https://s3-us-west-2.amazonaws.com/splk-bbartlett/splunk_newbucket.png) <br><br>
5. Once you've created your bucket, select your new bucket from the list of buckets.
6. Click "Upload" on the upper left of the page
7. Click "Add Files"
8. Select your license file.
9. Click "Start Upload" on the lower right of the page.
10. Once the license has finished uploading, you'll need the bucket name and the filename to use with the CloudFormation template.
## License
This project is licensed under Apache License 2.0 - see [LICENSE.txt](./LICENSE.txt) file for details
## Help
If you have any problems or general questions, please file an issue in the parent repository:
https://github.com/aws-quickstart/quickstart-splunk-enterprise/issues
38 changes: 0 additions & 38 deletions ci/defaults.json

This file was deleted.

27 changes: 0 additions & 27 deletions ci/taskcat.yml

This file was deleted.

1 change: 1 addition & 0 deletions docs/_deployment_guide.adoc
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
:type: cfn
1 change: 1 addition & 0 deletions docs/boilerplate
Submodule boilerplate added at 4ef16b
Binary file added docs/deployment_guide/images/cfn_outputs.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
10 changes: 10 additions & 0 deletions docs/deployment_guide/partner_editable/_settings.adoc
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
:quickstart-project-name: quickstart-splunk-enterprise
:quickstart-github-org: aws-quickstart
:partner-product-short-name: Splunk Enterprise
:partner-company-name: Splunk Inc.
:doc-month: May
:doc-year: 2021
:partner-contributors: Bill Bartlett, {partner-company-name}
:aws-contributors: Shivansh Singh, Amazon Web Services
:deployment_time: 25 minutes
:default_deployment_region: us-east-1
35 changes: 35 additions & 0 deletions docs/deployment_guide/partner_editable/additional_info.adoc
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
// Add steps as necessary for accessing the software, post-configuration, and testing. Don’t include full usage instructions for your software, but add links to your product documentation for that information.
//Should any sections not be applicable, remove them

== Post deployment steps
// If steps are required to test the deployment, add them here. If not, remove the heading

After the Quick Start has successfully completed, you can log into your {partner-product-name} deployment from a web browser and verify configuration.

==== Verify Distributed Search
. Begin by logging into {partner-product-name} search head to verify all of the indexers are available for search. To log into the {partner-product-name} search head, navigate your browser to the URL shown in the CloudFormation Outputs labeled "SearchHeadURL" with the credentials of "admin" and the password configured with the "SplunkAdminPassword" parameter when launching the Quick Start.
. Navigate to Settings -> 'Distributed search' menu item as shown below

image:../images/search-head-distributed-search-menu.png[distributed_search_menu,width=850,height=294,link="../docs/images/search-head-distributed-search-menu.png"]
[start=3]
. Click on "Search peers"

. A screen similar to the screenshot below indicates that distributed search in good standing. (This example was created with a 4 node indexer cluster.)

image:../images/search-head-distributed-search-success.png[distributed_search_success,width=850,height=202,link="../docs/images/search-head-distributed-search-success.png"]

==== Verify Indexer Replication Status
. Begin by logging into {partner-product-name} cluster master to verify all of the indexers are successfully replicating buckets across the cluster. To log into the {partner-product-name} cluster master, navigate your browser to the URL shown in the CloudFormation Outputs labeled "ClusterMasterURL" with the credentials of "admin" and the password configured with the "SplunkAdminPassword" parameter when launching the Quick Start.
. Navigate to Settings -> 'Indexer clustering' menu item as shown below

image:../images/indexer-clustering-menu.png[indexer_clustering_menu,width=850,height=294,link="../docs/images/indexer-clustering-menu.png"]
[start=3]
. A screen similar to the screenshot below indicates that both the search factor and replication factor are in good standing. (This example was created with a 4 node indexer cluster across 2 AZ)
.. _Please note that it will likely take a few minutes after the Quick Start has successfully launched before the buckets are replicated and this status window shows both search factor and replication factor being met. If there are errors for replication and/or search factor after 10 minutes, please see the link:#_faq[FAQ section] below._

image:../images/cluster-master-sfrf-met.png[indexer_clustering_success,width=850,height=202,link="../docs/images/cluster-master-sfrf-met.png"]

== Security
// Provide post-deployment best practices for using the technology on AWS, including considerations such as migrating data, backups, ensuring high performance, high availability, etc. Link to software documentation for detailed information.

The {partner-product-name} Quick Start exposes three user-configurable security group access parameters: 'WebClientLocation', 'HECClientLocation', and 'SSHClientLocation'. Be sure that the 'SSHClientLocation' parameter is accessible only on tightly controlled authorized network ranges as this allows direct access to the instances. The parameter 'WebClientLocation' allows connections to the {partner-product-name} web interfaces, while 'HECClientLocation' controls access to the load balancer in front of the {partner-product-name} HTTP Event Collector listener.
25 changes: 25 additions & 0 deletions docs/deployment_guide/partner_editable/architecture.adoc
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
:xrefstyle: short

Deploying this Quick Start for a new VPC with default parameters builds the following {partner-product-name} environment in the AWS Cloud.
[#architecture1]
.Quick Start architecture for {partner-product-name} on AWS
[link=images/splunk-enterprise-architecture-on-aws.png]
image::../images/splunk-enterprise-architecture-on-aws.png[Architecture,width=648,height=439]

As shown in <<architecture1>>, the Quick Start sets up the following:

* A VPC configured across two or three Availability Zones, depending on your selection. The Quick Start provisions one public subnet in each Availability Zone.
* Two Elastic Load Balancing (ELB) load balancers: one to load-balance HTTP web traffic to the search head instances, and the other to load-balance HTTP event traffic destined for the Splunk HTTP Event Collector (HEC) across all indexer instances.
* An IAM user with fine-grained permissions for access to AWS services necessary for the initial deployment process.
* Appropriate security groups for each instance or function to restrict access to only necessary protocols and ports.
* Amazon Simple Storage Service (Amazon S3) bucket for Splunk SmartStore usage.
* In the public subnets, EC2 instances for {partner-product-name}, including the following:
** {partner-product-name} indexer cluster with the number of indexers you specify (3-10), distributed across the number of Availability Zones you specify (2 or 3). The Splunk receiver (splunktcp) and Splunk HEC are enabled across all indexers.
** Splunk search head(s), either stand-alone or in a 3-node cluster, based on your input during deployment. In the latter case, the search heads are distributed across the number of Availability Zones you specify.
** Splunk license server and indexer cluster master, co-located.
** Splunk search head deployer, where applicable.


If you decide to deploy Splunk Enterprise into your existing VPC, please see link:#_deployment_options[deployment options] later in this guide. The Quick Start assumes that the infrastructure components already exist, and deploys Splunk Enterprise into the environment you specify.


11 changes: 11 additions & 0 deletions docs/deployment_guide/partner_editable/deployment_options.adoc
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
// There are generally two deployment options. If additional are required, add them here

This Quick Start provides two deployment options:

* *Deploy {partner-product-name} into a new VPC (end-to-end deployment)*. This option builds a new AWS environment consisting of the VPC, subnets, security groups, load balancers, and other infrastructure components. It then deploys {partner-product-name} into this new VPC.

* *Deploy {partner-product-name} into an existing VPC*. This option provisions {partner-product-name} in your existing AWS infrastructure.

The Quick Start provides separate templates for these options. It also lets you configure Classless Inter-Domain Routing (CIDR) blocks, instance types, and {partner-product-name} settings, as discussed later in this guide.

For further information about the dimensions of a Splunk Enterprise deployment, see the {partner-product-name} https://docs.splunk.com/Documentation/Splunk/latest/Capacity/DimensionsofaSplunkEnterprisedeployment[capacity planning manual^]. Finally, for additional information specific to an AWS deployment, see {partner-product-name} https://www.splunk.com/pdfs/technical-briefs/deploying-splunk-enterprise-on-amazon-web-services-technical-brief.pdf[AWS tech brief^] for instance sizing considerations.
1 change: 1 addition & 0 deletions docs/deployment_guide/partner_editable/licenses.adoc
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
This Quick Start requires a subscription to the Amazon Machine Image (AMI) for {partner-product-name}, which is available from AWS Marketplace. For subscription instructions, see step 2 under "Subscribe to the {partner-product-name} AMI" in the link:#_deployment_steps[deployment steps]. In addition, to fully utilize the Quick Start environment, a {partner-product-name} license is required. If you do not have one, please contact sales@splunk.com
3 changes: 3 additions & 0 deletions docs/deployment_guide/partner_editable/overview.adoc
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
This Quick Start provides architectural guidance and step-by-step instructions for a high availability deployment of {partner-product-name} on AWS. The guide addresses common scalability, high-availability, and security considerations for your deployment.

This guide is intended for a variety of audiences, including IT infrastructure architects, administrators, and DevOps professionals who are planning to implement or extend their Splunk Enterprise deployments on the AWS Cloud. It also provides direct links for viewing and launching AWS CloudFormation templates that assist with automating the deployment
9 changes: 9 additions & 0 deletions docs/deployment_guide/partner_editable/pre-reqs.adoc
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
// If no preperation is required, remove all content from here.
There are two important steps to verify prior to launching this Quick start:

* Ensure that the SmartStore bucket that is defined in the "SmartStoreBucketName" parameter *does not exist*. This Quick Start will attempt to create that bucket with an appropriate bucket policy. If that bucket already exists, the Quick Start will fail.
* Upload a valid Splunk license to an S3 bucket owned by the user launching the Quick Start. Take note of the bucket name as well as the path to the license file. An example may look something like this:
** Splunk license file is named "splunk.license" and uploaded to an S3 bucket called "my-s3-bucket" under the 'directory' called "license"
*** Parameter "SplunkLicenseBucket" should be configured to: my-s3-bucket
*** Parameter "SplunkLicensePath" should be configured to: license/splunk.license (note the lack of a leading "/" on the license path)

Loading