Skip to content
This repository has been archived by the owner on Oct 4, 2024. It is now read-only.

Doc edits #45

Open
wants to merge 47 commits into
base: doc-edits
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 40 commits
Commits
Show all changes
47 commits
Select commit Hold shift + click to select a range
8af4a53
add templates for smartstore
Aug 14, 2019
c52e00f
added smartstore support
theckm May 6, 2020
31f41c1
Update splunk-enterprise-master-ss.template
May 7, 2020
672a208
Create user_data.sh
May 13, 2020
5e09070
Update user_data.sh
May 13, 2020
3b5875b
updated for further testing using user_data.sh
billbartlett May 20, 2020
a385884
changed timeout for CM wait condition
billbartlett May 20, 2020
3dc459f
decrease cm timeout condition to 15 minutes
billbartlett May 20, 2020
f00266d
changed timeout for CM to 10 minutes.
billbartlett May 21, 2020
efac84e
housekeeping
billbartlett May 21, 2020
1ee5b78
fixed location of the user_data.sh download. it was inadvertently lo…
billbartlett May 21, 2020
884087a
added time to ansible start script to see how long the installation t…
billbartlett May 22, 2020
92cb531
fixed some syntax issues and added echo statements to the functions.
billbartlett May 22, 2020
3b3dc28
add splunk cluster master to user_data.sh process
billbartlett May 27, 2020
01e6913
updated to add splunk cluster manager to user_data script
billbartlett Jun 8, 2020
c151a01
adding license by default for testing
billbartlett Jul 9, 2020
989e85c
add 0.0.0.0/0 as access CIDR by default for easier testing
billbartlett Jul 9, 2020
c92fe87
.
billbartlett Jul 9, 2020
9641ec4
moved all of the code that runs on every machine to a 'base' function.
billbartlett Jul 13, 2020
b91040e
changed location that splunk license installs to
billbartlett Jul 13, 2020
555c1ac
changed minimum indexer count to 4 instead of 3
billbartlett Jul 16, 2020
56cdbf7
migrated search head cluster logic to user_data.sh
billbartlett Jul 16, 2020
5ffa5bd
updated functionality for clustered search heads
billbartlett Jul 23, 2020
0913c16
migrated indexer cluster to user_data.sh
billbartlett Jul 23, 2020
489450d
added sleep to ensure splunkd is fully up before editing cluster config
billbartlett Jul 23, 2020
e441d9a
updated instance types
billbartlett Jul 23, 2020
201f4a4
clean up instance type selection, and add nvme drive setup code for i…
billbartlett Sep 14, 2020
7ec5120
Update user_data.sh
billbartlett Oct 13, 2020
1d06a41
initial commit of YAML conversion
billbartlett Oct 13, 2020
ba45368
initial adoc commit
Jan 22, 2021
a8c2975
removing/renaming old templates, updating some docs
Feb 12, 2021
c3793e4
updating docs and associated images
Feb 25, 2021
f4c4a60
cleanup and rename of templates
Feb 25, 2021
b396f1e
cleaning up docs
Mar 9, 2021
311ae3d
updated readme to highlight a splunk license is now required for quic…
Apr 20, 2021
e0e87ae
updated quickstart cloudformation to utilize smartstore, as well as
billbartlett Apr 4, 2022
898d9c8
fix default parameter values
billbartlett Apr 7, 2022
73248c8
fix default values
billbartlett Apr 26, 2022
a64b47a
remove everything code-related
billbartlett Apr 28, 2022
defa08d
move documentation to deployment_guide subfolder
billbartlett Apr 28, 2022
8175690
update adocs
billbartlett May 10, 2022
e459a9b
added _deployment_guide.adoc
billbartlett May 20, 2022
2f55069
added boilerplate
billbartlett May 20, 2022
d389f42
updated boilerplate submodule to point to aws-ia instead of
billbartlett May 20, 2022
01b66c4
remove boilerplate
billbartlett May 20, 2022
25e1883
re-add boilerplate
billbartlett May 20, 2022
a179031
updated some of the documentation to align with new formatting.
billbartlett May 20, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ View the accompanying [deployment guide](https://fwd.aws/bGBmy) for everything y

### Prerequisites

Before getting started with the template configuration, you will need to make your Splunk Enterprise license privately accessible for CloudFormation template deployment via S3 download. The following steps will guide you through that process. *(Note: This step is not required, and you can upload your license from the Splunk web interface. It is, however, required that you have a non-trial Splunk Enterprise license to fully utilize the deployment our template creates. If you don't already have a Splunk Enterprise license, you can obtain one by contacting sales@splunk.com.)*
Before getting started with the template configuration, you will need to make your Splunk Enterprise license privately accessible for CloudFormation template deployment via S3 download. The following steps will guide you through that process. *(Note: This step is required. A non-trial Splunk Enterprise license is required to allow our template to configure the Splunk deployment. If you don't already have a Splunk Enterprise license, you can obtain one by contacting sales@splunk.com.)*

1. From the AWS Console, select "S3" under the "Storage" heading, or by simply typing "S3" into the search bar.
2. You can either select an existing private bucket to upload to, or create a new one. If you select an existing bucket, make sure its access policy does not grant public access. By default, all the S3 resources are private, so only the AWS account that created the resources can access them. For this exercise, I'm outlining how to create a new bucket.
Expand Down
38 changes: 0 additions & 38 deletions ci/defaults.json

This file was deleted.

27 changes: 0 additions & 27 deletions ci/taskcat.yml

This file was deleted.

Binary file added docs/deployment_guide/images/cfn_outputs.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
14 changes: 14 additions & 0 deletions docs/deployment_guide/partner_editable/_settings.adoc
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
:quickstart-project-name: quickstart-splunk-enterprise
:partner-product-name: Splunk Enterprise
:partner-company-name: Splunk Inc.
:doc-month: March
:doc-year: 2021
:partner-contributors: Bill Bartlett, {partner-company-name}
:quickstart-contributors: Shivansh Singh, Amazon Web Services
:deployment_time: 25 minutes
:default_deployment_region: us-east-1
// Uncomment these two attributes if you are leveraging
// - an AWS Marketplace listing.
// Additional content will be auto-generated based on these attributes.
:marketplace_subscription:
:marketplace_listing_url: https://aws.amazon.com/marketplace/pp/Splunk-Splunk-Enterprise/B00PUXWXNE
35 changes: 35 additions & 0 deletions docs/deployment_guide/partner_editable/additional_info.adoc
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
// Add steps as necessary for accessing the software, post-configuration, and testing. Don’t include full usage instructions for your software, but add links to your product documentation for that information.
//Should any sections not be applicable, remove them

== Post deployment steps
// If steps are required to test the deployment, add them here. If not, remove the heading

After the Quick Start has successfully completed, you can log into your {partner-product-name} deployment from a web browser and verify configuration.

==== Verify Distributed Search
. Begin by logging into {partner-product-name} search head to verify all of the indexers are available for search. To log into the {partner-product-name} search head, navigate your browser to the URL shown in the CloudFormation Outputs labeled "SearchHeadURL" with the credentials of "admin" and the password configured with the "SplunkAdminPassword" parameter when launching the Quick Start.
. Navigate to Settings -> 'Distributed search' menu item as shown below

image:../images/search-head-distributed-search-menu.png[distributed_search_menu,width=850,height=294,link="../docs/images/search-head-distributed-search-menu.png"]
[start=3]
. Click on "Search peers"

. A screen similar to the screenshot below indicates that distributed search in good standing. (This example was created with a 4 node indexer cluster.)

image:../images/search-head-distributed-search-success.png[distributed_search_success,width=850,height=202,link="../docs/images/search-head-distributed-search-success.png"]

==== Verify Indexer Replication Status
. Begin by logging into {partner-product-name} cluster master to verify all of the indexers are successfully replicating buckets across the cluster. To log into the {partner-product-name} cluster master, navigate your browser to the URL shown in the CloudFormation Outputs labeled "ClusterMasterURL" with the credentials of "admin" and the password configured with the "SplunkAdminPassword" parameter when launching the Quick Start.
. Navigate to Settings -> 'Indexer clustering' menu item as shown below

image:../images/indexer-clustering-menu.png[indexer_clustering_menu,width=850,height=294,link="../docs/images/indexer-clustering-menu.png"]
[start=3]
. A screen similar to the screenshot below indicates that both the search factor and replication factor are in good standing. (This example was created with a 4 node indexer cluster across 2 AZ)
.. _Please note that it will likely take a few minutes after the Quick Start has successfully launched before the buckets are replicated and this status window shows both search factor and replication factor being met. If there are errors for replication and/or search factor after 10 minutes, please see the link:#_faq[FAQ section] below._

image:../images/cluster-master-sfrf-met.png[indexer_clustering_success,width=850,height=202,link="../docs/images/cluster-master-sfrf-met.png"]

== Security
// Provide post-deployment best practices for using the technology on AWS, including considerations such as migrating data, backups, ensuring high performance, high availability, etc. Link to software documentation for detailed information.

The {partner-product-name} Quick Start exposes three user-configurable security group access parameters: 'WebClientLocation', 'HECClientLocation', and 'SSHClientLocation'. Be sure that the 'SSHClientLocation' parameter is accessible only on tightly controlled authorized network ranges as this allows direct access to the instances. The parameter 'WebClientLocation' allows connections to the {partner-product-name} web interfaces, while 'HECClientLocation' controls access to the load balancer in front of the {partner-product-name} HTTP Event Collector listener.
23 changes: 23 additions & 0 deletions docs/deployment_guide/partner_editable/architecture.adoc
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
Deploying this Quick Start for a new VPC with default parameters builds the following {partner-product-name} environment in the AWS Cloud.
[#architecture1]
.Quick Start architecture for {partner-product-name} on AWS
[link=images/splunk-enterprise-architecture-on-aws.png]
image::../images/splunk-enterprise-architecture-on-aws.png[Architecture,width=648,height=439]

As shown in figure 1, the Quick Start sets up the following:

* A VPC configured across two or three Availability Zones, depending on your selection. The Quick Start provisions one public subnet in each Availability Zone.
* Two Elastic Load Balancing (ELB) load balancers: one to load-balance HTTP web traffic to the search head instances, and the other to load-balance HTTP event traffic destined for the Splunk HTTP Event Collector (HEC) across all indexer instances.
* An IAM user with fine-grained permissions for access to AWS services necessary for the initial deployment process.
* Appropriate security groups for each instance or function to restrict access to only necessary protocols and ports.
* Amazon Simple Storage Service (Amazon S3) bucket for Splunk SmartStore usage.
* In the public subnets, EC2 instances for {partner-product-name}, including the following:
** {partner-product-name} indexer cluster with the number of indexers you specify (3-10), distributed across the number of Availability Zones you specify (2 or 3). The Splunk receiver (splunktcp) and Splunk HEC are enabled across all indexers.
** Splunk search head(s), either stand-alone or in a 3-node cluster, based on your input during deployment. In the latter case, the search heads are distributed across the number of Availability Zones you specify.
** Splunk license server and indexer cluster master, co-located.
** Splunk search head deployer, where applicable.


If you decide to deploy Splunk Enterprise into your existing VPC, please see link:#_deployment_options[deployment options] later in this guide. The Quick Start assumes that the infrastructure components already exist, and deploys Splunk Enterprise into the environment you specify.


49 changes: 49 additions & 0 deletions docs/deployment_guide/partner_editable/deploy_steps.adoc
Original file line number Diff line number Diff line change
@@ -0,0 +1,49 @@
// We need to work around Step numbers here if we are going to potentially exclude the AMI subscription
=== Sign in to your AWS account

. Sign in to your AWS account at https://aws.amazon.com with an AWS Identity and Access Management (IAM) user role that has the necessary permissions. For details, see link:#_planning_the_deployment[Planning the deployment] earlier in this guide.
. Make sure that your AWS account is configured correctly, as discussed in the link:#_technical_requirements[Technical requirements] section.

// Optional based on Marketplace listing. Not to be edited
ifdef::marketplace_subscription[]
=== Subscribe to the {partner-product-name} AMI

This Quick Start requires a subscription to the Amazon Machine Image (AMI) for {partner-product-name} in AWS Marketplace.

. Sign in to your AWS account.
. {marketplace_listing_url}[Open the page for the {partner-product-name} AMI in AWS Marketplace], and then choose *Continue to Subscribe*.
. Review the terms and conditions for software usage, and then choose *Accept Terms*. +
A confirmation page loads, and an email confirmation is sent to the account owner. For detailed subscription instructions, see the https://aws.amazon.com/marketplace/help/200799470[AWS Marketplace documentation^].

. When the subscription process is complete, close AWS Marketplace without further action. *Do not* provision the software from AWS Marketplace, as the Quick Start deploys the AMI for you.
endif::marketplace_subscription[]
// \Not to be edited

=== Launch the Quick Start

NOTE: You are responsible for the cost of the AWS services used while running this Quick Start reference deployment. There is no additional cost for using this Quick Start. For full details, see the pricing pages for each AWS service used by this Quick Start. Prices are subject to change.

. Sign in to your AWS account, and choose one of the following options to launch the AWS CloudFormation template. For help with choosing an option, see link:#_deployment_options[deployment options] earlier in this guide.

[cols=",]
|===
|https://fwd.aws/MNRVe[Deploy {partner-product-name} into a new VPC on AWS^]
|https://fwd.aws/DD3gQ[Deploy {partner-product-name} into an existing VPC on AWS^]
|===

WARNING: If you’re deploying {partner-product-name} into an existing VPC, make sure that your VPC has at least two subnets in different Availability Zones for the indexers and search head(s), and that the subnets aren’t shared. If you choose to deploy into three AZ, then your VPC will require at least three available AZ with three separate subnets. This Quick Start doesn’t support https://docs.aws.amazon.com/vpc/latest/userguide/vpc-sharing.html[shared subnets^].

Each deployment takes about {deployment_time} to complete.

[start=2]
. Check the AWS Region that’s displayed in the upper-right corner of the navigation bar, and change it if necessary. This Region is where {partner-product-name} will be deployed. The template is launched in the {default_deployment_region} Region by default.

[start=3]
. On the *Create stack* page, keep the default setting for the template URL, and then choose *Next*.
. On the *Specify stack details* page, change the stack name if needed. Review the parameters for the template. Provide values for the parameters that require input. For all other parameters, review the default settings and customize them as necessary.

// In the following tables, parameters are listed by category and described separately for the two deployment options:

// * Parameters for deploying {partner-product-name} into a new VPC

// * Parameters for deploying {partner-product-name} into an existing VPC
11 changes: 11 additions & 0 deletions docs/deployment_guide/partner_editable/deployment_options.adoc
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
// There are generally two deployment options. If additional are required, add them here

This Quick Start provides two deployment options:

* *Deploy {partner-product-name} into a new VPC (end-to-end deployment)*. This option builds a new AWS environment consisting of the VPC, subnets, security groups, load balancers, and other infrastructure components. It then deploys {partner-product-name} into this new VPC.

* *Deploy {partner-product-name} into an existing VPC*. This option provisions {partner-product-name} in your existing AWS infrastructure.

The Quick Start provides separate templates for these options. It also lets you configure Classless Inter-Domain Routing (CIDR) blocks, instance types, and {partner-product-name} settings, as discussed later in this guide.

For further information about the dimensions of a Splunk Enterprise deployment, see the {partner-product-name} https://docs.splunk.com/Documentation/Splunk/latest/Capacity/DimensionsofaSplunkEnterprisedeployment[capacity planning manual^]. Finally, for additional information specific to an AWS deployment, see {partner-product-name} https://www.splunk.com/pdfs/technical-briefs/deploying-splunk-enterprise-on-amazon-web-services-technical-brief.pdf[AWS tech brief^] for instance sizing considerations.
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
// Add any tips or answers to anticipated questions. This could include the following troubleshooting information. If you don’t have any other Q&A to add, change “FAQ” to “Troubleshooting.”

== FAQ
*Q.* Why is my search factor and/or replication factor not being met?

*A.* The most common reason for this is the cluster replication or search factor is set higher than the minimum indexers in a site. For example, if you create a 5 node cluster across two sites, you will have 3 indexers in site1, and 2 indexers in site2. In this example, if you also configured RF and/or SF = 3, Splunk will not be able to meet the required replication or search factor. (RF or SF = 3, but you only have 2 indexers in site2) For further reading, this topic is outlined in https://docs.splunk.com/Documentation/Splunk/8.1.2/Indexer/Bucketreplicationissues#Multisite_cluster_does_not_meet_its_replication_or_search_factors[Splunk documentation^].
1 change: 1 addition & 0 deletions docs/deployment_guide/partner_editable/licenses.adoc
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
This Quick Start requires a subscription to the Amazon Machine Image (AMI) for {partner-product-name}, which is available from AWS Marketplace. For subscription instructions, see step 2 under "Subscribe to the {partner-product-name} AMI" in the link:#_deployment_steps[deployment steps]. In addition, to fully utilize the Quick Start environment, a {partner-product-name} license is required. If you do not have one, please contact sales@splunk.com
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
This Quick Start provides architectural guidance and step-by-step instructions for a high availability deployment of {partner-product-name} on AWS. The guide addresses common scalability, high-availability, and security considerations for your deployment.

This guide is intended for a variety of audiences, including IT infrastructure architects, administrators, and DevOps professionals who are planning to implement or extend their Splunk Enterprise deployments on the AWS Cloud. It also provides direct links for viewing and launching AWS CloudFormation templates that assist with automating the deployment
9 changes: 9 additions & 0 deletions docs/deployment_guide/partner_editable/pre-reqs.adoc
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
// If no preperation is required, remove all content from here.
There are two important steps to verify prior to launching this Quick start:

* Ensure that the SmartStore bucket that is defined in the "SmartStoreBucketName" parameter *does not exist*. This Quick Start will attempt to create that bucket with an appropriate bucket policy. If that bucket already exists, the Quick Start will fail.
* Upload a valid Splunk license to an S3 bucket owned by the user launching the Quick Start. Take note of the bucket name as well as the path to the license file. An example may look something like this:
** Splunk license file is named "splunk.license" and uploaded to an S3 bucket called "my-s3-bucket" under the 'directory' called "license"
*** Parameter "SplunkLicenseBucket" should be configured to: my-s3-bucket
*** Parameter "SplunkLicensePath" should be configured to: license/splunk.license (note the lack of a leading "/" on the license path)

Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
This Quick Start deploys {partner-product-name} on the AWS Cloud

{partner-product-name} is the platform for turning data into doing. By monitoring and analyzing everything from customer clickstreams and transactions to security events and network activity, {partner-product-name} is a scalable and reliable data platform for investigating, monitoring, analyzing and acting on your data. With a full range of powerful search, analysis, alert, and visualization capabilities along with prepackaged content for many typical use cases, users can quickly discover and share insights.

For more details about the features and functionality of {partner-product-name}, see the https://docs.splunk.com/Documentation/Splunk[Splunk Enterprise documentation^].


1 change: 1 addition & 0 deletions docs/deployment_guide/partner_editable/regions.adoc
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
- All AWS Regions
15 changes: 15 additions & 0 deletions docs/deployment_guide/partner_editable/service_limits.adoc
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
// Replace the <n> in each row to specify the number of resources used in this deployment. Remove the rows for resources that aren’t used.

|===
|Resource |This deployment uses

// Space needed to maintain table headers
|VPCs |1
|AWS Identity and Access Management (IAM) security groups |2 or more
|IAM roles |2 or more
|Auto Scaling groups |1
|Classic Load Balancers |2
|EC2 Instances |5 or more
|EBS Volumes|5 or more
|S3 Buckets |1
|===
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
// Describe or link to specific knowledge requirements; for example: “familiarity with basic concepts in the areas of networking, database operations, and data encryption” or “familiarity with <software>.”

This Quick Start assumes familiarity with basic concepts of networking and Linux system administration, as well as basic knowledge of {partner-product-name}
1 change: 0 additions & 1 deletion submodules/quickstart-aws-vpc
Submodule quickstart-aws-vpc deleted from ac8786
Loading