Merge pull request #727 from weaveworks/update-docs-v0.15.0

update docs for v0.15.0

README.md

@@ -16,49 +16,33 @@ TF-controller offers many GitOps models:
 
 ## Quickstart and documentation
 
-To get started check out this [guide](https://docs.gitops.weave.works/docs/terraform/get-started/) on how to GitOps your Terraform resources with TF-controller and Flux.
+To get started check out this [guide](https://weaveworks.github.io/tf-controller/getting_started/) on how to GitOps your Terraform resources with TF-controller and Flux.
 
-Check out the [documentation](https://docs.gitops.weave.works/docs/terraform/overview/) for a list of [features](https://docs.gitops.weave.works/docs/terraform/overview/#features) and [use cases](https://docs.gitops.weave.works/docs/terraform/Using%20Terraform%20CRD/provision/).
+Check out the [documentation](https://weaveworks.github.io/tf-controller/) and [use cases](https://weaveworks.github.io/tf-controller/use_tf_controller/).
 
 ## Roadmap
 
-### Q3 2022
-  * [x] Performance
-  * [x] Scalability
-  * [x] Interop with Notification controller's Events and Alert
-  * [x] Validation webhooks for the planning stage
-
-### Q4 2022
-  * [x]  ~~CRD wrappers~~ A new component model for Terraform modules
-  * [x] Dependency management for TF-controller objects
-  * [x] AWS package for TF-controller (e.g. EKS, RDS, etc.)
-  * [x] CLI for Weave GitOps
-
-### Q1 2023
-  * [x] `v1alpha2` API
-  * [x] Flux v2.0.0-rc support
-  * [x] Better Terraform Cloud integration
-
 ### Q2 2023
-  * [ ] Write back and show plan in PRs (Atlantis-like experience)
-  * [ ] Type safety for custom backends
-  * [ ] Selectable Terraform versions
+  * [x] Selectable Terraform versions
   * [ ] Flux v2.0.0 GA support
-  * [ ] Break-the-glass feature
+  * [x] Break-the-glass feature
+  * [ ] Enhanced security (the lockdown mode)
+  * [x] Containerd compatibility verification 
 
 ### Q3 2023
-  * [ ] `v1beta1` API (stabilization)
-  * [ ] Observability - logging from the different stages of the runner
-  * [ ] CLI to GitOpsify existing Terraform workflows
-  * [ ] Enhanced security (the lockdown mode)
+  * [ ] Write back and show plan in PRs (Atlantis-like experience)
+  * [ ] CLI to GitOpsify existing Terraform workflows (UX improvement for CLI) 
+  * [ ] Type safety for custom backends
+  * [ ] Improvement GitOps dependency management 
   * [ ] External drift detector
-  * [ ] Cloud cost estimation
+  * [ ] Cloud cost estimation 
 
 ### Q4 2023
+  * [ ] Observability - logging from the different stages of the runner
+  * [ ] `v1beta1` API (stabilization) 
   * [ ] Azure package for TF-controller (e.g. AKS, CosmosDB, etc.)
   * [ ] GCP package for TF-controller (e.g. GKE, CloudSQL, etc.) 
   * [ ] ARM64 & Gravitron support
-  * [ ] Containerd compatibility verification
 
 ### Q1 2024
   * [ ] `v1beta2` API

api/v1alpha2/groupversion_info.go

@@ -14,7 +14,7 @@ See the License for the specific language governing permissions and
 limitations under the License.
 */
 
-// Package v1alpha1 contains API Schema definitions for the infra v1alpha1 API group
+// Package v1alpha2 contains API Schema definitions for the infra v1alpha2 API group
 // +kubebuilder:object:generate=true
 // +groupName=infra.contrib.fluxcd.io
 package v1alpha2

docs/getting_started.md

@@ -4,16 +4,17 @@
 
 Here are the requirements you need to set up before you start:
 
-  1. Flux v0.32.0 or later (not only the CLI, but also the controllers on the cluster). If you are not sure about the Flux version on your cluster, please re-bootstrap your cluster.
-  2. TF-controller uses **the Controller/Runner architecture**. The Controller acts as a client, and talks to each Runner's Pod via gRPC. Please make sure 
+  1. For Terraform Controller **v0.15+**, it requires **Flux v2.0** or later (not only the CLI, but also the controllers on the cluster). If you are not sure about the Flux version on your cluster, please re-bootstrap your cluster.
+  2. For Terraform Controller v0.13 and v0.14, Flux 2 v0.32 - v0.41 (of course, not only the CLI, but also the controllers on the cluster).
+  3. TF-controller uses **the Controller/Runner architecture**. The Controller acts as a client, and talks to each Runner's Pod via gRPC. Please make sure 
      1. **Each Runner's Pod in each Namespace** is allowed to open, and serve at **port 30000** (the gRPC port of a Runner), and the Controller can connect to it.
      2. **The Controller** needs to download tar.gz BLOBs from the **Source controller** via **port 80**.
      3. **The Controller** needs to post the events to the **Notification controller** via **port 80**.
 
 ## Installation
 
 Before using TF-controller, you have to install Flux by using either `flux install` or `flux bootstrap` command.
-Please note that TF-controller now requires **Flux v0.32.0** or later, so please make sure you have the latest version of Flux.
+Please note that TF-controller now requires **Flux v2.0** or later, so please make sure you have the latest version of Flux.
 After that you can install TF-controller with Flux HelmRelease by:
 
 ```shell
@@ -43,7 +44,7 @@ please see [chart readme](https://github.com/weaveworks/tf-controller/tree/main/
 Alternatively, you can install TF-controller via `kubectl`:
 
 ```shell
-export TF_CON_VER=v0.13.1
+export TF_CON_VER=v0.15.0
 kubectl apply -f https://github.com/weaveworks/tf-controller/releases/download/${TF_CON_VER}/tf-controller.crds.yaml
 kubectl apply -f https://github.com/weaveworks/tf-controller/releases/download/${TF_CON_VER}/tf-controller.rbac.yaml
 kubectl apply -f https://github.com/weaveworks/tf-controller/releases/download/${TF_CON_VER}/tf-controller.deployment.yaml
@@ -76,7 +77,7 @@ The GitOps automation mode could be enabled by setting `.spec.approvePlan=auto`.
 and automatically applied for you.
 
 ```yaml
-apiVersion: infra.contrib.fluxcd.io/v1alpha1
+apiVersion: infra.contrib.fluxcd.io/v1alpha2
 kind: Terraform
 metadata:
   name: helloworld

docs/how_to/index.md

@@ -1,3 +1,4 @@
 # How to
 
+  - [How to troubleshoot with **Break the Glass** mode](troubleshooting_with_break_the_glass_mode.md)
   - [How to **backup and restore** a Terraform state](backup_and_restore_a_Terraform_state.md)

docs/how_to/troubleshooting_with_break_the_glass_mode.md

@@ -0,0 +1,50 @@
+# Break the glass
+
+## What is break the glass?
+
+"Break the glass" refers to a troubleshooting mode specifically designed
+to provide a manual solution when the Terraform controller (TF-controller) 
+is not performing as expected. There are two primary methods of initiating this mode:
+
+1. Using the `tfctl` command-line tool.
+2. Setting the `spec.breakTheGlass` field to `true` in the Terraform object.
+
+## Using `tfctl` to Break the Glass
+
+To start a one-time troubleshooting session, you can use the `tfctl break-glass` command. For instance:
+
+```shell
+tfctl break-glass hello-world
+```
+
+This command initiates a session that allows you to execute any Terraform command
+to rectify the issues with your Terraform resources. It is noteworthy that this command
+does not require setting the `spec.breakTheGlass` field to `true` in the Terraform object.
+
+After resolving the issues, you can simply exit the shell. 
+GitOps will then continue to reconcile the Terraform object.
+
+## Break the glass with `spec.breakTheGlass` field
+
+This feature is particularly useful for troubleshooting Terraform objects at their initialization stage or in situations with unexpected errors.
+It is generally not recommended to use this mode routinely for fixing Terraform resources.
+
+You can enable the 'Break the Glass' feature for every reconciliation by setting the `breakTheGlass` field to `true` in the `spec` of the Terraform object.
+
+Here is a sample example:
+
+```yaml
+apiVersion: infra.contrib.fluxcd.io/v1alpha2
+kind: Terraform
+metadata:
+  name: hello-world
+  namespace: flux-system
+spec:
+  breakTheGlass: true
+  interval: 1m
+  path: ./
+  sourceRef:
+    kind: GitRepository
+    name: helloworld
+    namespace: flux-system
+```

docs/index.md

@@ -56,7 +56,9 @@ To get started, follow the [getting started](./getting_started.md) guide.
 
 ## Support Matrix
 
-|   Version   | Terraform | Source Controller | Flux v2 |
-|:-----------:|:---------:|:-----------------:|:-------:|
-| **v0.13.1** |  v1.3.1   |      v0.31.0      | v0.36.x |
-|   v0.12.0   |  v1.1.9   |      v0.26.1      | v0.32.x |
+| Version | Terraform | Source Controller | Flux v2 |
+|:-------:|:---------:|:-----------------:|:-------:|
+|  v0.15  |  v1.3.9   |      v1.0.x       | v2.0.x  |
+|  v0.14  |  v1.3.9   |      v0.31.0      | v0.41.x |
+|  v0.13  |  v1.3.1   |      v0.31.0      | v0.36.x |
+|  v0.12  |  v1.1.9   |      v0.26.1      | v0.32.x |

docs/tfctl.md

@@ -4,7 +4,13 @@
 
 ## Installation
 
-You can download the `tfctl` binary via the GitHub releases page: [https://github.com/weaveworks/tf-controller/releases](https://github.com/weaveworks/tf-controller/releases)
+To install `tfctl` via Homebrew, run the following command:
+
+```shell
+brew install weaveworks/tap/tfctl
+```
+
+You can also download the `tfctl` binary via the GitHub releases page: [https://github.com/weaveworks/tf-controller/releases](https://github.com/weaveworks/tf-controller/releases).
 
 ```
 Usage:

docs/use_tf_controller/index.md

@@ -12,6 +12,7 @@
   - [Use TF-controller to provision Terraform resources that are required **health checks**](to_provision_Terraform_resources_that_are_required_health_checks.md)
   - [Use TF-controller to provision resources and **destroy them when the Terraform object gets deleted**](to_provision_resources_and_destroy_them_when_the_Terraform_object_gets_deleted.md)
   - [Use TF-controller to **force unlock** Terraform states](to_force_unlock_Terraform_states.md)
+  - [Use TF-controller with Terraform Runners enabled via Env Variables](with_tf_runner_logging.md)
   - [Use TF-controller to provision resources with **customized Runner Pods**](to_provision_resources_with_customized_Runner_Pods.md)
   - [Use TF-controller with **Terraform Enterprise**](with_Terraform_Enterprise.md)
   - [Use TF-controller with **primitive modules**](with_primitive_modules.md)

docs/use_tf_controller/to_detect_drifts_only_without_plan_or_apply.md

@@ -3,7 +3,7 @@
 We can set `.spec.approvePlan` to `disable` to tell the controller to detect drifts of your Terraform resources only. Doing so will skip the `plan` and `apply` stages.
 
 ```yaml hl_lines="7"
-apiVersion: infra.contrib.fluxcd.io/v1alpha1
+apiVersion: infra.contrib.fluxcd.io/v1alpha2
 kind: Terraform
 metadata:
   name: hello-world

docs/use_tf_controller/to_force_unlock_Terraform_states.md

@@ -15,7 +15,7 @@ The recommended way is to do manual force unlock. To manually `force-unlock`, yo
 as the following example:
 
 ```yaml hl_lines="14-16"
-apiVersion: infra.contrib.fluxcd.io/v1alpha1
+apiVersion: infra.contrib.fluxcd.io/v1alpha2
 kind: Terraform
 metadata:
   name: helloworld

docs/use_tf_controller/to_plan_and_manually_apply_Terraform_resources.md

@@ -27,7 +27,7 @@ the `path` field specifies the location of the configuration files,
 and the `sourceRef` field points to the GitRepository object.
 
 ```yaml hl_lines="7"
-apiVersion: infra.contrib.fluxcd.io/v1alpha1
+apiVersion: infra.contrib.fluxcd.io/v1alpha2
 kind: Terraform
 metadata:
   name: helloworld
@@ -62,7 +62,7 @@ This process is known as the plan & manual approval workflow,
 as it involves generating a plan and requiring manual approval before the changes are applied.
 
 ```yaml hl_lines="7"
-apiVersion: infra.contrib.fluxcd.io/v1alpha1
+apiVersion: infra.contrib.fluxcd.io/v1alpha2
 kind: Terraform
 metadata:
   name: hello-world

docs/use_tf_controller/to_provision_Terraform_resources_that_are_required_health_checks.md

@@ -25,7 +25,7 @@ We can use standard Go template expressions, like `${{ .rdsAddress }}`, to refer
 We support two types of health checks, `tcp` amd `http`. The `tcp` type allows us to verify a TCP connection, while the `http` type is for verify an HTTP URL. The default timeout of each health check is 20 seconds.
 
 ```yaml hl_lines="14-25"
-apiVersion: infra.contrib.fluxcd.io/v1alpha1
+apiVersion: infra.contrib.fluxcd.io/v1alpha2
 kind: Terraform
 metadata:
   name: helloworld

docs/use_tf_controller/to_provision_resources_and_auto_approve.md

@@ -40,7 +40,7 @@ your infrastructure is properly managed and controlled.
 The following is an example of a `Terraform` object; we use the "auto-apply" mode:
 
 ```yaml hl_lines="8"
-apiVersion: infra.contrib.fluxcd.io/v1alpha1
+apiVersion: infra.contrib.fluxcd.io/v1alpha2
 kind: Terraform
 metadata:
   name: helloworld

docs/use_tf_controller/to_provision_resources_and_destroy_them_when_the_Terraform_object_gets_deleted.md

@@ -11,7 +11,7 @@ To enable destroy resources on object deletion, set `.spec.destroyResourcesOnDel
 ~> **WARNING:** This feature will destroy your resources on the cloud if the Terraform object gets deleted. Please use it with cautions.
 
 ```yaml hl_lines="8"
-apiVersion: infra.contrib.fluxcd.io/v1alpha1
+apiVersion: infra.contrib.fluxcd.io/v1alpha2
 kind: Terraform
 metadata:
   name: helloworld

docs/use_tf_controller/to_provision_resources_and_obtain_outputs.md

@@ -7,7 +7,7 @@ Outputs created by Terraform can be written to a secret using `.spec.writeOutput
 We can specify a target secret in `.spec.writeOutputsToSecret.name`, and the controller will write all outputs to the secret by default.
 
 ```yaml hl_lines="14-15"
-apiVersion: infra.contrib.fluxcd.io/v1alpha1
+apiVersion: infra.contrib.fluxcd.io/v1alpha2
 kind: Terraform
 metadata:
   name: helloworld
@@ -29,7 +29,7 @@ spec:
 We can choose only a subset of outputs by specify output names we'd like to write in the `.spec.writeOutputsToSecret.outputs` array.
 
 ```yaml hl_lines="16-18"
-apiVersion: infra.contrib.fluxcd.io/v1alpha1
+apiVersion: infra.contrib.fluxcd.io/v1alpha2
 kind: Terraform
 metadata:
   name: helloworld
@@ -59,7 +59,7 @@ TF-controller supports mapping output name using the `old_name:new_name` format.
 In the following example, we renamed `age_key` output as `age.agekey` entry for the `helloworld-output` Secret's data, so that other components in the GitOps pipeline could consume it.
 
 ```yaml hl_lines="16-17"
-apiVersion: infra.contrib.fluxcd.io/v1alpha1
+apiVersion: infra.contrib.fluxcd.io/v1alpha2
 kind: Terraform
 metadata:
   name: helloworld
@@ -84,7 +84,7 @@ As an example, operators such as [kubernetes-replicator](https://github.com/mitt
 allow replicating secrets from one namespace to another but use annotations to do so.
 
 ```yaml hl_lines="16-19"
-apiVersion: infra.contrib.fluxcd.io/v1alpha1
+apiVersion: infra.contrib.fluxcd.io/v1alpha2
 kind: Terraform
 metadata:
   name: helloworld

docs/use_tf_controller/to_provision_resources_with_customized_Runner_Pods.md

@@ -7,7 +7,7 @@ For example, for Azure AKS to grant pod active directory permissions using Azure
 a label like `aadpodidbinding: myIdentity` on the pod is required.
 
 ```yaml
-apiVersion: infra.contrib.fluxcd.io/v1alpha1
+apiVersion: infra.contrib.fluxcd.io/v1alpha2
 kind: Terraform
 metadata:
   name: helloworld
@@ -33,7 +33,7 @@ spec:
 By default, the Terraform controller uses `RUNNER_POD_IMAGE` environment variable to identify the Runner Pod's image to use. You can customize the image on the global level by updating the value of the environment variable or, you can specify an image to use per Terraform object for its reconciliation.
 
 ```yaml
-apiVersion: infra.contrib.fluxcd.io/v1alpha1
+apiVersion: infra.contrib.fluxcd.io/v1alpha2
 kind: Terraform
 metadata:
   name: helloworld
@@ -56,4 +56,4 @@ You can use [`runner.Dockerfile`](https://github.com/weaveworks/tf-controller/bl
 ## Customize Runner Pod Specifications
 
 You can also customize various Runner Pod `spec` fields to control and configure how the Runner Pod runs. 
-For example, you can configure Runner Pod `spec` affinity and tolerations if you need to run in on a specific set of nodes. Please see [RunnerPodSpec](https://weaveworks.github.io/tf-controller/References/terraform/#infra.contrib.fluxcd.io/v1alpha1.RunnerPodSpec) for a list of the configurable Runner Pod `spec` fields.
+For example, you can configure Runner Pod `spec` affinity and tolerations if you need to run in on a specific set of nodes. Please see [RunnerPodSpec](https://weaveworks.github.io/tf-controller/References/terraform/#infra.contrib.fluxcd.io/v1alpha2.RunnerPodSpec) for a list of the configurable Runner Pod `spec` fields.

docs/use_tf_controller/to_set_variables_for_Terraform_resources.md

@@ -31,7 +31,7 @@ Note that in the case of the same variable key being passed multiple times, the
 the lattermost instance of the key passed to `varsFrom`.
 
 ```yaml hl_lines="15-20 22-28"
-apiVersion: infra.contrib.fluxcd.io/v1alpha1
+apiVersion: infra.contrib.fluxcd.io/v1alpha2
 kind: Terraform
 metadata:
   name: helloworld
@@ -77,7 +77,7 @@ variable "cluster_spec" {
 ```
 
 ```yaml hl_lines="17-20"
-apiVersion: infra.contrib.fluxcd.io/v1alpha1
+apiVersion: infra.contrib.fluxcd.io/v1alpha2
 kind: Terraform
 metadata:
   name: helloworld

docs/use_tf_controller/with_GitOps_dependency_management.md

@@ -23,7 +23,7 @@ It is configured to use the `auto-apply` mode, and write outputs to the secret `
 
 ```yaml hl_lines="20-24"
 ---
-apiVersion: infra.contrib.fluxcd.io/v1alpha1
+apiVersion: infra.contrib.fluxcd.io/v1alpha2
 kind: Terraform
 metadata:
   name: aws-s3-bucket
@@ -68,7 +68,7 @@ Please note that we use `${{` and  `}}` as the delimiters for the variable name,
 
 ```yaml hl_lines="11 18 20-21"
 ---
-apiVersion: infra.contrib.fluxcd.io/v1alpha1
+apiVersion: infra.contrib.fluxcd.io/v1alpha2
 kind: Terraform
 metadata:
   name: aws-s3-bucket-acl
@@ -102,7 +102,7 @@ To avoid this, we need to add the `kustomize.toolkit.fluxcd.io/substitute: disab
 
 ```yaml hl_lines="8"
 ---
-apiVersion: infra.contrib.fluxcd.io/v1alpha1
+apiVersion: infra.contrib.fluxcd.io/v1alpha2
 kind: Terraform
 metadata:
   name: aws-s3-bucket-acl

docs/use_tf_controller/with_Terraform_Enterprise.md

@@ -53,7 +53,7 @@ In your Terraform object, you'll have to 1. disable the backend by setting `spec
 
 ```yaml hl_lines="10-14"
 ---
-apiVersion: infra.contrib.fluxcd.io/v1alpha1
+apiVersion: infra.contrib.fluxcd.io/v1alpha2
 kind: Terraform
 metadata:
   name: tfe-demo

docs/use_tf_controller/with_a_custom_backend.md

@@ -7,7 +7,7 @@ The tfstate is stored in a secret named: `tfstate-${workspace}-${secretSuffix}`.
 If you wish to use a custom backend, you can configure it by defining the `.spec.backendConfig.customConfiguration` with one of the backends such as **GCS** or **S3**, for example:
 
 ```yaml hl_lines="9-21"
-apiVersion: infra.contrib.fluxcd.io/v1alpha1
+apiVersion: infra.contrib.fluxcd.io/v1alpha2
 kind: Terraform
 metadata:
   name: helloworld

docs/use_tf_controller/with_an_OCI_Artifact_as_Source.md

@@ -30,7 +30,7 @@ spec:
   ref:
     tag: main
 ---
-apiVersion: infra.contrib.fluxcd.io/v1alpha1
+apiVersion: infra.contrib.fluxcd.io/v1alpha2
 kind: Terraform
 metadata:
   name: helloworld-tf-oci