Skip to content

Commit

Permalink
Synciq doc (#263)
Browse files Browse the repository at this point in the history
* added validation for network pool/rule ds

* added doc for synciq + async

* added synciq workflow guide

* add key values for action

* removed root path in example
  • Loading branch information
Krishnan-Priyanshu authored Nov 26, 2024
1 parent e2e08ff commit 92635dd
Show file tree
Hide file tree
Showing 8 changed files with 390 additions and 8 deletions.
41 changes: 41 additions & 0 deletions docs/guides/aync_resource_info.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
---
# Copyright (c) 2024 Dell Inc., or its subsidiaries. All Rights Reserved.

# Licensed under the Mozilla Public License Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at

# http://mozilla.org/MPL/2.0/


# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
page_title: "Adding Asynchronous Operations"
title: "Adding Asynchronous Operations"
linkTitle: "Addding Asynchronous Operations"
---

# Terraform Internal Behaviour
Terraform's concurrent execution capabilities allow multiple operations to run in parallel, each thread responsible for managing the lifecycle of a resource. However, even with this parallel execution, each individual thread must still execute its operations in a synchronous manner. By default, Terraform's create, update, and delete operations wait for a resource to reach its expected lifecycle state before proceeding with the next step.

For instance, when a resource is in the creation phase, Terraform will not advance to the subsequent step until the resource has successfully reached the completion phase. In scenarios where platform operations are performed asynchronously, which can result in extended creation times, Terraform will wait for the resource to reach the completion phase before proceeding, thereby introducing a delay in the overall workflow.

# Consideration for Asynchronous Operations In Terraform
When utilizing Terraform, it is essential to consider the implications of asynchronous operations of the platform. To ensure seamless automation, this guide provides recommendations for resources that support asynchronous operations, enabling users to effectively integrate these features into their infrastructure provisioning workflows.

1. The state file doesn't have the full information of the resource because the resource is not yet created.
2. The resource's stage must be refreshed to get its full information, including whether the resource failed to be created.
3. Terraform does not check the stage of a resource again after initiating its creation or deletion. Failed operations are only shown in subsequent refreshes of the resource.
4. The asynchronous resource must have no dependencies, because the resource is not fully created before another operation begins.

# List of Asynchronous Resources
1. SyncIQ replication Job
2. Snapshot Restore (Only SnapRevert Operation)





92 changes: 92 additions & 0 deletions docs/guides/syncIQ_workflow.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,92 @@
---
# Copyright (c) 2024 Dell Inc., or its subsidiaries. All Rights Reserved.

# Licensed under the Mozilla Public License Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at

# http://mozilla.org/MPL/2.0/


# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
page_title: "SyncIQ Workflow",
title: "SyncIQ Workflow"
linkTitle: "SyncIQ Workflow"
---


SyncIQ Workflow Explanation

SyncIQ is a data management solution that helps manage data movement between different locations. The workflow involves enabling the SyncIQ service, creating a SyncIQ policy, creating a replication job from the policy, and monitoring the job's status using a replication report.

Step 1: Enable SyncIQ Service

To begin, the SyncIQ service needs to be enabled. This is done using the powerscale_synciq_global_settings resource in Terraform. The service parameter is set to "on" to enable the service.

```
### Make sure SyncIQ service is enabled
resource "powerscale_synciq_global_settings" "enable_synciq" {
service = "on"
}
```
Step 2: Create SyncIQ Policy

Next, a SyncIQ policy needs to be created. A policy defines the rules for data movement, such as the source and target locations, and the action to take (e.g., sync). In this example, a policy named policy1 is created with the following settings:

- action is set to "sync" to synchronize data between the source and target locations.
- source_root_path is set to "/ifs" to specify the source location.
- target_host is set to "10.10.10.10" to specify the target location.
- target_path is set to "/ifs/policy1Sink" to specify the target path.

```
### Create SyncIQ policy with action sync
resource "powerscale_synciq_policy" "policy1" {
name = "policy1"
action = "sync" # action can be sync or copy
source_root_path = "/ifs/Source"
target_host = "10.10.10.10"
target_path = "/ifs/policy1Sink"
}
```

Step 3: Create Replication Job

A replication job is created from the SyncIQ policy using the powerscale_synciq_replication_job resource. The job is configured to run the policy (identified by the id parameter) and is not paused (i.e., is_paused is set to false).
```
### Create replication job from SyncIQ policy
resource "powerscale_synciq_replication_job" "job1" {
action = "run" # action can be run, test, resync_prep, allow_write or allow_write_revert
id = "policy1"
is_paused = false
}
```
Step 4: Monitor Replication Job Status

To monitor the status of the replication job, a replication report can be used. The powerscale_synciq_replication_report resource is used to filter the report to show only the replication job with the name Policy1.

```
### Use replication report to view the status of the job
data "powerscale_synciq_replication_report" "filtering" {
filter {
policy_name = "Policy1"
}
}
```
By following these steps, SyncIQ can be used to manage data movement between different locations and monitor the status of the replication jobs.








49 changes: 46 additions & 3 deletions docs/resources/synciq_replication_job.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,22 +19,65 @@ linkTitle: "powerscale_synciq_replication_job"
page_title: "powerscale_synciq_replication_job Resource - terraform-provider-powerscale"
subcategory: ""
description: |-
Resource for managing SyncIQReplicationJobResource on OpenManage Enterprise.
The PowerScale SyncIQ ReplicationJob resource provides a means of managing replication jobs on PowerScale clusters.
This resource allows for the manual triggering of replication jobs to replicate data from a source PowerScale cluster to a target PowerScale cluster.
Note: The replication job is an asynchronous operation, and this resource does not provide real-time monitoring of the job's status.
To check the status of the job,please use the powerscalesynciqreplication_report datasource.
---

# powerscale_synciq_replication_job (Resource)

Resource for managing SyncIQReplicationJobResource on OpenManage Enterprise.
The PowerScale SyncIQ ReplicationJob resource provides a means of managing replication jobs on PowerScale clusters.
This resource allows for the manual triggering of replication jobs to replicate data from a source PowerScale cluster to a target PowerScale cluster.
Note: The replication job is an asynchronous operation, and this resource does not provide real-time monitoring of the job's status.
To check the status of the job,please use the powerscale_synciq_replication_report datasource.


## Example Usage

```terraform
/*
Copyright (c) 2024 Dell Inc., or its subsidiaries. All Rights Reserved.
Licensed under the Mozilla Public License Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://mozilla.org/MPL/2.0/
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
# Available actions: Create and Update updates the syncIQ Replication Job. Delete will delete job and clear the state file.
# After `terraform apply` of this example file will perform action on the synciq replicaiton job according to the attributes set in the config
# PowerScale SynIQ Replication Job allows you to manage the SyncIQ Replication Jobs on the Powerscale array
resource "powerscale_synciq_replication_job" "job1" {
action = "run" # action can be run, test, resync_prep, allow_write or allow_write_revert
id = "TerraformPolicy" # id/name of the synciq policy, use synciq policy resource to create policy.
is_paused = false # change job state to running or paused.
}
# There are other attributes values as well. Please refer the documentation.
# After the execution of above resource block, job would have been extecuted/updated on the PowerScale array. For more information, Please check the terraform state file.
```

<!-- schema generated by tfplugindocs -->
## Schema

### Required

- `action` (String) Action for the job - run, test, resync_prep, allow_write, allow_write_revert
- `action` (String) Action for the job
run - to start the replication job using synciq policy
test - to test the replication job using synciq policy,
resync_prep - Resync_prep is a preparation step in PowerScale SyncIQ replication jobs that helps ensure a successful replication operation by performing a series of checks and verifications on the source and target volumes before starting the replication process.,
allow_write - allow_write determines whether the replication job allows writes to the target volume during the replication process. When configured, the target volume is writable, and any changes made to the target volume will be replicated to the source volume. This is useful in scenarios where you need to make changes to the target volume, such as updating files or creating new files, while the replication job is running.,
allow_write_revert - allow_write_revert determines whether the replication job allows writes to the target volume when reverting a replication job. When configure, the target volume is writable during the revert process, allowing changes made to the target volume during the revert process to be replicated to the source volume.
- `id` (String) ID/Name of the policy

### Optional
Expand Down
31 changes: 31 additions & 0 deletions examples/resources/powerscale_synciq_replication_job/provider.tf
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
/*
Copyright (c) 2023-2024 Dell Inc., or its subsidiaries. All Rights Reserved.
Licensed under the Mozilla Public License Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://mozilla.org/MPL/2.0/
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/

terraform {
required_providers {
powerscale = {
source = "registry.terraform.io/dell/powerscale"
}
}
}

provider "powerscale" {
username = var.username
password = var.password
endpoint = var.endpoint
insecure = var.insecure
}
29 changes: 29 additions & 0 deletions examples/resources/powerscale_synciq_replication_job/resource.tf
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
/*
Copyright (c) 2024 Dell Inc., or its subsidiaries. All Rights Reserved.
Licensed under the Mozilla Public License Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://mozilla.org/MPL/2.0/
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/

# Available actions: Create and Update updates the syncIQ Replication Job. Delete will delete job and clear the state file.
# After `terraform apply` of this example file will perform action on the synciq replicaiton job according to the attributes set in the config

# PowerScale SynIQ Replication Job allows you to manage the SyncIQ Replication Jobs on the Powerscale array
resource "powerscale_synciq_replication_job" "job1" {
action = "run" # action can be run, test, resync_prep, allow_write or allow_write_revert
id = "TerraformPolicy" # id/name of the synciq policy, use synciq policy resource to create policy.
is_paused = false # change job state to running or paused.
}

# There are other attributes values as well. Please refer the documentation.
# After the execution of above resource block, job would have been extecuted/updated on the PowerScale array. For more information, Please check the terraform state file.
23 changes: 18 additions & 5 deletions powerscale/provider/synciq_replication_job_resource.go
Original file line number Diff line number Diff line change
Expand Up @@ -91,8 +91,11 @@ func (r *SyncIQReplicationJobResource) Metadata(_ context.Context, req resource.
// Schema defines the schema for the resource.
func (r *SyncIQReplicationJobResource) Schema(_ context.Context, _ resource.SchemaRequest, resp *resource.SchemaResponse) {
resp.Schema = schema.Schema{
MarkdownDescription: "Resource for managing SyncIQReplicationJobResource on OpenManage Enterprise.",
Version: 1,
MarkdownDescription: `The PowerScale SyncIQ ReplicationJob resource provides a means of managing replication jobs on PowerScale clusters.
This resource allows for the manual triggering of replication jobs to replicate data from a source PowerScale cluster to a target PowerScale cluster.
Note: The replication job is an asynchronous operation, and this resource does not provide real-time monitoring of the job's status.
To check the status of the job,please use the powerscale_synciq_replication_report datasource.`,
Version: 1,
Attributes: map[string]schema.Attribute{
"id": schema.StringAttribute{
Required: true,
Expand All @@ -103,9 +106,19 @@ func (r *SyncIQReplicationJobResource) Schema(_ context.Context, _ resource.Sche
},
},
"action": schema.StringAttribute{
Required: true,
Description: "Action for the job - run, test, resync_prep, allow_write, allow_write_revert",
MarkdownDescription: "Action for the job - run, test, resync_prep, allow_write, allow_write_revert",
Required: true,
Description: `Action for the job
run - to start the replication job using synciq policy
test - to test the replication job using synciq policy,
resync_prep - Resync_prep is a preparation step in PowerScale SyncIQ replication jobs that helps ensure a successful replication operation by performing a series of checks and verifications on the source and target volumes before starting the replication process.,
allow_write - allow_write determines whether the replication job allows writes to the target volume during the replication process. When configured, the target volume is writable, and any changes made to the target volume will be replicated to the source volume. This is useful in scenarios where you need to make changes to the target volume, such as updating files or creating new files, while the replication job is running.,
allow_write_revert - allow_write_revert determines whether the replication job allows writes to the target volume when reverting a replication job. When configure, the target volume is writable during the revert process, allowing changes made to the target volume during the revert process to be replicated to the source volume.`,
MarkdownDescription: `Action for the job
run - to start the replication job using synciq policy
test - to test the replication job using synciq policy,
resync_prep - Resync_prep is a preparation step in PowerScale SyncIQ replication jobs that helps ensure a successful replication operation by performing a series of checks and verifications on the source and target volumes before starting the replication process.,
allow_write - allow_write determines whether the replication job allows writes to the target volume during the replication process. When configured, the target volume is writable, and any changes made to the target volume will be replicated to the source volume. This is useful in scenarios where you need to make changes to the target volume, such as updating files or creating new files, while the replication job is running.,
allow_write_revert - allow_write_revert determines whether the replication job allows writes to the target volume when reverting a replication job. When configure, the target volume is writable during the revert process, allowing changes made to the target volume during the revert process to be replicated to the source volume.`,
Validators: []validator.String{
stringvalidator.OneOf("run", "test", "resync_prep", "allow_write", "allow_write_revert"),
},
Expand Down
41 changes: 41 additions & 0 deletions templates/guides/aync_resource_info.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
---
# Copyright (c) 2024 Dell Inc., or its subsidiaries. All Rights Reserved.

# Licensed under the Mozilla Public License Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at

# http://mozilla.org/MPL/2.0/


# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
page_title: "Adding Asynchronous Operations"
title: "Adding Asynchronous Operations"
linkTitle: "Addding Asynchronous Operations"
---

# Terraform Internal Behaviour
Terraform's concurrent execution capabilities allow multiple operations to run in parallel, each thread responsible for managing the lifecycle of a resource. However, even with this parallel execution, each individual thread must still execute its operations in a synchronous manner. By default, Terraform's create, update, and delete operations wait for a resource to reach its expected lifecycle state before proceeding with the next step.

For instance, when a resource is in the creation phase, Terraform will not advance to the subsequent step until the resource has successfully reached the completion phase. In scenarios where platform operations are performed asynchronously, which can result in extended creation times, Terraform will wait for the resource to reach the completion phase before proceeding, thereby introducing a delay in the overall workflow.

# Consideration for Asynchronous Operations In Terraform
When utilizing Terraform, it is essential to consider the implications of asynchronous operations of the platform. To ensure seamless automation, this guide provides recommendations for resources that support asynchronous operations, enabling users to effectively integrate these features into their infrastructure provisioning workflows.

1. The state file doesn't have the full information of the resource because the resource is not yet created.
2. The resource's stage must be refreshed to get its full information, including whether the resource failed to be created.
3. Terraform does not check the stage of a resource again after initiating its creation or deletion. Failed operations are only shown in subsequent refreshes of the resource.
4. The asynchronous resource must have no dependencies, because the resource is not fully created before another operation begins.

# List of Asynchronous Resources
1. SyncIQ replication Job
2. Snapshot Restore (Only SnapRevert Operation)





Loading

0 comments on commit 92635dd

Please sign in to comment.