Masking sensitive information in AWS Step Functions Express

Overview

This repository contains a demonstration of how to use the Amazon CloudWatch logs masking feature to mask sensitive information in AWS Step Functions Express.

This demo will create all necessary cloud resources with the AWS Cloud Development Kit (CDK) to create the express Step Function with the configured log stream and audit destination.

Details

This is possible, because for express workflows, the execution history and detailed infos are gathered through CloudWatch logs, see also here. This allows you to use CloudWatch Data Masking capabilities, especially for data fields such as Address, Email, Name, and many more.

For an example execution with PII relevant data and active masking of Address, Email and Name, this looks like this:

{
    "name": "John Miller",
    "address": "2113 7th Ave, Seattle, WA 98121, United States",
    "key3": "test@example.com",
    "order-id": "test-order-id"
}

1. Execution details

In the execution details, you can see that the log input (and also other data) is masked.

2. Log groups overview

In the log groups overview, you can see that the Data Protection is enabled, and that it identified 4 sensitive informations.

3. Log stream masked

In the log stream you can see the masked log events, with the display option to unmask (top right).

4. Log stream unmasked

If you select to temporarly unmask protected data, you will see the original input.

5. Log stream unmasked with insufficient permissions

If your role lacks the permission of logs:Unmask you are not allowed to see the details, see also here for more information.

6. Log stream audit destination

If you have enabled an audit destination, you are able to review the audit findings report.

Prerequisites

Before you begin, make sure you have the following prerequisites in place:

1. AWS Account: You will need an AWS account to deploy and run this projen CDK-based project.

2. AWS CLI: Ensure that you have the AWS CLI installed and configured with the necessary AWS credentials. You can install it following the instructions here.

3. Node.js: This project is built using Node.js, so make sure you have Node.js installed. You can download it from the official website.

Usage

To deploy and run this projen CDK-based project, follow these steps:

1. Clone this repository to your local machine:

git clone https://github.com/moralesl/masking-sensitive-express-step-functions.git

2. Navigate to the project directory:

cd masking-sensitive-express-step-functions

3. Install project dependencies:

npm install && npx projen

4. Deploy the CDK stack to your AWS account:

npx projen deploy

5. After the deployment is complete, the CDK will output a state machine ARN, export it so that you can use it to invoke the API

export STATE_MACHINE_ARN=<Set the output URL>

It should look similar to this

Outputs: masking-sensitive-express-step-functions-dev.LoggingAndMaskingWorkflowArn = arn:aws:states:eu-central-1:123456789012:stateMachine:LoggingAndMaskingWorkflow

Result would be for this example

export STATE_MACHINE_ARN=arn:aws:states:eu-central-1:123456789012:stateMachine:LoggingAndMaskingWorkflow

6. Invoke the state machine, make sure that you have valid AWS credentials

npx projen invoke-state-machine

7. Retrieve the logs

sam logs --stack-name masking-sensitive-express-step-functions-dev

This should result in a similar output like this:

states/LoggingAndMaskingWorkflow/2024-04-24-11-50/00000000 2024-04-24T11:54:30.672000 {
  "id": "1",
  "type": "ExecutionStarted",
  "details": {
    "input": "{\"name\":\"***********\",\"address\":\"**********************************************\",\"key3\":\"****************\",\"order-id\": \"test-order-id\"}",
    "inputDetails": {
      "truncated": false
    },
    "roleArn": "arn:aws:iam::123456789012:role/masking-sensitive-express-LoggingAndMaskingWorkflow-PrqsMkZA1j2J"
  },
  "previous_event_id": "0",
  "event_timestamp": "1713959670672",
  "execution_arn": "arn:aws:states:eu-central-1:123456789012:express:LoggingAndMaskingWorkflow:cli-test-run:4135929f-8d60-4ea5-b3c3-b243bec43271",
  "redrive_count": "0"
}
states/LoggingAndMaskingWorkflow/2024-04-24-11-50/00000000 2024-04-24T11:54:30.680000 {
  "id": "2",
  "type": "PassStateEntered",
  "details": {
    "input": "{\"name\":\"***********\",\"address\":\"**********************************************\",\"key3\":\"****************\",\"order-id\": \"test-order-id\"}",
    "inputDetails": {
      "truncated": false
    },
    "name": "LogInput"
  },
  "previous_event_id": "0",
  "event_timestamp": "1713959670680",
  "execution_arn": "arn:aws:states:eu-central-1:123456789012:express:LoggingAndMaskingWorkflow:cli-test-run:4135929f-8d60-4ea5-b3c3-b243bec43271",
  "redrive_count": "0"
}
states/LoggingAndMaskingWorkflow/2024-04-24-11-50/00000000 2024-04-24T11:54:30.680000 {
  "id": "3",
  "type": "PassStateExited",
  "details": {
    "name": "LogInput",
    "output": "{\"name\":\"***********\",\"address\":\"**********************************************\",\"key3\":\"****************\",\"order-id\": \"test-order-id\"}",
    "outputDetails": {
      "truncated": false
    }
  },
  "previous_event_id": "2",
  "event_timestamp": "1713959670680",
  "execution_arn": "arn:aws:states:eu-central-1:123456789012:express:LoggingAndMaskingWorkflow:cli-test-run:4135929f-8d60-4ea5-b3c3-b243bec43271",
  "redrive_count": "0"
}
states/LoggingAndMaskingWorkflow/2024-04-24-11-50/00000000 2024-04-24T11:54:30.680000 {
  "id": "4",
  "type": "ExecutionSucceeded",
  "details": {
    "output": "{\"name\":\"***********\",\"address\":\"**********************************************\",\"key3\":\"****************\",\"order-id\": \"test-order-id\"}",
    "outputDetails": {
      "truncated": false
    }
  },
  "previous_event_id": "3",
  "event_timestamp": "1713959670680",
  "execution_arn": "arn:aws:states:eu-central-1:123456789012:express:LoggingAndMaskingWorkflow:cli-test-run:4135929f-8d60-4ea5-b3c3-b243bec43271",
  "redrive_count": "0"
}

8. To clean up the resources, run the following command:

npx projen destroy

Security

See CONTRIBUTING for more information.

License

This library is licensed under the MIT-0 License. See the LICENSE file.