Merge pull request #2404 from humem001/mikeh-feature-apigw-rest-stepf…

…unctions-express-sync-bedrock-sam New serverless pattern - APIGW, StepFunctions (Synchronous Express), Prompt Chaining with Bedrock
aws-samples · Sep 3, 2024 · bffcb8b · bffcb8b
2 parents 6bf1db5 + a474658
commit bffcb8b
Show file tree

Hide file tree

Showing 7 changed files with 485 additions and 0 deletions.
diff --git a/apigw-rest-stepfunctions-express-sync-bedrock-sam/Readme.md b/apigw-rest-stepfunctions-express-sync-bedrock-sam/Readme.md
@@ -0,0 +1,98 @@
+# Prompt Chaining with Amazon API Gateway, AWS Step Functions and Amazon Bedrock.
+
+The AWS Serverless Application Model (SAM) template deploys an Amazon API Gateway HTTP API endpoint connected to an AWS Step Functions state machine. This example demonstrates how to invoke an Express state machine synchronously and utilize AWS Step Functions intrinsic functions to chain two prompts, which are then used to invoke the Amazon Bedrock language model. The output from the state machine execution is returned to the client within 29 seconds, using the HTTP API. This no-code example showcases how the results from the first prompt can be used to provide context for the second prompt, allowing the language model to deliver a highly-curated response. By chaining these prompts, the system can leverage the capabilities of the language model to generate more meaningful and contextual outputs.
+
+Learn more about this pattern at [Serverless Land Patterns](https://serverlessland.com/patterns/apigw-rest-stepfunctions-express-sync-bedrock-sam)
+
+Important: this application uses various AWS services and there are costs associated with these services after the Free Tier usage - please see the [AWS Pricing page](https://aws.amazon.com/pricing/) for details. You are responsible for any AWS costs incurred. No warranty is implied in this example.
+
+## Requirements
+
+* [Create an AWS account](https://portal.aws.amazon.com/gp/aws/developer/registration/index.html) if you do not already have one and log in. The IAM user that you use must have sufficient permissions to make necessary AWS service calls and manage AWS resources.
+* [AWS CLI](https://docs.aws.amazon.com/cli/latest/userguide/install-cliv2.html) installed and configured
+* [Git Installed](https://git-scm.com/book/en/v2/Getting-Started-Installing-Git)
+* [AWS Serverless Application Model](https://docs.aws.amazon.com/serverless-application-model/latest/developerguide/serverless-sam-cli-install.html) (AWS SAM) installed
+* [NOTE! Manage Access to Amazon Bedrock Foundation Models](https://docs.aws.amazon.com/bedrock/latest/userguide/model-access.html) at the time of writing, this example uses Amazon Bedrock foundation model cohere.command-text-v14
+
+
+## Deployment Instructions
+
+1. Create a new directory, navigate to that directory in a terminal and clone the GitHub repository:
+    ``` 
+    git clone https://github.com/aws-samples/serverless-patterns
+    ```
+2. Change directory to the pattern directory:
+    ```
+    cd apigw-rest-stepfunctions-express-sync-bedrock-sam
+    ```
+3. From the command line, use AWS SAM build to prepare an application for subsequent steps in the developer workflow, such as local testing or deploying to the AWS Cloud:
+    ```
+    sam build
+    ```    
+4. From the command line, use AWS SAM to deploy the AWS resources for the pattern as specified in the template.yml file:
+    ```
+    sam deploy --guided
+    ```
+5. During the prompts:
+    * Enter a stack name
+    * Select the desired AWS Region
+    * Allow SAM to create roles with the required permissions if needed.
+
+6. Note the outputs from the SAM deployment process. These contain the resource names and/or ARNs which are used for testing.
+ 
+## Architecture
+![apigw-1](images/apigw.png)
+
+## How it Works
+In this example, the state machine is invoked with a JSON payload
+```asl
+{
+  "prompt_one": "Write a 500 word blog post on The Beatles"
+}
+```
+During execution, the Task state calls the Bedrock API and the response is passed to the task 'result_one'.
+```asl
+{
+  "result_one.$": "$.Body.generations[0].text"
+}
+```
+A Pass state is then used to format the data using an Intrinsic Function (States.) which is passed to the next state.
+```asl
+{
+  "convo_one.$": "States.Format('{}',$.result_one.result_one)"
+}
+```
+The second prompt is then executed with new instructions and the results from the first execution. This provides the prompt with more context
+```asl
+{
+  "prompt.$": "States.Format('{}\n{}','Human: Now write a short story based on the following. Assistant:', $.convo_one.convo_one)",
+  "max_tokens": 1000
+}
+```
+By default, the state then sends the task result as output.
+
+
+## Testing
+
+The stack will output the **api endpoint**. You can use *Postman* or *curl* to send a POST request to the API Gateway endpoint.
+
+```
+curl -H "Content-type: application/json" -X POST -d '{"prompt_one": "Write a 500 word blog post on The Beatles"}' <Your API endpoint>
+
+```
+After runnning the above command, API Gateway will invoke the State machine and return the results back to the client instead of just the State machine's execution Id. 
+
+## Cleanup
+
+1. Delete the stack
+    ```bash
+    sam delete
+    ```
+2. Confirm the stack has been deleted
+    ```bash
+    aws cloudformation list-stacks --query "StackSummaries[?contains(StackName,'STACK_NAME')].StackStatus"
+    ```
+----
+Copyright 2024 Amazon.com, Inc. or its affiliates. All Rights Reserved.
+
+SPDX-License-Identifier: MIT-0
diff --git a/apigw-rest-stepfunctions-express-sync-bedrock-sam/api.yaml b/apigw-rest-stepfunctions-express-sync-bedrock-sam/api.yaml
@@ -0,0 +1,41 @@
+openapi: "3.0.1"
+info:
+  title: "step-function-integration"
+  version: "2022-06-20T05:39:09Z"
+servers:
+  variables:
+    basePath:
+      default: "/dev"
+paths:
+  /:
+    post:
+      responses:
+        "200":
+          description: "200 response"
+          content:
+            application/json:
+              schema:
+                $ref: "#/components/schemas/Empty"
+      x-amazon-apigateway-integration:
+        type: "aws"
+        credentials: 
+          Fn::Sub: "${RestApiRole.Arn}"
+        httpMethod: "POST"
+        uri: 
+          "arn:aws:apigateway:${AWS::Region}:states:action/StartSyncExecution"
+        responses:
+          default:
+            statusCode: "200"
+            responseTemplates:
+              application/json: "#set ($parsedPayload = $util.parseJson($input.json('$.output')))\n\
+                $parsedPayload"
+        requestTemplates:
+          application/json: "#set($data = $util.escapeJavaScript($input.json('$')))\n\
+            \    {\n        \"input\": \"$data\",\n        \"stateMachineArn\": \"\
+            arn:aws:states:${AWS::Region}:${AWS::AccountId}:stateMachine:StateMachineExpressSync\"\n    }"
+        passthroughBehavior: "when_no_templates"
+components:
+  schemas:
+    Empty:
+      title: "Empty Schema"
+      type: "object"
diff --git a/...functions-express-sync-bedrock-sam/apigw-rest-stepfunctions-express-sync-bedrock-sam.json b/...functions-express-sync-bedrock-sam/apigw-rest-stepfunctions-express-sync-bedrock-sam.json
@@ -0,0 +1,101 @@
+{
+    "title": "Prompt Chaining with Amazon API Gateway and AWS Step Functions",
+    "description": "Prompt Chaining no-code example with Amazon API Gateway HTTP API, AWS Step Functions and Amazon Bedrock.",
+    "language": "",
+    "level": "200",
+    "framework": "SAM",
+    "introBox": {
+        "headline": "How it works",
+        "text": [
+            "The provided example demonstrates the deployment of a serverless application using the AWS Serverless Application Model (SAM) template.",
+            "This serverless application includes an Amazon API Gateway HTTP API endpoint that is connected to an AWS Step Functions state machine.",
+            "The state machine is designed to invoke an Express workflow synchronously.",
+            "It utilizes AWS Step Functions' intrinsic functions to chain two prompts, which are then used to invoke the Amazon Bedrock language model.",
+            "The output from the state machine execution is returned to the client within 29 seconds through the HTTP API.",
+            "This no-code example showcases how the results from the first prompt can be used to provide context for the second prompt, allowing the language model to deliver a highly-curated response.",
+            "By chaining these prompts, the system can leverage the capabilities of the language model to generate more meaningful and contextual outputs."
+        ]
+    },
+    "gitHub": {
+        "template": {
+            "repoURL": "https://github.com/aws-samples/serverless-patterns/tree/main/apigw-rest-stepfunctions-express-sync-bedrock-sam",
+            "templateURL": "serverless-patterns/apigw-rest-stepfunctions-express-sync-bedrock-sam",
+            "projectFolder": "apigw-rest-stepfunctions-express-sync-bedrock-sam",
+            "templateFile": "template.yml"
+        }
+    },
+    "resources": {
+        "bullets": [
+            {
+                "text": "Amazon API Gateway",
+                "link": "https://aws.amazon.com/api-gateway/"
+            },
+            {
+                "text": "Amazon Bedrock",
+                "link": "https://aws.amazon.com/bedrock/"
+            },
+            {
+                "text": "AWS Step Functions Intrinsic functions",
+                "link": "https://docs.aws.amazon.com/step-functions/latest/dg/amazon-states-language-intrinsic-functions.html"
+            },
+            {
+                "text": "Synchronous and Asynchronous Express Workflows",
+                "link": "https://docs.aws.amazon.com/step-functions/latest/dg/concepts-express-synchronous.html"
+            }
+        ]
+    },
+    "deploy": {
+        "text": [
+            "sam deploy"
+        ]
+    },
+    "testing": {
+        "text": [
+            "See the GitHub repo for detailed testing instructions."
+        ]
+    },
+    "cleanup": {
+        "text": [
+            "<code>sam delete</code>"
+        ]
+    },
+    "authors": [
+        {
+            "name": "Mike Hume",
+            "image": "https://media.licdn.com/dms/image/D4E03AQEiUfmBiUOw_A/profile-displayphoto-shrink_200_200/0/1718324029612?e=1727308800&v=beta&t=ybhm76l-CP5xcUsHbdq2IaJOlfyycvQ6gNwuCSd3Z0w",
+            "bio": "AWS Senior Solutions Architect & UKPS Serverless Lead.",
+            "linkedin": "michael-hume-4663bb64",
+            "twitter": ""
+        }
+    ],
+    "patternArch": {
+        "icon1": {
+            "x": 20,
+            "y": 50,
+            "service": "apigw",
+            "label": "API Gateway HTTP API"
+        },
+        "icon2": {
+            "x": 50,
+            "y": 50,
+            "service": "sfn",
+            "label": "AWS Step Functions"
+        },
+        "icon3": {
+            "x": 80,
+            "y": 50,
+            "service": "bedrock",
+            "label": "Amazon Bedrock"
+        },
+        "line1": {
+            "from": "icon1",
+            "to": "icon2",
+            "label": ""
+        },
+        "line2": {
+            "from": "icon2",
+            "to": "icon3",
+            "label": ""
+        }
+    }
+}
diff --git a/apigw-rest-stepfunctions-express-sync-bedrock-sam/example-pattern.json b/apigw-rest-stepfunctions-express-sync-bedrock-sam/example-pattern.json
@@ -0,0 +1,71 @@
+{
+  "title": "Prompt Chaining with Amazon API Gateway and AWS Step Functions.",
+  "description": "Prompt Chaining no-code example with Amazon API Gateway HTTP API, AWS Step Functions and Amazon Bedrock.",
+  "language": "",
+  "level": "200",
+  "framework": "SAM",
+  "introBox": {
+    "headline": "How it works",
+    "text": [
+      "The provided example demonstrates the deployment of a serverless application using the AWS Serverless Application Model (SAM) template.",
+      "This serverless application includes an Amazon API Gateway HTTP API endpoint that is connected to an AWS Step Functions state machine.",
+      "The state machine is designed to invoke an Express workflow synchronously.",
+      "It utilizes AWS Step Functions' intrinsic functions to chain two prompts, which are then used to invoke the Amazon Bedrock language model.",
+      "The output from the state machine execution is returned to the client within 29 seconds through the HTTP API.", 
+      "This no-code example showcases how the results from the first prompt can be used to provide context for the second prompt, allowing the language model to deliver a highly-curated response.", 
+      "By chaining these prompts, the system can leverage the capabilities of the language model to generate more meaningful and contextual outputs."
+    ]
+  },
+  "gitHub": {
+    "template": {
+      "repoURL": "https://github.com/aws-samples/serverless-patterns/tree/main/apigw-rest-stepfunctions-express-sync-bedrock-sam",
+      "templateURL": "serverless-patterns/apigw-rest-stepfunctions-express-sync-bedrock-sam",
+      "projectFolder": "apigw-rest-stepfunctions-express-sync-bedrock-sam",
+      "templateFile": "template.yml"
+    }
+  },
+  "resources": {
+    "bullets": [
+      {
+        "text": "Amazon API Gateway",
+        "link": "https://aws.amazon.com/api-gateway/"
+      },
+      {
+        "text": "Amazon Bedrock",
+        "link": "https://aws.amazon.com/bedrock/"
+      },
+      {
+        "text": "AWS Step Functions Intrinsic functions",
+        "link": "https://docs.aws.amazon.com/step-functions/latest/dg/amazon-states-language-intrinsic-functions.html"
+      },
+      {
+        "text": "Synchronous and Asynchronous Express Workflows",
+        "link": "https://docs.aws.amazon.com/step-functions/latest/dg/concepts-express-synchronous.html"
+      }
+    ]
+  },
+  "deploy": {
+    "text": [
+      "sam deploy"
+    ]
+  },
+  "testing": {
+    "text": [
+      "See the GitHub repo for detailed testing instructions."
+    ]
+  },
+  "cleanup": {
+    "text": [
+      "<code>sam delete</code>"
+    ]
+  },
+  "authors": [
+    {
+      "name": "Mike Hume",
+      "image": "https://media.licdn.com/dms/image/D4E03AQEiUfmBiUOw_A/profile-displayphoto-shrink_200_200/0/1718324029612?e=1727308800&v=beta&t=ybhm76l-CP5xcUsHbdq2IaJOlfyycvQ6gNwuCSd3Z0w",
+      "bio": "AWS Senior Solutions Architect & UKPS Serverless Lead.",
+      "linkedin": "michael-hume-4663bb64",
+      "twitter": ""
+    }
+  ]
+}
diff --git a/apigw-rest-stepfunctions-express-sync-bedrock-sam/images/apigw.png b/apigw-rest-stepfunctions-express-sync-bedrock-sam/images/apigw.png
diff --git a/apigw-rest-stepfunctions-express-sync-bedrock-sam/statemachine/stateMachine.asl.json b/apigw-rest-stepfunctions-express-sync-bedrock-sam/statemachine/stateMachine.asl.json
@@ -0,0 +1,50 @@
+{
+  "Comment": "An example of using Bedrock to chain prompts and their responses together.",
+  "StartAt": "Invoke model with first prompt",
+  "States": {
+    "Invoke model with first prompt": {
+      "Type": "Task",
+      "Resource": "arn:aws:states:::bedrock:invokeModel",
+      "Parameters": {
+        "ModelId": "cohere.command-text-v14",
+        "Body": {
+          "prompt.$": "$.prompt_one",
+          "max_tokens": 2000
+        },
+        "ContentType": "application/json",
+        "Accept": "*/*"
+      },
+      "Next": "Add first result to conversation history",
+      "ResultPath": "$.result_one",
+      "ResultSelector": {
+        "result_one.$": "$.Body.generations[0].text"
+      }
+    },
+    "Add first result to conversation history": {
+      "Type": "Pass",
+      "Next": "Invoke model with second prompt",
+      "Parameters": {
+        "convo_one.$": "States.Format('{}',$.result_one.result_one)"
+      },
+      "ResultPath": "$.convo_one"
+    },
+    "Invoke model with second prompt": {
+      "Type": "Task",
+      "Resource": "arn:aws:states:::bedrock:invokeModel",
+      "Parameters": {
+        "ModelId": "cohere.command-text-v14",
+        "Body": {
+          "prompt.$": "States.Format('{}\n{}','Human: Now write a short story based on the following. Assistant:', $.convo_one.convo_one)",
+          "max_tokens": 1000
+        },
+        "ContentType": "application/json",
+        "Accept": "*/*"
+      },
+      "ResultSelector": {
+        "result_two.$": "$.Body.generations[0].text"
+      },
+      "ResultPath": "$.result_two",
+      "End": true
+    }
+  }
+}