v1.29.2

Summary

This patch release fixes a bug in the Serverless Application Repository (SAR) Extras Layer for customers using Parser/Pydantic, where the module wasn't available.

Changes

🐛 Bug and hot fixes

• fix(deps): bump dev dep mako version to address CVE-2022-40023 (#1524) by @rubenfonseca
• fix(internal): SAR layer build not including pydantic (#1519) by @am29d

🔧 Maintenance

• fix(deps): bump dev dep mako version to address CVE-2022-40023 (#1524) by @rubenfonseca
• chore(deps): bump release-drafter/release-drafter from 5.20.1 to 5.21.0 (#1520) by @dependabot
• chore(deps-dev): bump mkdocs-material from 8.5.0 to 8.5.1 (#1521) by @dependabot
• chore(deps-dev): bump mypy-boto3-dynamodb from 1.24.60 to 1.24.74 (#1522) by @dependabot

This release was made possible by the following contributors:

@dependabot, @dependabot[bot], @rubenfonseca and Release bot

v1.29.1

Due to a problem with the release pipeline, we had to yank the v1.29.0 release from PyPI. To fix the problem we are releasing v1.29.1 with no additional changes.

Summary

This release adds a new major integration with Apache Kafka (both Amazon Managed Streaming for Apache Kafka (MSK) and self-managed clusters). We also extended the Event Source Data Classes to support CloudWatch dashboard custom widgets.

Apache Kafka

Powertools can now seamlessly consume events sourced from Apache Kafka (MSK, self-managed). Apache Kafka as an event source operates similarly to using Amazon Simple Queue Service (Amazon SQS) or Amazon Kinesis.

You can easily consume a Kafka event by using the KafkaEvent Data Class:

If you want extra flexibility and validation, you can also use the Pydantic models KafkaMskEventModel (for Apache MSK) and KafkaSelfManagedEventModel (for self-managed clusters):

If your Kafka event payload has a known structure, you could also leverage the KafkaEnvelope model.

Thank you @lyoung-confluent and @ran-isenberg for your contribution!

🌟 Would you like to see more integrations between Apache Kafka and Lambda Powertools? Drop us a line in the python channel of our Discord.

CloudWatch dashboards custom widgets

We also added support for handling events generated by CloudWatch dashboard custom widgets:

Big thanks to @sthuber90 for your contribution!

Changes

🌟New features and non-breaking changes

• feat(parser): add KafkaMskEventModel and KafkaSelfManagedEventModel (#1499) by @ran-isenberg
• feat(data-classes): add KafkaEvent and KafkaEventRecord (#1485) by @lyoung-confluent
• feat(event_sources): add CloudWatch dashboard custom widget event (#1474) by @sthuber90
• chore(ci): add workflow to suggest splitting large PRs (#1480) by @heitorlessa

📜 Documentation updates

• feat(parser): add KafkaMskEventModel and KafkaSelfManagedEventModel (#1499) by @ran-isenberg
• feat(data-classes): add KafkaEvent and KafkaEventRecord (#1485) by @lyoung-confluent
• feat(event_sources): add CloudWatch dashboard custom widget event (#1474) by @sthuber90

🔧 Maintenance

• chore(deps): bump flake8-bugbear from 22.8.23 to 22.9.11 (#1512) by @dependabot
• chore(deps): bump mkdocs-material from 8.4.3 to 8.4.4 (#1513) by @dependabot
• chore(deps): bump types-requests from 2.28.9 to 2.28.10 (#1508) by @dependabot
• chore(deps): bump aws-cdk-aws-apigatewayv2-integrations-alpha from 2.40.0a0 to 2.41.0a0 (#1510) by @dependabot
• chore(deps-dev): bump mypy-boto3-ssm from 1.24.39.post2 to 1.24.69 (#1509) by @dependabot
• chore(deps-dev): bump aws-cdk-lib from 2.40.0 to 2.41.0 (#1507) by @dependabot
• chore(deps-dev): bump aws-cdk-aws-apigatewayv2-integrations-alpha from 2.39.1a0 to 2.40.0a0 (#1496) by @dependabot
• chore(deps-dev): bump mkdocs-material from 8.4.2 to 8.4.3 (#1504) by @dependabot
• chore(deps): bump pydantic from 1.10.1 to 1.10.2 (#1502) by @dependabot
• chore(deps-dev): bump pytest from 7.1.2 to 7.1.3 (#1497) by @dependabot
• chore(deps-dev): bump black from 22.6.0 to 22.8.0 (#1494) by @dependabot
• chore(deps-dev): bump aws-cdk-lib from 2.39.1 to 2.40.0 (#1495) by @dependabot
• chore(maintenance): add discord link to first PR and first issue (#1493) by @rubenfonseca
• chore(deps): bump pydantic from 1.10.0 to 1.10.1 (#1491) by @dependabot
• chore(deps-dev): bump flake8-variables-names from 0.0.4 to 0.0.5 (#1490) by @dependabot
• chore(deps-dev): bump mkdocs-material from 8.4.1 to 8.4.2 (#1483) by @dependabot
• chore(ci): create reusable docs publishing workflow (#1482) by @heitorlessa
• chore(deps-dev): bump mypy-boto3-dynamodb from 1.24.55.post1 to 1.24.60 (#1481) by @dependabot
• chore(ci): add workflow to suggest splitting large PRs (#1480) by @heitorlessa
• chore(ci): add linter for GitHub Actions as pre-commit hook (#1479) by @heitorlessa
• chore(deps-dev): bump black from 21.12b0 to 22.8.0 (#1515) by @dependabot
• chore(deps-dev): bump mkdocs-material from 8.4.4 to 8.5.0 (#1514) by @dependabot
• chore(ci): add linter for GitHub Actions as pre-commit hook (#1479) by @heitorlessa

This release was made possible by the following contributors:

@dependabot, @dependabot[bot], @heitorlessa, @leandrodamascena, @lyoung-confluent, @ran-isenberg, @rubenfonseca, @sthuber90 and Release bot

v1.28.0

Summary

This release brings major updates: upcoming v2 to drop Python 3.6 support, Lambda Function URL support in Parser(Pydantic), and.... 🥁 🥁 🥁 our new public Discord Server - join us

Oh hey, please welcome Leandro as our new full-time maintainer - @leandrodamascena🎉

Lambda Function URL support in Parser

This release adds support for Lambda Function URL for Parser (Pydantic Model) - big thanks to @ran-isenberg. It extends the work previously done in v1.27.0 for Event Source Data Classes.

Upcoming v2 and breaking changes

We’ve got a new RFC for our first major version v2.0.0 - aiming to end of September. We’re striving to make the bare minimum breaking changes given the timeframe we have. Please get acquainted with the breaking changes (e.g., legacy sqs_batch_processor, Event Handler to support multi-value headers) and do share your feedback.

Improved documentation code snippets

We know that a good documentation is critical for adopting any open-source project. We continue to make progress in rewriting code snippets to be more realistic, and apply our code quality standards like we do for any other change.

In this release, we've refactored the JMESPath Functions,, Validation, Typing, and Middleware factory documentation to include real examples of using these features. You can follow progress on upcoming refactoring in our board of activities.

👀 Take a look at this gigantic yet realistic all-in-one middleware combining Powertools features.

Here’s a sneak peek:

End-to-testing (E2E)

We’ve published our internal framework to create and run E2E testing via GitHub Actions. This allow us to start working on further complement our functional testing, and soon begin wider coverage on integration tests. We’ll be documenting how customers can contribute E2E soon - you can find the internal details here.

Changes

🌟New features and non-breaking changes

• feat(parser): add support for Lambda Function URL (#1442) by @ran-isenberg

📜 Documentation updates

• docs(typing): snippets split, improved, and lint (#1465) by @leandrodamascena
• docs(middleware-factory): snippets split, improved, and lint (#1451) by @leandrodamascena
• chore(batch): deprecate sqs_batch_processor (#1463) by @rubenfonseca
• docs(validation): snippets split, improved, and lint (#1449) by @leandrodamascena
• feat(parser): add support for Lambda Function URL (#1442) by @ran-isenberg
• chore(deps): bump pydantic from 1.9.1 to 1.9.2 (#1448) by @dependabot
• docs(jmespath_util): snippets split, improved, and lint (#1419) by @leandrodamascena

🐛 Bug and hot fixes

• fix(ci): calculate parallel jobs based on infrastructure needs (#1475) by @heitorlessa

🔧 Maintenance

• chore(ci): prevent concurrent git update in critical workflows (#1478) by @heitorlessa
• fix(ci): calculate parallel jobs based on infrastructure needs (#1475) by @heitorlessa
• chore(deps-dev): bump flake8-bugbear from 22.8.22 to 22.8.23 (#1473) by @dependabot
• chore(tests): enable end-to-end test workflow (#1470) by @heitorlessa
• chore(governance): add Leandro as maintainer (#1468) by @leandrodamascena
• chore(tests): build and deploy Lambda Layer stack once (#1466) by @heitorlessa
• chore(batch): deprecate sqs_batch_processor (#1463) by @rubenfonseca
• chore(tests): refactor E2E logger to ease maintenance, writing tests and parallelization (#1460) by @heitorlessa
• docs(validation): snippets split, improved, and lint (#1449) by @leandrodamascena
• chore(tests): refactor E2E tracer to ease maintenance, writing tests and parallelization (#1457) by @heitorlessa
• chore(deps): bump release-drafter/release-drafter from 5.20.0 to 5.20.1 (#1458) by @dependabot
• chore(deps): bump pydantic from 1.9.1 to 1.9.2 (#1448) by @dependabot

This release was made possible by the following contributors:

@am29d, @dependabot, @dependabot[bot], @heitorlessa, @leandrodamascena, @peterschutt, @ran-isenberg, @rubenfonseca, @sthuber90 and Release bot

v1.27.0

Summary

This is our feature release after our extended pause, we couldn't be more excited to bring you: Lambda Function URL, Metrics now supports up to 29 dimensions, and Pre-configured GitPod.io dev environment for contributors.

Huge thanks to a new contributor: @leandrodamascena

Event Handler - Lambda Function URL

You can now use LambdaFunctionUrlResolver to seamless develop with Lambda Function URL. Developer experience remains exactly the same for other REST Event Handlers, making it easier to switch between Application Load Balancer, and API Gateway (REST and HTTP) APIs.

Correlation ID support and documentation have been documented accordingly - here's how it looks like!

Improved onboarding to first contribution

For new contributors and those new to Python, @leandrodamascena contributed a pre-configured Cloud-based development environment based on Gitpod.io product. When visiting our contribution guide, you will now see a link to instantly spin up a dev environment pre-configured with all dependencies, linters, and git pre-commit hooks to fast track reviews.

Increasing Metrics dimension limit to 29

Amazon CloudWatch Metrics made tons of improvements 2 days ago. This release takes advantage of the new metric dimension to increase our validation limit to 29 (MAX-1). By default, we add a service dimension to more easily identify metrics for a given service, hence the limit being 29 and not 30.

Next release

We plan on closing the gap on releases with Lambda Layer versioning by automatically generating documentation for all regions. This means publishing 5 additional regions customers asked for support and making it easier to find out which Lambda Layer has which PyPi version. Beyond that, docs and Event Handler improvements, and maybe a surprise if time allows.

Changes

🌟New features and non-breaking changes

• feat(metrics): update max user-defined dimensions from 9 to 29 (#1417) by @leandrodamascena
• feat(ci): create reusable changelog generation (#1418) by @heitorlessa
• feat(event_handlers): Add support for Lambda Function URLs (#1408) by @rubenfonseca

📜 Documentation updates

• feat(metrics): update max user-defined dimensions from 9 to 29 (#1417) by @leandrodamascena
• feat(event_handlers): Add support for Lambda Function URLs (#1408) by @rubenfonseca
• chore(docs): remove pause sentence from roadmap (#1409) by @rubenfonseca

🐛 Bug and hot fixes

• fix(event_sources): add test for Function URL AuthZ (#1421) by @rubenfonseca

🔧 Maintenance

• fix(event_sources): add test for Function URL AuthZ (#1421) by @rubenfonseca
• chore(deps): bump constructs from 10.1.1 to 10.1.66 (#1414) by @dependabot
• chore(docs): remove pause sentence from roadmap (#1409) by @rubenfonseca
• chore(governance): remove devcontainer in favour of gitpod.io (#1411) by @leandrodamascena
• chore(governance): add pre-configured dev environment with GitPod.io to ease contributions (#1403) by @leandrodamascena
• chore(deps): bump constructs from 10.1.1 to 10.1.65 (#1407) by @dependabot
• chore(deps-dev): bump types-requests from 2.28.6 to 2.28.7 (#1406) by @dependabot
• chore(deps): bump constructs from 10.1.1 to 10.1.64 (#1405) by @dependabot
• chore(deps): bump attrs from 21.4.0 to 22.1.0 (#1397) by @dependabot
• chore(deps): bump constructs from 10.1.1 to 10.1.63 (#1402) by @dependabot
• chore(deps-dev): bump types-requests from 2.28.5 to 2.28.6 (#1401) by @dependabot
• chore(deps): bump constructs from 10.1.1 to 10.1.60 (#1399) by @dependabot
• chore(deps): bump jsii from 1.57.0 to 1.63.2 (#1400) by @dependabot

This release was made possible by the following contributors:

@dependabot, @dependabot[bot], @heitorlessa, @leandrodamascena, @rubenfonseca, Docs and Release bot

v1.26.7

Summary

This release adds a change to the idempotency utility, expiring in-progress executions if the Lambda function times out halfway through.

The new feature requires no code changes. However, if you're using just the idempotent_function decorator, you will want to register the Lambda context on your handler to take advantage of this feature:

To prevent against extended failed retries when a Lambda function times out, Powertools now calculates and includes the remaining invocation available time as part of the idempotency record. If a second invocation happens after this timestamp, and the record is marked as INPROGRESS, we will execute the invocation again as if it was in the EXPIRED state (e.g, expire_seconds field elapsed).

We also enabled static analysis (mypy) on the new statically typed documentation snippets - big thanks to @VictorCovalski for spotting one issue in Event Handler that later led to 50 other fixes.

Oh! Did I also tell you we’ve got a new full time maintainer? Big welcome to Ruben 😉 https://twitter.com/rubenfonseca

Changes

🌟New features and non-breaking changes

• feat(idempotency): handle lambda timeout scenarios for INPROGRESS records (#1387) by @rubenfonseca

📜 Documentation updates

• feat(idempotency): handle lambda timeout scenarios for INPROGRESS records (#1387) by @rubenfonseca
• docs(examples): enforce and fix all mypy errors (#1393) by @heitorlessa

🐛 Bug and hot fixes

• fix(event_handlers): ImportError when importing Response from top-level event_handler (#1388) by @VictorCovalski

🔧 Maintenance

• chore(deps): bump jsii from 1.57.0 to 1.63.1 (#1390) by @dependabot
• chore(deps): bump constructs from 10.1.1 to 10.1.59 (#1396) by @dependabot
• chore(deps-dev): bump flake8-isort from 4.1.1 to 4.1.2.post0 (#1384) by @dependabot
• docs(examples): enforce and fix all mypy errors (#1393) by @heitorlessa
• chore(ci): drop 3.6 from workflows (#1395) by @heitorlessa
• chore(maintainers): add Ruben as a maintainer (#1392) by @rubenfonseca
• chore(ci): add conditional to skip pypi release (#1366) by @heitorlessa

This release was made possible by the following contributors:

@VictorCovalski, @dependabot, @dependabot[bot], @heitorlessa, @rubenfonseca and Release bot

v1.26.6

Third-time is the luck of the Irish (new security release mechanisms)

Summary

This patch release addresses an internal issue for Event Handler REST API compression feature when headers value is None -- Huge thanks for spotting and fixing it as a first-time contributor @tbuatois!!

We now have public Lambda Layers deployed across all AWS commercial regions (22) - stellar multi-month work by @am29d. We have one last work to complete before we share the additional ARNs with everyone -- automate documentation generation for every Layer Version::Region mapping.

Lastly on Operational Excellence, we've implemented further security layers for GitHub Actions following GitHub's Hardening Guide, and created a Maintenance Issue template to make tech debt work more visible to all customers and future maintainers.

Changes

🐛 Bug and hot fixes

• fix(event_handlers): handle lack of headers when using auto-compression feature (#1325) by @tbuatois

🔧 Maintenance

• chore(ci): use gh environment for beta and prod layer deploy (#1356) by @am29d
• chore(ci): remove core group from codeowners (#1358) by @heitorlessa
• chore(ci): use OIDC and encrypt release secrets (#1355) by @heitorlessa
• chore(ci): introduce codeowners (#1352) by @heitorlessa
• chore(ci): lockdown workflow_run by origin (#1350) by @heitorlessa
• chore(deps): bump constructs from 10.1.1 to 10.1.52 (#1343) by @dependabot
• chore(deps-dev): bump mypy-boto3-cloudwatch from 1.24.0 to 1.24.35 (#1342) by @dependabot
• chore(layers): replace layers account secret (#1329) by @heitorlessa
• chore(governance): add new maintenance issue template for tech debt and governance (#1326) by @heitorlessa
• chore(layers): expand to all aws commercial regions (#1324) by @heitorlessa

This release was made possible by the following contributors:

@am29d, @dependabot, @dependabot[bot], @heitorlessa, @tbuatois and Release bot

v1.26.5

Summary

This patch release addresses a packaging regression that introduced mypy boto3 typing packages as prod dependencies instead of dev dependencies -- huge thanks to @tb102122 for spotting this 30m ago.

Everything else in this release is related to operational excellence on improving GitHub Actions workflow performance, and last steps to make our new Lambda Layer release pipeline ready.

Changes

🐛 Bug and hot fixes

• fix(deps): correct mypy types as dev dependency (#1322) by @heitorlessa
• fix(ci): keep layer version permission resource (#1318) by @am29d
• fix(ci): lambda layer workflow release version and conditionals (#1316) by @am29d
• fix(ci): remove additional quotes in PR action (#1317) by @heitorlessa
• fix(ci): install poetry before calling setup/python with cache (#1315) by @rubenfonseca

🔧 Maintenance

• fix(deps): correct mypy types as dev dependency (#1322) by @heitorlessa
• fix(ci): lambda layer workflow release version and conditionals (#1316) by @am29d
• chore(ci): adds caching when installing python dependencies (#1311) by @rubenfonseca

This release was made possible by the following contributors:

@am29d, @dependabot[bot], @heitorlessa, @rubenfonseca and Release bot

v1.26.4

Summary

This release continues our focus on Operational Excellence with several GitHub Actions to improve our automation on governance tasks (e.g., PR labelling, security, release automation, dependencies, etc.).

We also moved the Lambda Layer release pipeline to Github Actions (#1278). This will expand the official Lambda Layer to all commercial AWS regions, once operational. Huge thanks to @am29d!

In addition, we laid the groundwork for end-to-end (E2E) testing, starting with the core features (logger, metrics, tracer). Thank you @mploski!

Changes

🐛 Bug and hot fixes

• fix(parser): raise ValidationError when SNS->SQS keys are intentionally missing (#1299) by @rubenfonseca
• fix(ci): fixes typos and small issues on github scripts (#1302) by @rubenfonseca

🔧 Maintenance

• chore(ci): increase release automation and limit to one manual step (#1297) by @heitorlessa
• chore(deps): bump constructs from 10.1.1 to 10.1.49 (#1308) by @dependabot
• chore(deps): bump fastjsonschema from 2.15.3 to 2.16.1 (#1309) by @dependabot
• fix(parser): raise ValidationError when SNS->SQS keys are intentionally missing (#1299) by @rubenfonseca
• chore(deps): bump constructs from 10.1.1 to 10.1.46 (#1306) by @dependabot
• chore(ci): lockdown 3rd party workflows to pin sha (#1301) by @heitorlessa
• chore(ci): automatically add area label based on title (#1300) by @rubenfonseca
• chore(deps): bump jsii from 1.61.0 to 1.62.0 (#1294) by @dependabot
• chore(ci): add end to end testing mechanism (#1247) by @mploski
• chore(deps-dev): bump mypy-boto3-appconfig from 1.24.0 to 1.24.29 (#1295) by @dependabot

This release was made possible by the following contributors:

@am29d, @dependabot, @dependabot[bot], @heitorlessa, @mploski and @rubenfonseca

v1.26.3

Summary

This patch release addresses an issue with Lambda handlers with custom signatures (*args, **kwargs) when using logger.inject_lambda_context decorator - thanks to @RonRandemaker

Thanks to @kbakk, we now have officially documented how to inject standard logging record attributes when using Logger.

This release continues our focus on Operational Excellence with several GitHub Actions to improve our automation on governance tasks (e.g., PR labeling, Comments on issues closed, Markdown linting rules etc.).

Last but not least, we've started revamping all documentation examples, including making some of them more realistic to how customers would code.

Ah! Before I forget...huge thanks to new contributors @RonRademaker, @kbakk.

Changes

📜 Documentation updates

• docs(metrics): snippets split, improved, and lint (#1272) by @heitorlessa
• docs(logger): document enriching logs with logrecord attributes (#1271) by @kbakk
• docs(logger): snippets split, improved, and lint (#1262) by @heitorlessa
• docs(tracer): snippets split, improved, and lint (#1261) by @heitorlessa
• docs(tracer): split and lint code snippets (#1260) by @heitorlessa
• docs(lint): add markdownlint rules and automation (#1256) by @heitorlessa

🐛 Bug and hot fixes

• fix(logger): preserve std keys when using custom formatters (#1264) by @heitorlessa
• fix(logger): support additional args for handlers when injecting lambda context (#1276) by @RonRademaker

🔧 Maintenance

• chore(deps): bump dependabot/fetch-metadata from 1.3.2 to 1.3.3 (#1273) by @dependabot
• chore(deps-dev): bump flake8-bugbear from 22.6.22 to 22.7.1 (#1274) by @dependabot
• chore(deps): bump aws-xray-sdk from 2.9.0 to 2.10.0 (#1270) by @dependabot
• chore(deps): bump dependabot/fetch-metadata from 1.1.1 to 1.3.2 (#1269) by @dependabot
• docs(logger): snippets split, improved, and lint (#1262) by @heitorlessa
• docs(tracer): snippets split, improved, and lint (#1261) by @heitorlessa
• docs(tracer): split and lint code snippets (#1260) by @heitorlessa
• docs(lint): add markdownlint rules and automation (#1256) by @heitorlessa
• chore(deps-dev): bump flake8-bugbear from 22.4.25 to 22.6.22 (#1258) by @dependabot
• chore(deps-dev): bump mypy-boto3-dynamodb from 1.24.0 to 1.24.12 (#1255) by @dependabot
• chore(deps-dev): bump mypy-boto3-secretsmanager from 1.24.0 to 1.24.11.post3 (#1252) by @dependabot
• chore(test-perf): use pytest-benchmark to improve reliability (#1250) by @heitorlessa

This release was made possible by the following contributors:

@RonRademaker, @dependabot, @dependabot[bot], @heitorlessa and @kbakk

v1.26.2

Summary

This release addresses CORS Pre-flight issue when using Event Handler with ALB. Prior to this release, ALB would return HTTP 502 if Lambda would return a null body for a Pre-flight (HTTP OPTIONS) request. This release sets body to an empty string for compliance, and it doesn't affect API Gateway/AppSync features.

Shoutout to @francispeabody for spotting this behaviour, as CORS Specification doesn't dictate whether there should be a response or not leaving to each service to make their own judgement.

Changes

🐛 Bug and hot fixes

• fix(event-handler): body to empty string in CORS preflight (ALB non-compliant) (#1249) by @heitorlessa

🔧 Maintenance

• chore(deps): bump actions/setup-python from 3 to 4 (#1244) by @dependabot
• chore(deps-dev): bump mypy-boto3-appconfig from 1.23.0.post1 to 1.24.0 (#1233) by @dependabot
• chore(deps-dev): bump mypy-boto3-dynamodb from 1.23.0.post1 to 1.24.0 (#1234) by @dependabot
• chore(deps-dev): bump mypy-boto3-secretsmanager from 1.23.8 to 1.24.0 (#1232) by @dependabot
• chore(deps-dev): bump mypy-boto3-ssm from 1.23.0.post1 to 1.24.0 (#1231) by @dependabot
• chore(deps-dev): bump mypy from 0.960 to 0.961 (#1241) by @dependabot
• chore(metrics): revert dimensions test and split (#1243) by @heitorlessa

This release was made possible by the following contributors:

@am29d, @dependabot, @dependabot[bot] and @heitorlessa