Skip to content

feat(idempotency): support dataclasses & pydantic models payloads #908

New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

Merged

Conversation

michaelbrewer
Copy link
Contributor

@michaelbrewer michaelbrewer commented Dec 18, 2021

Issue #, if available:

Description of changes:

Add support for serialization of python dataclasses, but checking if __dataclass_fields__ exists and calling dataclasses.asdict. This happens early on in the idempotency work flow.

UX

Dataclasses support

from dataclasses import dataclass

from aws_lambda_powertools.utilities.idempotency import (
    DynamoDBPersistenceLayer, IdempotencyConfig, idempotent_function)

dynamodb = DynamoDBPersistenceLayer(table_name="idem")
config =  IdempotencyConfig(
    event_key_jmespath="order_id",  # extract and hash this field only as the idempotency key
    use_local_cache=True,
)

@dataclass
class OrderItem:
    sku: str
    description: str

@dataclass
class Order:
    item: OrderItem
    order_id: int


@idempotent_function(data_keyword_argument="order", config=config, persistence_store=dynamodb)
def process_order(order: Order):
    return f"processed order {order.order_id}"


order_item = OrderItem(sku="fake", description="sample")
order = Order(item=order_item, order_id="fake-id")

# `order` parameter must be called as a keyword argument to work
process_order(order=order)

Pydantic support

from aws_lambda_powertools.utilities.idempotency import (
    DynamoDBPersistenceLayer, IdempotencyConfig, idempotent_function)
from aws_lambda_powertools.utilities.parser import BaseModel

dynamodb = DynamoDBPersistenceLayer(table_name="idem")
config =  IdempotencyConfig(
    event_key_jmespath="order_id",  # extract and hash this field only as the idempotency key
    use_local_cache=True,
)


class OrderItem(BaseModel):
    sku: str
    description: str


class Order(BaseModel):
    item: OrderItem
    order_id: int


@idempotent_function(data_keyword_argument="order", config=config, persistence_store=dynamodb)
def process_order(order: Order):
    return f"processed order {order.order_id}"


order_item = OrderItem(sku="fake", description="sample")
order = Order(item=order_item, order_id="fake-id")

# `order` parameter must be called as a keyword argument to work
process_order(order=order)

Checklist

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

@pull-request-size pull-request-size bot added the size/M Denotes a PR that changes 30-99 lines, ignoring generated files. label Dec 18, 2021
@codecov-commenter
Copy link

codecov-commenter commented Dec 18, 2021

Codecov Report

Merging #908 (4f08bab) into develop (9abeb32) will increase coverage by 0.00%.
The diff coverage is 100.00%.

Impacted file tree graph

@@           Coverage Diff            @@
##           develop     #908   +/-   ##
========================================
  Coverage    99.88%   99.88%           
========================================
  Files          118      118           
  Lines         5261     5272   +11     
  Branches       602      605    +3     
========================================
+ Hits          5255     5266   +11     
  Misses           2        2           
  Partials         4        4           
Impacted Files Coverage Δ
...wertools/utilities/idempotency/persistence/base.py 99.36% <ø> (-0.01%) ⬇️
...ws_lambda_powertools/utilities/idempotency/base.py 100.00% <100.00%> (ø)
aws_lambda_powertools/tracing/tracer.py 100.00% <0.00%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 9abeb32...4f08bab. Read the comment docs.

Copy link
Contributor

@heitorlessa heitorlessa left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤦 my comments were kept in pending... only one ask really - could you add a test with event_jmespath to use a given field of a dataclass?

@heitorlessa heitorlessa changed the title feat(idempontency): add support for serialization of python dataclasses feat(idempotency): add support for serialization of python dataclasses Dec 20, 2021
@heitorlessa heitorlessa added the p1 label Dec 20, 2021
@michaelbrewer michaelbrewer changed the title feat(idempotency): add support for serialization of python dataclasses feat(idempotency): add support for python dataclasses or pydantic function payloads Dec 22, 2021
@heitorlessa heitorlessa changed the title feat(idempotency): add support for python dataclasses or pydantic function payloads feat(idempotency): support dataclasses & pydantic models as payloads Dec 22, 2021
@heitorlessa heitorlessa changed the title feat(idempotency): support dataclasses & pydantic models as payloads feat(idempotency): support dataclasses & pydantic models payloads Dec 22, 2021
@pull-request-size pull-request-size bot added size/L Denotes a PR that changes 100-499 lines, ignoring generated files. and removed size/M Denotes a PR that changes 30-99 lines, ignoring generated files. labels Dec 22, 2021
@heitorlessa heitorlessa added the feature New feature or functionality label Dec 22, 2021
@boring-cyborg boring-cyborg bot added the documentation Improvements or additions to documentation label Dec 22, 2021
@heitorlessa heitorlessa added this to the 1.24.0 milestone Dec 22, 2021
@heitorlessa heitorlessa merged commit 321f493 into aws-powertools:develop Dec 22, 2021
@heitorlessa heitorlessa deleted the feat-idempotency-function-dataclasses branch December 22, 2021 12:29
heitorlessa added a commit to ran-isenberg/aws-lambda-powertools-python that referenced this pull request Dec 31, 2021
…tools-python into complex

* 'develop' of https://github.com/awslabs/aws-lambda-powertools-python: (24 commits)
  docs: consistency around admonitions and snippets (aws-powertools#919)
  chore(deps-dev): bump mypy from 0.920 to 0.930 (aws-powertools#925)
  fix(event-sources): handle dynamodb null type as none, not bool (aws-powertools#929)
  fix(apigateway): support @app.not_found() syntax & housekeeping (aws-powertools#926)
  docs: Added GraphQL Sample API to Examples section of README.md (aws-powertools#930)
  feat(idempotency): support dataclasses & pydantic models payloads (aws-powertools#908)
  feat(tracer): ignore tracing for certain hostname(s) or url(s) (aws-powertools#910)
  feat(event-sources): cache parsed json in data class (aws-powertools#909)
  fix(warning): future distutils deprecation (aws-powertools#921)
  docs(batch): remove leftover from legacy
  docs(layer): bump Lambda Layer to version 6
  chore: bump to 1.23.0
  docs(apigateway): add new not_found feature (aws-powertools#915)
  docs: external reference to cloudformation custom resource helper (aws-powertools#914)
  feat(logger): allow handler with custom kwargs signature (aws-powertools#913)
  chore: minor housekeeping before release (aws-powertools#912)
  chore(deps-dev): bump mypy from 0.910 to 0.920 (aws-powertools#903)
  feat(batch): new BatchProcessor for SQS, DynamoDB, Kinesis (aws-powertools#886)
  fix(parser): overload parse when using envelope (aws-powertools#885)
  fix(parser): kinesis sequence number is str, not int (aws-powertools#907)
  ...
heitorlessa added a commit to huonw/aws-lambda-powertools-python that referenced this pull request Dec 31, 2021
…tools-python into feature/905-datetime

* 'develop' of https://github.com/awslabs/aws-lambda-powertools-python:
  feat(feature_flags): support beyond boolean values (JSON values) (aws-powertools#804)
  docs: consistency around admonitions and snippets (aws-powertools#919)
  chore(deps-dev): bump mypy from 0.920 to 0.930 (aws-powertools#925)
  fix(event-sources): handle dynamodb null type as none, not bool (aws-powertools#929)
  fix(apigateway): support @app.not_found() syntax & housekeeping (aws-powertools#926)
  docs: Added GraphQL Sample API to Examples section of README.md (aws-powertools#930)
  feat(idempotency): support dataclasses & pydantic models payloads (aws-powertools#908)
  feat(tracer): ignore tracing for certain hostname(s) or url(s) (aws-powertools#910)
  feat(event-sources): cache parsed json in data class (aws-powertools#909)
  fix(warning): future distutils deprecation (aws-powertools#921)
# for free to join this conversation on GitHub. Already have an account? # to comment
Labels
documentation Improvements or additions to documentation feature New feature or functionality p1 size/L Denotes a PR that changes 100-499 lines, ignoring generated files. tests
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants