0.28.1

• Fixes an issue when attempting to call reading functions on a multipart web request in a handler would previously result in an "Could not find starting boundary in ..." error. This fix will now again make it possible to upload files as multipart web requests, and read those files from await request.post() in service http handlers. #1814

0.28.0

• Supports aiohttp 3.10.x versions. Dropped support for aiohttp versions prior to 3.9.5.
• Enforces more recent versions of dependencies for OTEL (>=1.27.0, >=0.48b0) when installed with opentelemetry extras.
• Enforces more recent versions of protobuf (>=4.25.0) when installed with protobuf extras.
• Requires yarl to use version 1.16.0+.
• Support for Python 3.13. Python 3.13 has been added to test matrix and trove classifiers.
• Dropped support for Python 3.8.

0.27.2

• Correction of message attributes type definition. Now also aligns better with updated types-aiobotocore definitions.

0.27.1

• Support for aiobotocore 2.12.x releases and 2.13.x releases.
• Support for recent versions of OTEL prometheus exporter (opentelemetry-exporter-prometheus).
• Updated tests run via GHA to use latest version of localstack.

0.27.0

Send messages directly to SQS queues (SQS.SendMessage)

• Added the function tomodachi.sqs_send_message (tomodachi.aws_sns_sqs_send_message) to send a message directly to an SQS queue. It mostly works the same way as the tomodachi.sns_publish (tomodachi.aws_sns_sqs_publish) function, but sends a message directly to the specified SQS queue.

  This is useful when you know the recipient queue of a message, for example when manually sending messages to a dead-letter queue or to do command-like messaging instead of event-based messaging.

• The message body of tomodachi.sqs_send_message calls is by default built using the tomodachi.transport.aws_sns_sqs.MessageBodyFormatter implementation, which will encode the message body as JSON and embed it within the "Message" property also using the service specified message envelope functionality.

  The message includes some additional properties, mimicing how SNS notifications are represented when they are sent to subscribed SQS queues (with the exception that these messages are of type "Message" to differentiate from SNS notifications that are by AWS typed "Notification".

  For custom or more advanced setups requiring more flexibility, the implementation can be overridden by providing a callable to the message_body_formatter keyword argument of sqs_send_message calls. The callable receives a message context object and returns the raw message (as a string) that will be sent to SQS as the message body.

  Setting message_body_formatter to None will disable default message body formatting (and incidentally also the message envelope builder), causing the message body to be as is specified within the data argument to the sqs_send_message call.

• Trying to send messages to an SQS queue that does not exist will result in a tomodachi.transport.aws_sns_sqs.QueueDoesNotExistError exception being raised, which can be caught also by tomodachi.transport.aws_sns_sqs.AWSSNSSQSException, botocore.exceptions.ClientError or <botocore_sqs_client>.exceptions.QueueDoesNotExist.

Message attributes' improvements and fixes

• Receives and propagates message attributes for all messages received on a queue, which means message attribute propagation will work for messages published to SNS, using either raw message delivery or not, and/or messages that were directly sent to SQS via SQS.SendMessage calls.

  Currently, messages on SQS queues consumed and parsed by tomodachi services still need to be JSON formatted and be embedded within the "Message" property (which is how SNS notification messages are represented by default, unless a queue is setup for raw message delivery).

• Fixes a bug that was causing message attributes in binary form to be invalidly processed.

OTEL updates

• OTEL auto instrumentation for the new tomodachi.sqs_send_message functionality.
• OTEL attributes for AWS SNS+SQS messaging have been updated according to the latest OTEL semantic conventions, where the original publish destination is recorded as messaging.destination_publish.name.
• OTEL metrics for AWS SNS+SQS messaging uses the new metric instrument name messaging.aws_sqs.duration, which was previously named messaging.amazonsqs.duration.

Type hints and input validation

• Additional support for the types-aiobotocore-* packages to provide improved type hints internally.
• AWS credentials provided to the aiobotocore connector are using stricter type hint annotations.
• Stricter argument validation when requesting aiobotocore connector clients to prevent issues caused by a mistyped AWS service, protecting against accidental use of wrong client for a specific service.

Handler arguments

• Added message_type (str | None) to the list of keyword argument provided transport values that can be used in function signatures for AWS SNS+SQS handlers. Holds the value "Notification" for messages received as part of an SNS notification, unless the queue uses raw message delivery. In other cases the value is set to the value from a message body's JSON property "Type" if it exists.
• Tailored for more advanced workflows, where more flexibility is needed, raw_message_body (str) has also been added to the list of keyword argument provided transport values for AWS SNS+SQS handlers. The raw_message_body value is set to the full content (non-decoded, as a string) from a received message' "Body", which can be used to implement custom listener with greater access to the raw message data.
• Fixes an issue where the topic argument to message handlers in functions was always populated with the value of the topic argument to the @tomodachi.aws_sns_sqs decorator, even if the message was sent to a different topic. The topic argument to message handlers in functions will now be populated with the actual topic name destination to where the message was published, if published via SNS. If the message was not published to a topic, the value is set to empty string.

Other

• Fixes issue where different OTEL tracers for the same TracerProvider could end up with different resource attributes, iff a service class' name attribute's value were to be used as the service.name OTEL resource attribute (instead set to "unknown_service" for tracers that were created before the service initialized).
• Multi-service execution won't accurately distinguish the service name of OTEL tracers, OTEL meters and OTEL loggers. In the rare case where there's multiple tomodachi services started within the same Python process, the name from the first instrumented service class will now be used as the service.name OTEL resource attribute for all tracers derived from the same tracer provider. The recommended solution if this is an issue, is to split the services into separate processes instead.
• Refactoring of tomodachi.aws_sns_sqs.get_queue_url and tomodachi.aws_sns_sqs.get_queue_url_from_arn with better support for additional types of input and/or potentially prefixing of queues. The function also now caches the queue URL to avoid unnecessary calls to the AWS API.
• Support for aiobotocore 2.10.x releases and 2.11.x releases.
• Documented the lifecycle termination process for services, for example when they receive a SIGINT or SIGTERM signal. Added details describing how handlers are awaited and for how long as part of the graceful termination sequence.
• Converted repo documentation from RST format to Markdown. It finally happened.

0.26.4

• Fixes an issue with the custom backport class of ConsoleRenderer extending structlog.dev.ConsoleRenderer (which caused an exception to be raised upon service start, when using structlog 23.3.0 or 24.1.0).
• Uses the native ConsoleRenderer class from structlog when a modern version of structlog is used (23.3.0 or newer).

0.26.3

• Updated version constraints of the opentelemetry-exporter-prometheus dependency to use the correct (non-yanked) version range. Now >=0.40b0,<1.0.0 from July 2023 and forward. Previously constrained to use the older, now yanked, 1.12.0rc1 version from May 2022. The 0.x releases of opentelemetry-exporter-prometheus are more recent than the deprecated 1.x releases.
• Support for aiobotocore 2.8.x releases and 2.9.x releases.

0.26.2

• Support for Python 3.12. Python 3.12 has been added to test matrix and trove classifiers.
• Supports aiohttp 3.9.x versions.
• Updated OpenTelemetry distro to handle MetricReaders and MetricExporters in a more flexible way (similar to how it was changed to be done in opentelemetry-sdk 1.21.0).
• Log entries from tomodachi.logging on OTEL instrumented services will now include TraceContext (span_id, trace_id and parent_span_id) on non-recording (but valid) spans.

0.26.1

• Passes FIFO message attributes (message_deduplication_id and message_group_id) to tomodachi handler functions as keyword argument provided transport values. Details in #1810. (github: @filipsnastins)
• Uses less strict version constraints on OpenTelemetry dependencies to make it easier to use newer versions of OpenTelemetry libraries together with tomodachi services.

0.26.0

Summary

🎉 This release is mainly focused on improving observability when operating servies. It introduces OpenTelemetry instrumentation and a completely refactored interface for logging.

New and updated

• Major refactoring of how logging is done, introducing tomodachi.get_logger() built on structlog. Contextual logger available for all handlers, etc.
• Provides the option of hooking in tomodachi to a custom logger the user provides.
• Adds instrumentation for opentelemetry (OTEL / OpenTelemetry) that can be enabled if tomodachi is installed using the opentelemetry extras. The OpenTelemetry instrumentation usage is explained in detail in #1790.
• OTEL auto instrumentation can be achieved by starting services using either the tomodachi run argument --opentelemetry-instrument (equivalent to setting env: TOMODACHI_OPENTELEMETRY_INSTRUMENT=1) or using the opentelemetry-instrument CLI.
• An experimental meter provider for exporting OpenTelemetry metrics on a Prometheus server can be used by installing the opentelemetry-exporter-prometheus extras and using the OTEL_PYTHON_METER_PROVIDER=tomodachi_prometheus environment value in combination with OTEL instrumentation.
• Adds the option to enable exemplars in the Prometheus client for OpenTelemetry metrics, to be able to link to traces from collected metrics in Grafana, et al.
• The HTTP body for requests with a body is read before the handler is called and if the connection was closed prematurely before the body could be read, the request will be ignored.
• Replaces the banner shown when starting a service without --production. The banner now includes the operating system, architecture, which Python runtime from which virtualenv is used, etc. in order to aid debugging during development for issues caused by environment misconfiguration.
• Updated the CLI usage output from --help.
• Added a value tomodachi.__build_time__ which includes the timestamp when the build for the installed release was done. The time that has passed since build time will be included in the start banner.
• Makes use of asyncio tasks instead of simply awaiting the coroutines so that the context from contextvars will be propagated correctly and not risk being corrupted by handlers.
• Added an internal lazy meta importer to ease renaming and deprecations of modules.
• Added additional lazy loading of submodules.
• Each argument for tomodachi run is now accompanied with an environment variable to do the same. For example --log-level warning can be achieved by setting TOMODACHI_LOG_LEVEL=warning.
• Updated documentation with additional information.

Potentially breaking changes

• The complete refactoring of logging changes how log entries are being emitted, both in the way that the event / message of the log records has changed, but also how a log handler is now also added to the logging.root logger on service start.
• Third party log records will if propagation is enabled also be processed in tomodachi.logging which may cause duplicate log output depending on how the third party logger is configured.
• Removed the log_setup() function that previously was added to the service object on class initialization and that was used to setup log output to disk.
• Tracebacks for uncaught exceptions are now extracted to only include frames relevant to the service application and not the internal tomodachi frames, which usually will be uninteresting for debugging.

Bug fixes

• Fixes exception catching of lifecycle handlers (_start_service, _started_service, etc.) which previously could stall a service raising an exception while starting, instead of exiting with a non-zero exit code.
• Bug fix for an issue which could cause the watcher to fail to restart the service after a syntax error was encountered.
• Adds some missing type hint annotations.
• Added additional logging of uncaught exceptions that previously may have been silenced.
• Fixed that the --log-level CLI argument value is actually applied to loggers.
• Fix for a race condition which could freeze a process if a service was manually stopped (interrupted with ctrl+c) before it had called its first lifecycle function (_start_service).

Deprecations

• Added deprecation warnings for more legacy functionality to give notice that those functions will be removed in a future release.
• The use of the log() function added to the service object is deprecated. Use the structlog logger from tomodachi.get_logger() instead.
• Using the RequestHandler.get_request_ip is deprecated. Instead use the tomodachi.get_forwarded_remote_ip() function.
• Deprecated the use of the CLI argument -c (--config) which could be used to set object attributes from a JSON file. A better pattern is to read optional config data from an environment variable.