Skip to content

Allow server SDKs to replace invalid UTF-8 character with '�' #3996

New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

Draft
wants to merge 3 commits into
base: main
Choose a base branch
from

Conversation

drganjoo
Copy link
Contributor

@drganjoo drganjoo commented Feb 4, 2025

When the input contains invalid UTF-8 characters (such as surrogate pairs or invalid character ranges), these characters will be replaced with the replacement character "�" (\uFFFD). This replacement only occurs if the codegen flag 'replaceInvalidUtf8' is set to true in smithy-build-template.json.

@drganjoo drganjoo force-pushed the fahadzub/replace-invalid-utf8 branch 2 times, most recently from 2e86d87 to 10fb59e Compare February 4, 2025 15:31
@drganjoo drganjoo force-pushed the fahadzub/replace-invalid-utf8 branch 4 times, most recently from 14fdd2e to 5ffe67d Compare February 4, 2025 16:27
@drganjoo drganjoo force-pushed the fahadzub/replace-invalid-utf8 branch from 5ffe67d to 5ec624b Compare February 4, 2025 16:28
Copy link

github-actions bot commented Feb 4, 2025

A new generated diff is ready to view.

A new doc preview is ready to view.

@landonxjames
Copy link
Contributor

We were considering something like this for the client a few months ago because some customers were encountering non-utf-8 headers from S3: #3830. It broke some of our protocol tests and we decided not to follow it up because there wasn't consistent cross-SDK guidance about how to handle these headers.

# for free to join this conversation on GitHub. Already have an account? # to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants