Add ensure_ascii option

[Viicos]

Change Summary

Part of pydantic/pydantic#11202.

Related issue number

Checklist

• Unit tests for the changes exist
• Documentation reflects the changes where applicable
• Pydantic tests pass with this pydantic-core (except for expected changes)
• My PR is ready to review, please add a comment including the phrase "please review" to assign reviewers

[Viicos]

As per @cetra3's suggestion. The match statement in to_json_bytes isn't going to scale well, so maybe an alternative API should be used in the future.

[codspeed-hq]

CodSpeed Performance Report

Merging #1689 will not alter performance

_{Comparing ensure-ascii (27a843b) with main (9a25aa6)}

Summary

✅ 157 untouched benchmarks

[Viicos]

Looks like a false positive from Clippy? There are already a bunch.

[D-stefaang]

Any chance you'd follow the core python default ensure_ascii=true here? That'd be a breaking change so that's unlikely.
Maybe a mention in the documentation?
https://docs.python.org/3/library/json.html#json.dump

FYI, the httpx library handles unicode nicely, the underlying requests library doesn't... unless explicitly sending UTF-8 bytes.

In [8]: import httpx

In [9]: data = g.model_dump_json()

In [10]: httpx.post('https://httpbin.org/post', data=data)
[info     ] HTTP Request: POST https://httpbin.org/post "HTTP/1.1 200 OK" [httpx]
Out[10]: <Response [200 OK]>

In [11]: import requests

In [12]: requests.post('https://httpbin.org/post', data=data)
---------------------------------------------------------------------------
UnicodeEncodeError 
...
UnicodeEncodeError: 'latin-1' codec can't encode characters in position 110-111: Body ('湫莓') is not valid Latin-1. Use body.encode('utf-8') if you want to send it encoded in UTF-8.

In [13]: requests.post('https://httpbin.org/post', data=data.encode())
Out[13]: <Response [200 OK]>

[Viicos]

Any chance you'd follow the core python default ensure_ascii=true here?

This can only be done in Pydantic V3, and still I don't know if the motivation is strong enough to do the change.