[refurb] Count codepoints not bytes for slice-to-remove-prefix-or-suffix (FURB188)

[dylwil3]

This PR fixes the calculation of string length for the purposes of verifying when to suggest removeprefix/removesuffix (FURB188). Before, we used text_len which was counting bytes rather than codepoints (chars) and therefore disagreed with Python's len for non-ASCII text.

Closes #13620

[github-actions]

ruff-ecosystem results

Linter (stable)

✅ ecosystem check detected no linter changes.

Linter (preview)

✅ ecosystem check detected no linter changes.

Formatter (stable)

✅ ecosystem check detected no format changes.

Formatter (preview)

✅ ecosystem check detected no format changes.

[dscorbett]

Another subtlety worth testing is strings with surrogates. In Python, each surrogate counts as 1 and surrogate pairs are not special so they count as 2; for example, len("\ud800\udc00") == 2 whereas len("\U00010000") == 1. I don’t know how Ruff distinguishes those two example strings or if chars().count() matches what Python does.

[dylwil3]

Another subtlety worth testing is strings with surrogates. In Python, each surrogate counts as 1 and surrogate pairs are not special so they count as 2; for example, len("\ud800\udc00") == 2 whereas len("\U00010000") == 1. I don’t know how Ruff distinguishes those two example strings or if chars().count() matches what Python does.

TIL @dscorbett - neat! Added a test for this, and it appears to be handled correctly (I think this happens in the guts of the parser, so by the time I'm looking at string_val here, the subtleties have been smoothed out).

[dscorbett]

I think the reason it works is that Ruff’s representation of a Python string as a Rust string replaces surrogates with replacement characters. That is fine for counting the code points but could be a problem for other rules.

[MichaReiser]

Nice, thanks. I only have two nit comments.

[MichaReiser]

I suggest converting to a u64 considering that you have to use usize::try_from anyways (for 32 bit platforms)

Suggested change

	.and_then(ast::Int::as_u32)
	.and_then(|x| usize::try_from(x).ok())
	.and_then(ast::Int::as_u64)
	.and_then(|x| usize::try_from(x).ok())

[MichaReiser]

Or you could consider adding a as_usize method to ast::Int

[dylwil3]

went with the latter

[MichaReiser]

Suggested change

	.is_some_and(|x| x == string_val.to_str().chars().count()),
	.is_some_and(|x| x == string_val.chars().count()),

[MichaReiser]

Suggested change

	.is_some_and(|x| x == string_val.to_str().chars().count())
	.is_some_and(|x| x == string_val.chars().count())