Query params replacing space (%20) with underscore (_)

[DAVe3283]

Shlink version

4.2.1

PHP version

8.3.12

How do you serve Shlink

Docker image

Database engine

PostgreSQL

Database version

13.9

Current behavior

Spaces in query parameters are replaced by underscores.

For example, https://internal.fqdn/F4UVN?spaces are valid=true redirects to http://www.example.com/?spaces_are_valid=true
(note the underscores instead of spaces)

Expected behavior

Spaces need to be preserved in the query parameters.

For example, in Tableau, query parameters can be used to set filters on the linked dashboard, but most filters contain spaces in their names. When the spaces are changed to underscores, it breaks the filtering.

Minimum steps to reproduce

1. Create a short URL to any domain (http://www.example.com is good since it doesn't do anything with query parameters).
2. Ensure Forward query params on redirect is enabled.
3. Go to the short URL and add some query parameters with spaces (e.g. ?Some Filter=1)
4. Observe they have been replaced with underscores (e.g. ?Some_Filter=1)

[acelaya]

Yeah, I can reproduce that. That's weird.

I'll check why this is happening.

[Fibonacci78]

Some additional information on this bug. A subset of percent encoded characters in parameters are being replaced with the non-percent encoded versions. This replacement does not happen if the percent encoded character is in the URL that is being shortened.

Here is a test sample:

• %20 -> _
• %22 -> "
• %25 -> %25
• %2D -> -
• %2E -> _
• %3C -> <
• %3E -> >
• %5C -> %5C
• %5E -> %5E
• %5F -> _
• %60 -> %60
• %7B -> %7B
• %7C -> %7C

Space and period are placed with underscore, but others seem to be replaced with the correct equivalent character. I don't see any particulate logic to what is or is not being replaced.

[acelaya]

So, I have been investigating this, and at first I didn't realize you were setting spaces in the parameter names. If you set them in the values, it works as expected.

The reason this is happening is because, for historical reasons, PHP ensures query parameter names are valid variable names, because they get promoted as variables. That implies spaces are converted to underscores (see the first example here https://www.php.net/manual/en/function.parse-str.php), but there are other conversions as you found.

And the reason the query string needs to be parsed is because Shlink needs to merge the query params to be forwarded, with any potential query param in the long URL.

So for example, if the short URL is https://s.test/abc and it has been configured to redirect to https://example.com?foo=bar&baz=something, when you visit https://s.test/abc?foo=overwritten&new=param, Shlink will redirect to https://example.com?foo=overwritten&baz=something&new=param.

Additionally, there are some special query params that are stripped, etc. So Shlink needs to parse the raw query strings, apply some logic, and serialize the query string back.

However, I have access to the raw query string as well, so I could try to apply some custom deserialization logic that prevents this behavior and preserves query param names verbatim.

[acelaya]

This should now be fixed. I'll release v4.2.2 soon.

[acelaya]

I have just released Shlink 4.2.2 which includes the fix for this.

[Fibonacci78]

This update is working as expected. Thank you!