Lambdas implementing RequestStreamHandler are double decoding query parameters (Jersey)

[boardthatpowder]

• Framework version: 1.1
• Implementations: Jersey

Scenario

Client > APi Gateway > Lambda.

When using JerseyLambdaContainerHandler proxyStream(inputStream, outputStream, context) in a RequestStreamHandler implementation, query params are de-coded a second time before passed to the appropriate JAX-RS controller. This causes issues with strings containing +, which are encoded as %2B, decoded once as +, but then decoded a second time as .

Expected behavior

Passing %2B in a query parameter from a client should reach the JAX-RS controller as +.

Actual behavior

Passing %2B in a query parameter from a client reaches the JAX-RS controller as .

Steps to reproduce

Create an endpoint:

@GET
@Path("/test")
public Response getAll(@QueryParam("param") String param) {
  ...
}

Proxy the JAX-RS with a streaming lambda:

public class StreamLambdaHandler implements RequestStreamHandler {

	private static final ResourceConfig jerseyApplication = new JerseyConfig();

	private static final JerseyLambdaContainerHandler<AwsProxyRequest, AwsProxyResponse> handler = JerseyLambdaContainerHandler.getAwsProxyHandler(jerseyApplication);

	@Override
	public void handleRequest(InputStream inputStream, OutputStream outputStream, Context context) throws IOException {
		handler.proxyStream(inputStream, outputStream, context);
		outputStream.close();
	}
}

Expose the lambda via API Gateway. Make a request to GET /test?param=ab%2Bcd.

Full log output

[mvd199]

Using debugger I was able to track it down to the method:

org.glassfish.jersey.server.spi.internal.ParameterValueHelper.getParameterValues

It receives correct request “hello+world” and returns “hello world”.
It uses valueProviders argument to parse query parameters, I have not figured yet how those valueProviders configured.

Link to source:
https://github.com/jersey/jersey/blob/master/core-server/src/main/java/org/glassfish/jersey/server/spi/internal/ParameterValueHelper.java

[mvd199]

Was able to fix the issue by adding @Encoded attribute to the argument in my test handler.

public String test(@QueryParam("key") @Encoded String value)

[boardthatpowder]

After trying out @Encoded, it doesn't work in all situations.

Given the string /+=, then encode as %2F%2B%3D.

Using @Encoded, the JAX-RS controller receives it as %2F+=.

Without @Encoded, the JAX-RS controller receives it as / =`.

[sapessi]

I've done some tests calling through the browser. The API Gateway endpoint hits a Lambda function (proxy integration) that simply dumps the event. This is what I see:

• param1=/+=, non encoded in query string: Results in "/ =," as a value in the queryStringParameters map from API Gateway
• param1=%2F%2B%3D encoded as a browser would using https://www.urlencoder.org/: "/+="

The issue seems to be that when a + is not URL encoded API Gateway itself removes it from the string. The framework never has a chance to see the character. I think API Gateway's behavior is correct. The encoding rules for form values, which include query string values, state that the + sign is used to encode a space " ". For this reason, when you send API Gateway a literal + in the value of a query string parameter, it decodes it as a space. Wikipedia, RFC.

I'll start investigating the behavior of your second issue, where the "/" character is the only one that remains encoded when you ask Jersey for the @Encoded value. I'll update this comment once I have something.

[boardthatpowder]

Hey Stefano, I'm always encoding the query string that contains the /+= special characters before calling API Gateway, therefore I'm always sending it as %2F%2B%3D respectively. Regarding my previous comment, the responses I provided of using a JAX-RS controller both with and without the @Encoded annotation both had the encoded %2F%2B%3D value passed as the query string parameter to API Gateway.

Somewhere in the chain between API Gateway and Lambda, where multiple encoding/decoding may be happening before forwarding on, could it be possible that pure percent encoding/decoding is occurring in some places whereas in other places its using URLEncoder/URLDecoder which handles + in addition to percents?

[sapessi]

Here's an update on the second issue. You are calling API Gateway with the encoded value of %2F%2B%3D. API Gateway receives this and automatically decodes it. What I see in the map of query string parameters is /+=. Not to interfere with the values, I use Jersey's UriBuilder to set the query string with the exact same value I was passed from API Gateway:

• / = invalid character after the url path so it gets encoded automatically to %2F
• + = valid character, in a query string value this represents the encoded space literal " "
• = = valid character, this is allowed in a query string value as is

When you ask Jersey for the decoded value, it turns the %2F back into a / and reads the + as a space. When you ask for the encoded value it returns exactly what it's described above. The logic to encode characters is defined here - you are looking for the QUERY template.

I'll think this through and see if I can come up with a solution. It may be as simple as using Jersey's own encoder before setting the query string so that it is already encoded as Jersey expects it.

[sapessi]

Hey @boardthatpowder, any chance you can test with the 1.2-SNAPSHOT in the core branch of this repo? Simply download or clone the core branch to your local box then run mvn install. Once that's installed in your local cache you should be able to just change the dependency in your pom to 1.2-SNAPSHOT

[sapessi]

release 1.1.1 is going out to maven central