Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

feat: add Forwarded header parsing and real IP extraction with tests #2744

Open
wants to merge 2 commits into
base: master
Choose a base branch
from

Conversation

debug-ing
Copy link

For issue #2694

  • Added parser for the "Forwarded" header to extract the "for" field.
  • Implemented real IP extraction from the "Forwarded" headers.
  • Added unit tests to validate header parsing and real IP extraction functionality.

@debug-ing
Copy link
Author

cc : @aldas

@debug-ing
Copy link
Author

@aldas fixed lint error

@debug-ing
Copy link
Author

debug-ing commented Feb 12, 2025

cc: @aldas

1 similar comment
@debug-ing
Copy link
Author

cc: @aldas

@aldas
Copy link
Contributor

aldas commented Feb 15, 2025

Just for history sake. I did some research myself to understand what this header is and how it is implemented elsewhere

  1. RFC 7239 for this Forwarded header
    https://datatracker.ietf.org/doc/html/rfc7239

  2. Mozilla explanation of that header
    https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Forwarded

  3. Commit that add this header support to HAproxy
    haproxy/haproxy@b2bb925 HAproxy

    Current implementation proxy_http_parse_7239
    https://github.com/haproxy/haproxy/blob/32691e7c255a944224e0ac17d0f83f217be6c5d6/src/http_ext.c#L963

  4. Proposal to Go standard library proposal: net/http/httputil: support RFC 7239 Forwarded header in ReverseProxy
    proposal: net/http/httputil: support RFC 7239 Forwarded header in ReverseProxy golang/go#20526

  5. old Go library that implements Forwarded parsing
    https://github.com/stanvit/go-forwarded

  6. Discussion for Caddy to add this header support
    Add support for Forwarded header (RFC 7239) caddyserver/caddy#3262

  7. Nginx wiki about Forwarded header
    https://github.com/nginxinc/nginx-wiki/blob/master/source/start/topics/examples/forwarded.rst "Using the Forwarded header"
    https://trac.nginx.org/nginx/ticket/1316

  8. PHP framework: Symphony commit to add support for header
    Added new Forwarded header support for Request::getClientIps symfony/symfony#11379

  9. Previous PR for that header in Echo
    Fix issue #2694 #2702

@@ -40,6 +40,8 @@ type Context interface {
// Scheme returns the HTTP protocol scheme, `http` or `https`.
Scheme() string

SchemeForwarded() *Forwarded
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Adding new SchemeForwarded method to Context interface is a backwards incompatible change. It has to be some utility function or specific IPExtractor implementation

func (c *context) RealIP() string {
if c.echo != nil && c.echo.IPExtractor != nil {
return c.echo.IPExtractor(c.request)
}
// Check if the "Forwarded" header is present in the request.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

After seeing how complex or how less understood Forwarded header is - I think it is better not to add it directly to RealIP method. It is better suited as separate IPExtractor implementation

func (c *context) SchemeForwarded() *Forwarded {
// Parse and get "Forwarded" header.
// See : https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Forwarded
if scheme := c.request.Header.Get(HeaderForwarded); scheme != "" {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This only parses single Header. From mozilla article I understand that you can have actually multiple Forwarded Headers in request.

RFC says https://datatracker.ietf.org/doc/html/rfc7239

A proxy server that wants to add a new "Forwarded" header field value
can either append it to the last existing "Forwarded" header field
after a comma separator or add a new field at the end of the header
block
.

Mozilla rephrases this as
https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Forwarded

If there are multiple proxy servers between the client and server, they may each specify their own forwarding information. This can be done by adding a new Forwarded header to the end of the header block, or by appending the information to the end of the last Forwarded header in a comma-separated list.

There is Nginx issue https://trac.nginx.org/nginx/ticket/1316 that has example for multiple headers in request


func (c *context) parseForwarded(input string) (Forwarded, error) {
forwarded := Forwarded{}
entries := strings.Split(input, ",")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To be honest - I do not understand when is semicolon used and when is colon used.
Mozilla documentation says that Directives are key=value pairs, separated by a semicolon. and This can be done by adding a new Forwarded header to the end of the header block, or by appending the information to the end of the last Forwarded header in a comma-separated list.

Can semicolon and comma be mixed?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Additionally - what about quoted strings that contain comma or semicolon?
RFC says that field value can be value = token / quoted-string where

These 2 are defined as

token = <Defined in [RFC7230], Section 3.2.6>
quoted-string = <Defined in [RFC7230], Section 3.2.6>

I think rfc7230 section-3.2.6 allows unescaped commas in quoted field value

}

key := strings.TrimSpace(parts[0])
value, err := url.QueryUnescape(strings.TrimSpace(parts[1]))
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is url.QueryUnescape correct by rfc7230 section-3.2.6?

RFC says that value is defined as:

forwarded-pair = token "=" value
value = token / quoted-string
token = <Defined in [RFC7230], Section 3.2.6>
quoted-string = <Defined in [RFC7230], Section 3.2.6>

@aldas
Copy link
Contributor

aldas commented Feb 15, 2025

This probably be better off as separate library

# for free to join this conversation on GitHub. Already have an account? # to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants