Skip to content

Commit 6444ecf

Browse files
author
Daniel Rogers
committed
Add support for regular expression matching and sanitizing of headers in FastAPI.
1 parent f58d16b commit 6444ecf

File tree

3 files changed

+128
-51
lines changed

3 files changed

+128
-51
lines changed

‎CHANGELOG.md

+2
Original file line numberDiff line numberDiff line change
@@ -31,6 +31,8 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
3131
([#1415](https://github.com/open-telemetry/opentelemetry-python-contrib/pull/1415))
3232
- `opentelemetry-instrumentation-falcon` Add support for regular expression matching and sanitization of HTTP headers.
3333
([#1412](https://github.com/open-telemetry/opentelemetry-python-contrib/pull/1412))
34+
- `opentelemetry-instrumentation-fastapi` Add support for regular expression matching and sanitization of HTTP headers.
35+
([#1403](https://github.com/open-telemetry/opentelemetry-python-contrib/pull/1403))
3436
- `opentelemetry-instrumentation-flask` Add support for regular expression matching and sanitization of HTTP headers.
3537
([#1413](https://github.com/open-telemetry/opentelemetry-python-contrib/pull/1413))
3638
- `opentelemetry-instrumentation-pyramid` Add support for regular expression matching and sanitization of HTTP headers.

‎instrumentation/opentelemetry-instrumentation-fastapi/src/opentelemetry/instrumentation/fastapi/__init__.py

+69-26
Original file line numberDiff line numberDiff line change
@@ -34,8 +34,9 @@ async def foobar():
3434
3535
Exclude lists
3636
*************
37-
To exclude certain URLs from being tracked, set the environment variable ``OTEL_PYTHON_FASTAPI_EXCLUDED_URLS``
38-
(or ``OTEL_PYTHON_EXCLUDED_URLS`` as fallback) with comma delimited regexes representing which URLs to exclude.
37+
To exclude certain URLs from tracking, set the environment variable ``OTEL_PYTHON_FASTAPI_EXCLUDED_URLS``
38+
(or ``OTEL_PYTHON_EXCLUDED_URLS`` to cover all instrumentations) to a string of comma delimited regexes that match the
39+
URLs.
3940
4041
For example,
4142
@@ -45,7 +46,7 @@ async def foobar():
4546
4647
will exclude requests such as ``https://site/client/123/info`` and ``https://site/xyz/healthcheck``.
4748
48-
You can also pass the comma delimited regexes to the ``instrument_app`` method directly:
49+
You can also pass comma delimited regexes directly to the ``instrument_app`` method:
4950
5051
.. code-block:: python
5152
@@ -54,9 +55,12 @@ async def foobar():
5455
Request/Response hooks
5556
**********************
5657
57-
Utilize request/response hooks to execute custom logic to be performed before/after performing a request. The server request hook takes in a server span and ASGI
58-
scope object for every incoming request. The client request hook is called with the internal span and an ASGI scope which is sent as a dictionary for when the method receive is called.
59-
The client response hook is called with the internal span and an ASGI event which is sent as a dictionary for when the method send is called.
58+
This instrumentation supports request and response hooks. These are functions that get called
59+
right after a span is created for a request and right before the span is finished for the response.
60+
61+
- The server request hook is passed a server span and ASGI scope object for every incoming request.
62+
- The client request hook is called with the internal span and an ASGI scope when the method ``receive`` is called.
63+
- The client response hook is called with the internal span and an ASGI event when the method ``send`` is called.
6064
6165
.. code-block:: python
6266
@@ -76,54 +80,93 @@ def client_response_hook(span: Span, message: dict):
7680
7781
Capture HTTP request and response headers
7882
*****************************************
79-
You can configure the agent to capture predefined HTTP headers as span attributes, according to the `semantic convention <https://github.com/open-telemetry/opentelemetry-specification/blob/main/specification/trace/semantic_conventions/http.md#http-request-and-response-headers>`_.
83+
You can configure the agent to capture specified HTTP headers as span attributes, according to the
84+
`semantic convention <https://github.com/open-telemetry/opentelemetry-specification/blob/main/specification/trace/semantic_conventions/http.md#http-request-and-response-headers>`_.
8085
8186
Request headers
8287
***************
83-
To capture predefined HTTP request headers as span attributes, set the environment variable ``OTEL_INSTRUMENTATION_HTTP_CAPTURE_HEADERS_SERVER_REQUEST``
84-
to a comma-separated list of HTTP header names.
88+
To capture HTTP request headers as span attributes, set the environment variable
89+
``OTEL_INSTRUMENTATION_HTTP_CAPTURE_HEADERS_SERVER_REQUEST`` to a comma delimited list of HTTP header names.
8590
8691
For example,
87-
8892
::
8993
9094
export OTEL_INSTRUMENTATION_HTTP_CAPTURE_HEADERS_SERVER_REQUEST="content-type,custom_request_header"
9195
92-
will extract ``content-type`` and ``custom_request_header`` from request headers and add them as span attributes.
96+
will extract ``content-type`` and ``custom_request_header`` from the request headers and add them as span attributes.
97+
98+
Request header names in FastAPI are case-insensitive. So, giving the header name as ``CUStom-Header`` in the environment
99+
variable will capture the header named ``custom-header``.
100+
101+
Regular expressions may also be used to match multiple headers that correspond to the given pattern. For example:
102+
::
103+
104+
export OTEL_INSTRUMENTATION_HTTP_CAPTURE_HEADERS_SERVER_REQUEST="Accept.*,X-.*"
93105
94-
It is recommended that you should give the correct names of the headers to be captured in the environment variable.
95-
Request header names in fastapi are case insensitive. So, giving header name as ``CUStom-Header`` in environment variable will be able capture header with name ``custom-header``.
106+
Would match all request headers that start with ``Accept`` and ``X-``.
96107
97-
The name of the added span attribute will follow the format ``http.request.header.<header_name>`` where ``<header_name>`` being the normalized HTTP header name (lowercase, with - characters replaced by _ ).
98-
The value of the attribute will be single item list containing all the header values.
108+
To capture all request headers, set ``OTEL_INSTRUMENTATION_HTTP_CAPTURE_HEADERS_SERVER_REQUEST`` to ``".*"``.
109+
::
110+
111+
export OTEL_INSTRUMENTATION_HTTP_CAPTURE_HEADERS_SERVER_REQUEST=".*"
112+
113+
The name of the added span attribute will follow the format ``http.request.header.<header_name>`` where ``<header_name>``
114+
is the normalized HTTP header name (lowercase, with ``-`` replaced by ``_``). The value of the attribute will be a
115+
single item list containing all the header values.
99116
100-
Example of the added span attribute,
117+
For example:
101118
``http.request.header.custom_request_header = ["<value1>,<value2>"]``
102119
103120
Response headers
104121
****************
105-
To capture predefined HTTP response headers as span attributes, set the environment variable ``OTEL_INSTRUMENTATION_HTTP_CAPTURE_HEADERS_SERVER_RESPONSE``
106-
to a comma-separated list of HTTP header names.
122+
To capture HTTP response headers as span attributes, set the environment variable
123+
``OTEL_INSTRUMENTATION_HTTP_CAPTURE_HEADERS_SERVER_RESPONSE`` to a comma delimited list of HTTP header names.
107124
108125
For example,
109-
110126
::
111127
112128
export OTEL_INSTRUMENTATION_HTTP_CAPTURE_HEADERS_SERVER_RESPONSE="content-type,custom_response_header"
113129
114-
will extract ``content-type`` and ``custom_response_header`` from response headers and add them as span attributes.
130+
will extract ``content-type`` and ``custom_response_header`` from the response headers and add them as span attributes.
131+
132+
Response header names in FastAPI are case-insensitive. So, giving the header name as ``CUStom-Header`` in the environment
133+
variable will capture the header named ``custom-header``.
115134
116-
It is recommended that you should give the correct names of the headers to be captured in the environment variable.
117-
Response header names captured in fastapi are case insensitive. So, giving header name as ``CUStomHeader`` in environment variable will be able capture header with name ``customheader``.
135+
Regular expressions may also be used to match multiple headers that correspond to the given pattern. For example:
136+
::
118137
119-
The name of the added span attribute will follow the format ``http.response.header.<header_name>`` where ``<header_name>`` being the normalized HTTP header name (lowercase, with - characters replaced by _ ).
120-
The value of the attribute will be single item list containing all the header values.
138+
export OTEL_INSTRUMENTATION_HTTP_CAPTURE_HEADERS_SERVER_RESPONSE="Content.*,X-.*"
121139
122-
Example of the added span attribute,
140+
Would match all response headers that start with ``Content`` and ``X-``.
141+
142+
To capture all response headers, set ``OTEL_INSTRUMENTATION_HTTP_CAPTURE_HEADERS_SERVER_RESPONSE`` to ``".*"``.
143+
::
144+
145+
export OTEL_INSTRUMENTATION_HTTP_CAPTURE_HEADERS_SERVER_RESPONSE=".*"
146+
147+
The name of the added span attribute will follow the format ``http.response.header.<header_name>`` where ``<header_name>``
148+
is the normalized HTTP header name (lowercase, with ``-`` replaced by ``_``). The value of the attribute will be a
149+
single item list containing all the header values.
150+
151+
For example:
123152
``http.response.header.custom_response_header = ["<value1>,<value2>"]``
124153
154+
Sanitizing headers
155+
******************
156+
In order to prevent storing sensitive data such as personally identifiable information (PII), session keys, passwords,
157+
etc, set the environment variable ``OTEL_INSTRUMENTATION_HTTP_CAPTURE_HEADERS_SANITIZE_FIELDS``
158+
to a comma delimited list of HTTP header names to be sanitized. Regexes may be used, and all header names will be
159+
matched in a case-insensitive manner.
160+
161+
For example,
162+
::
163+
164+
export OTEL_INSTRUMENTATION_HTTP_CAPTURE_HEADERS_SANITIZE_FIELDS=".*session.*,set-cookie"
165+
166+
will replace the value of headers such as ``session-id`` and ``set-cookie`` with ``[REDACTED]`` in the span.
167+
125168
Note:
126-
Environment variable names to capture http headers are still experimental, and thus are subject to change.
169+
The environment variable names used to capture HTTP headers are still experimental, and thus are subject to change.
127170
128171
API
129172
---

‎instrumentation/opentelemetry-instrumentation-fastapi/tests/test_fastapi_instrumentation.py

+57-25
Original file line numberDiff line numberDiff line change
@@ -32,6 +32,7 @@
3232
from opentelemetry.test.globals_test import reset_trace_globals
3333
from opentelemetry.test.test_base import TestBase
3434
from opentelemetry.util.http import (
35+
OTEL_INSTRUMENTATION_HTTP_CAPTURE_HEADERS_SANITIZE_FIELDS,
3536
OTEL_INSTRUMENTATION_HTTP_CAPTURE_HEADERS_SERVER_REQUEST,
3637
OTEL_INSTRUMENTATION_HTTP_CAPTURE_HEADERS_SERVER_RESPONSE,
3738
_active_requests_count_attrs,
@@ -529,24 +530,23 @@ def test_mark_span_internal_in_presence_of_span_from_other_framework(self):
529530
)
530531

531532

533+
@patch.dict(
534+
"os.environ",
535+
{
536+
OTEL_INSTRUMENTATION_HTTP_CAPTURE_HEADERS_SANITIZE_FIELDS: ".*my-secret.*",
537+
OTEL_INSTRUMENTATION_HTTP_CAPTURE_HEADERS_SERVER_REQUEST: "Custom-Test-Header-1,Custom-Test-Header-2,Custom-Test-Header-3,Regex-Test-Header-.*,Regex-Invalid-Test-Header-.*,.*my-secret.*",
538+
OTEL_INSTRUMENTATION_HTTP_CAPTURE_HEADERS_SERVER_RESPONSE: "Custom-Test-Header-1,Custom-Test-Header-2,Custom-Test-Header-3,my-custom-regex-header-.*,invalid-regex-header-.*,.*my-secret.*",
539+
},
540+
)
532541
class TestHTTPAppWithCustomHeaders(TestBase):
533542
def setUp(self):
534543
super().setUp()
535-
self.env_patch = patch.dict(
536-
"os.environ",
537-
{
538-
OTEL_INSTRUMENTATION_HTTP_CAPTURE_HEADERS_SERVER_REQUEST: "Custom-Test-Header-1,Custom-Test-Header-2,Custom-Test-Header-3",
539-
OTEL_INSTRUMENTATION_HTTP_CAPTURE_HEADERS_SERVER_RESPONSE: "Custom-Test-Header-1,Custom-Test-Header-2,Custom-Test-Header-3",
540-
},
541-
)
542-
self.env_patch.start()
543544
self.app = self._create_app()
544545
otel_fastapi.FastAPIInstrumentor().instrument_app(self.app)
545546
self.client = TestClient(self.app)
546547

547548
def tearDown(self) -> None:
548549
super().tearDown()
549-
self.env_patch.stop()
550550
with self.disable_logging():
551551
otel_fastapi.FastAPIInstrumentor().uninstrument_app(self.app)
552552

@@ -559,6 +559,9 @@ async def _():
559559
headers = {
560560
"custom-test-header-1": "test-header-value-1",
561561
"custom-test-header-2": "test-header-value-2",
562+
"my-custom-regex-header-1": "my-custom-regex-value-1,my-custom-regex-value-2",
563+
"My-Custom-Regex-Header-2": "my-custom-regex-value-3,my-custom-regex-value-4",
564+
"My-Secret-Header": "My Secret Value",
562565
}
563566
content = {"message": "hello world"}
564567
return JSONResponse(content=content, headers=headers)
@@ -573,12 +576,20 @@ def test_http_custom_request_headers_in_span_attributes(self):
573576
"http.request.header.custom_test_header_2": (
574577
"test-header-value-2",
575578
),
579+
"http.request.header.regex_test_header_1": ("Regex Test Value 1",),
580+
"http.request.header.regex_test_header_2": (
581+
"RegexTestValue2,RegexTestValue3",
582+
),
583+
"http.request.header.my_secret_header": ("[REDACTED]",),
576584
}
577585
resp = self.client.get(
578586
"/foobar",
579587
headers={
580588
"custom-test-header-1": "test-header-value-1",
581589
"custom-test-header-2": "test-header-value-2",
590+
"Regex-Test-Header-1": "Regex Test Value 1",
591+
"regex-test-header-2": "RegexTestValue2,RegexTestValue3",
592+
"My-Secret-Header": "My Secret Value",
582593
},
583594
)
584595
self.assertEqual(200, resp.status_code)
@@ -602,6 +613,9 @@ def test_http_custom_request_headers_not_in_span_attributes(self):
602613
headers={
603614
"custom-test-header-1": "test-header-value-1",
604615
"custom-test-header-2": "test-header-value-2",
616+
"Regex-Test-Header-1": "Regex Test Value 1",
617+
"regex-test-header-2": "RegexTestValue2,RegexTestValue3",
618+
"My-Secret-Header": "My Secret Value",
605619
},
606620
)
607621
self.assertEqual(200, resp.status_code)
@@ -623,6 +637,13 @@ def test_http_custom_response_headers_in_span_attributes(self):
623637
"http.response.header.custom_test_header_2": (
624638
"test-header-value-2",
625639
),
640+
"http.response.header.my_custom_regex_header_1": (
641+
"my-custom-regex-value-1,my-custom-regex-value-2",
642+
),
643+
"http.response.header.my_custom_regex_header_2": (
644+
"my-custom-regex-value-3,my-custom-regex-value-4",
645+
),
646+
"http.response.header.my_secret_header": ("[REDACTED]",),
626647
}
627648
resp = self.client.get("/foobar")
628649
self.assertEqual(200, resp.status_code)
@@ -653,24 +674,23 @@ def test_http_custom_response_headers_not_in_span_attributes(self):
653674
self.assertNotIn(key, server_span.attributes)
654675

655676

677+
@patch.dict(
678+
"os.environ",
679+
{
680+
OTEL_INSTRUMENTATION_HTTP_CAPTURE_HEADERS_SANITIZE_FIELDS: ".*my-secret.*",
681+
OTEL_INSTRUMENTATION_HTTP_CAPTURE_HEADERS_SERVER_REQUEST: "Custom-Test-Header-1,Custom-Test-Header-2,Custom-Test-Header-3,Regex-Test-Header-.*,Regex-Invalid-Test-Header-.*,.*my-secret.*",
682+
OTEL_INSTRUMENTATION_HTTP_CAPTURE_HEADERS_SERVER_RESPONSE: "Custom-Test-Header-1,Custom-Test-Header-2,Custom-Test-Header-3,my-custom-regex-header-.*,invalid-regex-header-.*,.*my-secret.*",
683+
},
684+
)
656685
class TestWebSocketAppWithCustomHeaders(TestBase):
657686
def setUp(self):
658687
super().setUp()
659-
self.env_patch = patch.dict(
660-
"os.environ",
661-
{
662-
OTEL_INSTRUMENTATION_HTTP_CAPTURE_HEADERS_SERVER_REQUEST: "Custom-Test-Header-1,Custom-Test-Header-2,Custom-Test-Header-3",
663-
OTEL_INSTRUMENTATION_HTTP_CAPTURE_HEADERS_SERVER_RESPONSE: "Custom-Test-Header-1,Custom-Test-Header-2,Custom-Test-Header-3",
664-
},
665-
)
666-
self.env_patch.start()
667688
self.app = self._create_app()
668689
otel_fastapi.FastAPIInstrumentor().instrument_app(self.app)
669690
self.client = TestClient(self.app)
670691

671692
def tearDown(self) -> None:
672693
super().tearDown()
673-
self.env_patch.stop()
674694
with self.disable_logging():
675695
otel_fastapi.FastAPIInstrumentor().uninstrument_app(self.app)
676696

@@ -688,6 +708,12 @@ async def _(websocket: fastapi.WebSocket):
688708
"headers": [
689709
(b"custom-test-header-1", b"test-header-value-1"),
690710
(b"custom-test-header-2", b"test-header-value-2"),
711+
(b"Regex-Test-Header-1", b"Regex Test Value 1"),
712+
(
713+
b"regex-test-header-2",
714+
b"RegexTestValue2,RegexTestValue3",
715+
),
716+
(b"My-Secret-Header", b"My Secret Value"),
691717
],
692718
}
693719
)
@@ -727,6 +753,13 @@ def test_web_socket_custom_request_headers_in_span_attributes(self):
727753

728754
self.assertSpanHasAttributes(server_span, expected)
729755

756+
@patch.dict(
757+
"os.environ",
758+
{
759+
OTEL_INSTRUMENTATION_HTTP_CAPTURE_HEADERS_SANITIZE_FIELDS: ".*my-secret.*",
760+
OTEL_INSTRUMENTATION_HTTP_CAPTURE_HEADERS_SERVER_REQUEST: "Custom-Test-Header-1,Custom-Test-Header-2,Custom-Test-Header-3,Regex-Test-Header-.*,Regex-Invalid-Test-Header-.*,.*my-secret.*",
761+
},
762+
)
730763
def test_web_socket_custom_request_headers_not_in_span_attributes(self):
731764
not_expected = {
732765
"http.request.header.custom_test_header_3": (
@@ -799,16 +832,15 @@ def test_web_socket_custom_response_headers_not_in_span_attributes(self):
799832
self.assertNotIn(key, server_span.attributes)
800833

801834

835+
@patch.dict(
836+
"os.environ",
837+
{
838+
OTEL_INSTRUMENTATION_HTTP_CAPTURE_HEADERS_SERVER_REQUEST: "Custom-Test-Header-1,Custom-Test-Header-2,Custom-Test-Header-3",
839+
},
840+
)
802841
class TestNonRecordingSpanWithCustomHeaders(TestBase):
803842
def setUp(self):
804843
super().setUp()
805-
self.env_patch = patch.dict(
806-
"os.environ",
807-
{
808-
OTEL_INSTRUMENTATION_HTTP_CAPTURE_HEADERS_SERVER_REQUEST: "Custom-Test-Header-1,Custom-Test-Header-2,Custom-Test-Header-3",
809-
},
810-
)
811-
self.env_patch.start()
812844
self.app = fastapi.FastAPI()
813845

814846
@self.app.get("/foobar")

0 commit comments

Comments
 (0)