Add additional metrics reporting size of artifacts served

[bmbouter]

Need / Background

When running on Pulp on cloud services the egress network costs serving data to off-cloud hosts can be very expensive. This is true regardless of if Pulp is serving the data directly (local filesystem storage) or if it's using an object storage backend.

Understanding these costs is key. It would be excellent to know answers to these questions:

• How much data did I serve for this Artifact request
• Which Repository (or at least Distribution) is this Artifact member of?
• Which Domain (if any) is this Artifact a member of?
• Where did this Artifact go (in a network sense, e.g. IP)?

Proposal

Have the pulpcore-content app serve up this raw data as part of the per-request open telemetry data that is already reported as part of basic enabling of otel support in pulpcore-content. Specifically have it report:

• Bytes of the Artifact served
• Domain the request is being handled as

The Distribution URL would be contained in the URL of the request which is already reported, so no additional work is needed there. Also the IP of the client is also contained in the data already so additional work isn't needed for that.

One Detail about Object Storage

Note, in the case where Pulp is using Object storage, the Bytes reported needs to be the Artifact size (which Pulp knows in its DB). With Object storage the actual response from Pulp is a redirect which is very small. We need the Bytes of the Artifact, not the Bytes of the redirect request served.

[ipanova]

I think we have multi-artifact content in pulp-gem, how these metrics would be working there? Usually there is 1 to 1 mapping between content and artifact so metrics for such case makes sense, but what about case with multi-artifact content?