perf: remove duplicate ids in ExpMetricNames api

[keita-determined]

Ticket

https://hpe-aiatscale.slack.com/archives/CLCE8D998/p1724182588030219

Description

Remove duplicate ids to improve API performance.
In a certain environment that has relatively lots of run data (50k~), v1/experiments/metrics-stream/metric-names takes some time. For example, https://gcloud.determined.ai/api/v1/experiments/metrics-stream/metric-names?ids=1&ids=1&ids=1&ids=1&ids=1&ids=1&ids=1&ids=1&ids=1&ids=1&ids=1&ids=1&ids=1&ids=1&ids=1&ids=1&ids=2&ids=3&ids=4&ids=5 takes 6s. It would take longer if there are more run data or more ids query parameters.
This PR is to improve the API response speed performance by removing the duplicate ids. (Sometimes the client (web frontend or user) call the API with duplicate ids)

After this fix, the above API call takes around 1.8s, so 3 times faster than before. We can do some more improvement around the API, but this is the first step.

hey -n 15 -c 1 \
  -H 'Cache-Control: no-cache' \
  -H 'Cookie: [redacted]' \
  -H 'Pragma: no-cache'\
  -H 'Authorization: Bearer token' \ 
'https://gcloud.determined.ai/api/v1/experiments/metrics-stream/metric-names?ids=1&ids=1&ids=1&ids=1&ids=1&ids=1&ids=1&ids=1&ids=1&ids=1&ids=1&ids=1&ids=1&ids=1&ids=1&ids=1&ids=2&ids=3&ids=4&ids=5'

Before

After

Test Plan

• call ExpMetricNames api with duplicate ids, and check if duplicated ids are removed in the API function
  • id: [1, 1, 1, 3, 4, 1] -> [1, 3, 4]

Checklist

• Changes have been manually QA'd
• New features have been approved by the corresponding PM
• User-facing API changes have the "User-facing API Change" label
• Release notes have been added as a separate file under docs/release-notes/
  See Release Note for details.
• Licenses have been included for new code which was copied and/or modified from any external code

[netlify]

✅ Deploy Preview for determined-ui ready!

Name	Link
🔨 Latest commit	c645aeb
🔍 Latest deploy log	https://app.netlify.com/sites/determined-ui/deploys/66c501e30213e7000867005d
😎 Deploy Preview	https://deploy-preview-9848--determined-ui.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

---

To edit notification comments on pull requests, go to your Netlify site configuration.

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 54.71%. Comparing base (e13de20) to head (c645aeb).
Report is 3 commits behind head on main.

Additional details and impacted files

@@           Coverage Diff           @@
##             main    #9848   +/-   ##
=======================================
  Coverage   54.70%   54.71%           
=======================================
  Files        1261     1261           
  Lines      156012   156014    +2     
  Branches     3588     3588           
=======================================
+ Hits        85353    85357    +4     
+ Misses      70527    70525    -2     
  Partials      132      132           

Flag	Coverage Δ
backend	45.16% <100.00%> (+<0.01%)	⬆️
harness	72.60% <ø> (ø)
web	54.47% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

Files	Coverage Δ
master/internal/api_experiment.go	56.80% <100.00%> (+0.04%)	⬆️

... and 4 files with indirect coverage changes

[maxrussell]

LGTM