enhance: Optimize datacoord meta mutex

[bigsheeper]

1. Use a separate collection mutex.
2. Avoid holding locks for extended periods, such as in GetQuotaInfo. (TODO)

issue: #40551

[sre-ci-robot]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: bigsheeper
To complete the pull request process, please assign liliu-z after the PR has been reviewed.
You can assign the PR to them by writing /assign @liliu-z in a comment when ready.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

• internal/datacoord/OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 70.00%. Comparing base (0420dc1) to head (e481353).
Report is 4 commits behind head on master.

❌ Your project check has failed because the head coverage (70.00%) is below the target coverage (77.00%). You can increase the head coverage or adjust the target coverage.

❗ There is a different number of reports uploaded between BASE (0420dc1) and HEAD (e481353). Click for more details.

HEAD has 1 upload less than BASE

Flag	BASE (0420dc1)	HEAD (e481353)
	2	1

Additional details and impacted files

@@             Coverage Diff             @@
##           master   #40552       +/-   ##
===========================================
- Coverage   80.50%   70.00%   -10.51%     
===========================================
  Files        1474      309     -1165     
  Lines      207669    27722   -179947     
===========================================
- Hits       167192    19406   -147786     
+ Misses      34400     8316    -26084     
+ Partials     6077        0     -6077     

Components	Coverage Δ
Client	∅ <ø> (∅)
Core	70.00% <ø> (ø)
Go	∅ <ø> (∅)

see 1165 files with indirect coverage changes

🚀 New features to boost your workflow:

• ❄ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[bigsheeper]

related to #37630

[mergify]

@bigsheeper go-sdk check failed, comment rerun go-sdk can trigger the job again.

[mergify]

@bigsheeper E2e jenkins job failed, comment /run-cpu-e2e can trigger the job again.