[improve][io] Make connectors load sensitive fields from secrets

[jiangpengcheng]

Fixes #21674

Main Issue: #xyz

PIP: #xyz

Motivation

Some connectors are using config to pass sensitive fields , like token, password, which is not secure

Modifications

use the IOConfileUtils.loadWithSecrets to load config for connectors

Verifying this change

• Make sure that the change passes the CI checks.

• This change added tests and can be verified as follows:

  • Added unittests for connectors load sensitive fields from secrets

Does this pull request potentially affect one of the following parts:

If the box was checked, please highlight the changes

• Dependencies (add or upgrade a dependency)
• The public API
• The schema
• The default values of configurations
• The threading model
• The binary protocol
• The REST endpoints
• The admin CLI options
• The metrics
• Anything that affects deployment

Documentation

• doc
• doc-required
• doc-not-needed
• doc-complete

Matching PR in forked repository

PR in forked repository: jiangpengcheng#18

[shibd]

Overall LGTM. left some small comments.

[jiangpengcheng]

/pulsarbot rerun-failure-checks

[codecov-commenter]

Codecov Report

Merging #21675 (70fd8d2) into master (fe2d6d3) will increase coverage by 36.72%.
Report is 11 commits behind head on master.
The diff coverage is 100.00%.

Additional details and impacted files

@@              Coverage Diff              @@
##             master   #21675       +/-   ##
=============================================
+ Coverage     36.69%   73.42%   +36.72%     
- Complexity    12236    32775    +20539     
=============================================
  Files          1716     1893      +177     
  Lines        131130   140741     +9611     
  Branches      14323    15504     +1181     
=============================================
+ Hits          48116   103335    +55219     
+ Misses        76625    29308    -47317     
- Partials       6389     8098     +1709     

Flag	Coverage Δ
inttests	24.13% <0.00%> (ø)
systests	24.71% <0.00%> (-0.06%)	⬇️
unittests	72.70% <100.00%> (+40.90%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files	Coverage Δ
...ava/org/apache/pulsar/io/common/IOConfigUtils.java	83.33% <100.00%> (ø)
.../org/apache/pulsar/io/dynamodb/DynamoDBSource.java	13.63% <100.00%> (ø)
...pache/pulsar/io/dynamodb/DynamoDBSourceConfig.java	48.21% <100.00%> (ø)
.../pulsar/io/influxdb/InfluxDBGenericRecordSink.java	80.00% <100.00%> (ø)
...he/pulsar/io/influxdb/v1/InfluxDBAbstractSink.java	83.33% <100.00%> (ø)
...ache/pulsar/io/influxdb/v1/InfluxDBSinkConfig.java	92.00% <100.00%> (ø)
...org/apache/pulsar/io/influxdb/v2/InfluxDBSink.java	72.46% <100.00%> (ø)
...ache/pulsar/io/influxdb/v2/InfluxDBSinkConfig.java	92.00% <100.00%> (ø)
...va/org/apache/pulsar/io/jdbc/JdbcAbstractSink.java	75.26% <100.00%> (+75.26%)	⬆️
...java/org/apache/pulsar/io/jdbc/JdbcSinkConfig.java	85.29% <100.00%> (+85.29%)	⬆️
... and 18 more

... and 1434 files with indirect coverage changes

[lhotari]

I think that this is a very useful improvement, but does this introduce a breaking change for existing users of the connectors?