HADOOP-18980. S3A credential provider remapping: make extensible (#6406)

[virajjasani]

Contributed by Viraj Jasani

[hadoop-yetus]

🎊 +1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	17m 25s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+0 🆗	detsecrets	0m 0s		detect-secrets was not available.
+0 🆗	markdownlint	0m 0s		markdownlint was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 3 new or modified test files.
			_ branch-3.4 Compile Tests _
+0 🆗	mvndep	4m 6s		Maven dependency ordering for branch
+1 💚	mvninstall	44m 55s		branch-3.4 passed
+1 💚	compile	18m 27s		branch-3.4 passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04
+1 💚	compile	17m 3s		branch-3.4 passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08
+1 💚	checkstyle	5m 27s		branch-3.4 passed
+1 💚	mvnsite	2m 34s		branch-3.4 passed
+1 💚	javadoc	1m 45s		branch-3.4 passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04
+1 💚	javadoc	1m 32s		branch-3.4 passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08
+1 💚	spotbugs	3m 47s		branch-3.4 passed
+1 💚	shadedclient	38m 17s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+0 🆗	mvndep	0m 32s		Maven dependency ordering for patch
+1 💚	mvninstall	1m 26s		the patch passed
+1 💚	compile	17m 44s		the patch passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04
+1 💚	javac	17m 44s		the patch passed
+1 💚	compile	16m 33s		the patch passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08
+1 💚	javac	16m 33s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
+1 💚	checkstyle	4m 49s		the patch passed
+1 💚	mvnsite	2m 27s		the patch passed
+1 💚	javadoc	1m 41s		the patch passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04
+1 💚	javadoc	1m 32s		the patch passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08
+1 💚	spotbugs	4m 1s		the patch passed
+1 💚	shadedclient	38m 15s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
+1 💚	unit	19m 16s		hadoop-common in the patch passed.
+1 💚	unit	3m 4s		hadoop-aws in the patch passed.
+1 💚	asflicense	0m 58s		The patch does not generate ASF License warnings.
		273m 59s

Subsystem	Report/Notes
Docker	ClientAPI=1.44 ServerAPI=1.44 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6525/1/artifact/out/Dockerfile
GITHUB PR	#6525
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets markdownlint
uname	Linux 3a00207190c6 5.15.0-88-generic #98-Ubuntu SMP Mon Oct 2 15:18:56 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	branch-3.4 / e31bb15
Default Java	Private Build-1.8.0_392-8u392-ga-1~20.04-b08
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_392-8u392-ga-1~20.04-b08
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6525/1/testReport/
Max. process+thread count	2637 (vs. ulimit of 5500)
modules	C: hadoop-common-project/hadoop-common hadoop-tools/hadoop-aws U: .
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6525/1/console
versions	git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

[virajjasani]

@steveloughran this one is for branch-3.4 backport.

[steveloughran]

afraid another change has broken your merge. rebase and force push

[hadoop-yetus]

🎊 +1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	18m 15s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 1s		codespell was not available.
+0 🆗	detsecrets	0m 1s		detect-secrets was not available.
+0 🆗	markdownlint	0m 1s		markdownlint was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 3 new or modified test files.
			_ branch-3.4 Compile Tests _
+0 🆗	mvndep	4m 6s		Maven dependency ordering for branch
+1 💚	mvninstall	44m 43s		branch-3.4 passed
+1 💚	compile	18m 21s		branch-3.4 passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04
+1 💚	compile	17m 5s		branch-3.4 passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08
+1 💚	checkstyle	5m 20s		branch-3.4 passed
+1 💚	mvnsite	2m 38s		branch-3.4 passed
+1 💚	javadoc	1m 46s		branch-3.4 passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04
+1 💚	javadoc	1m 31s		branch-3.4 passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08
+1 💚	spotbugs	3m 43s		branch-3.4 passed
+1 💚	shadedclient	37m 58s		branch has no errors when building and testing our client artifacts.
-0 ⚠️	patch	38m 23s		Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary.
			_ Patch Compile Tests _
+0 🆗	mvndep	0m 32s		Maven dependency ordering for patch
+1 💚	mvninstall	1m 24s		the patch passed
+1 💚	compile	18m 14s		the patch passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04
+1 💚	javac	18m 14s		the patch passed
+1 💚	compile	16m 37s		the patch passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08
+1 💚	javac	16m 37s		the patch passed
+1 💚	blanks	0m 1s		The patch has no blanks issues.
+1 💚	checkstyle	4m 40s		the patch passed
+1 💚	mvnsite	2m 26s		the patch passed
+1 💚	javadoc	1m 38s		the patch passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04
+1 💚	javadoc	1m 33s		the patch passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08
+1 💚	spotbugs	4m 4s		the patch passed
+1 💚	shadedclient	38m 52s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
+1 💚	unit	19m 14s		hadoop-common in the patch passed.
+1 💚	unit	3m 6s		hadoop-aws in the patch passed.
+1 💚	asflicense	0m 57s		The patch does not generate ASF License warnings.
		275m 14s

Subsystem	Report/Notes
Docker	ClientAPI=1.44 ServerAPI=1.44 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6525/2/artifact/out/Dockerfile
GITHUB PR	#6525
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets markdownlint
uname	Linux 05b3a7686dc8 5.15.0-88-generic #98-Ubuntu SMP Mon Oct 2 15:18:56 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	branch-3.4 / a0c3a90
Default Java	Private Build-1.8.0_392-8u392-ga-1~20.04-b08
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_392-8u392-ga-1~20.04-b08
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6525/2/testReport/
Max. process+thread count	3137 (vs. ulimit of 5500)
modules	C: hadoop-common-project/hadoop-common hadoop-tools/hadoop-aws U: .
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6525/2/console
versions	git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

[steveloughran]

+1

looking at the tests, I can see there some corner cases of key=val split missing:

• duplicate entries key=val1, key=val2. should the parser fail or just return the latest (as is done today)
• empty ,,
• =val
• key=

do you want to cover those? the last two should fail, other ones, well, I see no problem with them passing.

[virajjasani]

Addendum PR to cover corner cases: #6546