Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

YARN-11790: TestAmFilter#testProxyUpdate fails in some networks #7478

Merged
merged 1 commit into from
Mar 7, 2025

Conversation

cnauroth
Copy link
Contributor

@cnauroth cnauroth commented Mar 6, 2025

Description of PR

JIRA: YARN-11790: TestAmFilter#testProxyUpdate fails in some networks.

While working on SPARK-51408, I discovered an issue that can cause their AmIpFilterSuite#testProxyUpdate to fail in certain unusual network setups. This test is a fork of a YARN test from Hadoop. As part of the solution, I included some additional randomization on a fake host name intended to simulate an unresolvable host. I'd like to bring the same change back into the original YARN test to make sure this doesn't become a problem in Hadoop test runs and we keep the code in sync.

How was this patch tested?

mvn -o \
    -pl hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-web-proxy \
    test \
    -Dtest=TestAmFilter

For code changes:

  • Does the title or this PR starts with the corresponding JIRA issue id (e.g. 'HADOOP-17799. Your PR title ...')?
  • Object storage: have the integration tests been executed and the endpoint declared according to the connector-specific documentation?
  • If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under ASF 2.0?
  • If applicable, have you updated the LICENSE, LICENSE-binary, NOTICE-binary files?

@cnauroth
Copy link
Contributor Author

cnauroth commented Mar 6, 2025

Hello @slfan1989 . This is a one-liner test change that keeps in sync with another recent test change in Spark. Can I please get your code review? Thank you.

CC: @pan3793

@slfan1989
Copy link
Contributor

Hello @slfan1989 . This is a one-liner test change that keeps in sync with another recent test change in Spark. Can I please get your code review? Thank you.

CC: @pan3793

Thanks for the contribution! LGTM.

@hadoop-yetus
Copy link

🎊 +1 overall

Vote Subsystem Runtime Logfile Comment
+0 🆗 reexec 0m 19s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 0s No case conflicting files found.
+0 🆗 codespell 0m 0s codespell was not available.
+0 🆗 detsecrets 0m 0s detect-secrets was not available.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
+1 💚 test4tests 0m 0s The patch appears to include 1 new or modified test files.
_ trunk Compile Tests _
+1 💚 mvninstall 24m 5s trunk passed
+1 💚 compile 0m 17s trunk passed with JDK Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu120.04
+1 💚 compile 0m 17s trunk passed with JDK Private Build-1.8.0_442-8u442-b06us1-0ubuntu120.04-b06
+1 💚 checkstyle 0m 18s trunk passed
+1 💚 mvnsite 0m 21s trunk passed
+1 💚 javadoc 0m 24s trunk passed with JDK Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu120.04
+1 💚 javadoc 0m 19s trunk passed with JDK Private Build-1.8.0_442-8u442-b06us1-0ubuntu120.04-b06
+1 💚 spotbugs 0m 34s trunk passed
+1 💚 shadedclient 19m 57s branch has no errors when building and testing our client artifacts.
_ Patch Compile Tests _
+1 💚 mvninstall 0m 12s the patch passed
+1 💚 compile 0m 13s the patch passed with JDK Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu120.04
+1 💚 javac 0m 13s the patch passed
+1 💚 compile 0m 10s the patch passed with JDK Private Build-1.8.0_442-8u442-b06us1-0ubuntu120.04-b06
+1 💚 javac 0m 10s the patch passed
+1 💚 blanks 0m 0s The patch has no blanks issues.
+1 💚 checkstyle 0m 9s the patch passed
+1 💚 mvnsite 0m 13s the patch passed
+1 💚 javadoc 0m 12s the patch passed with JDK Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu120.04
+1 💚 javadoc 0m 13s the patch passed with JDK Private Build-1.8.0_442-8u442-b06us1-0ubuntu120.04-b06
+1 💚 spotbugs 0m 32s the patch passed
+1 💚 shadedclient 20m 32s patch has no errors when building and testing our client artifacts.
_ Other Tests _
+1 💚 unit 0m 55s hadoop-yarn-server-web-proxy in the patch passed.
+1 💚 asflicense 0m 24s The patch does not generate ASF License warnings.
71m 47s
Subsystem Report/Notes
Docker ClientAPI=1.48 ServerAPI=1.48 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7478/1/artifact/out/Dockerfile
GITHUB PR #7478
Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets
uname Linux ebf4a0ea7317 5.15.0-130-generic #140-Ubuntu SMP Wed Dec 18 17:59:53 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/bin/hadoop.sh
git revision trunk / b14a2c4
Default Java Private Build-1.8.0_442-8u442-b06us1-0ubuntu120.04-b06
Multi-JDK versions /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu120.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_442-8u442-b06us1-0ubuntu120.04-b06
Test Results https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7478/1/testReport/
Max. process+thread count 560 (vs. ulimit of 5500)
modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-web-proxy U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-web-proxy
Console output https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7478/1/console
versions git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

@slfan1989 slfan1989 merged commit 972b6e0 into apache:trunk Mar 7, 2025
4 checks passed
@slfan1989
Copy link
Contributor

@cnauroth Thanks for the contribution! @pan3793 Thanks for the review! Merged into trunk.

slfan1989 pushed a commit that referenced this pull request Mar 7, 2025
… Contributed by Chris Nauroth.

Reviewed-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Shilun Fan <slfan1989@apache.org>
@slfan1989
Copy link
Contributor

@cnauroth I cherry-pick this pr to branch-3.4.

cc: @steveloughran

@cnauroth
Copy link
Contributor Author

cnauroth commented Mar 7, 2025

Thank @slfan1989 and @pan3793 !

# for free to join this conversation on GitHub. Already have an account? # to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants