Introduce StreamParsingStrategy to support builds with large logs

[sghill]

This implementation is currently opt-in by setting the system property hudson.plugins.logparser.ParsingStrategy to hudson.plugins.logparser.StreamParsingStrategy on agents, but it'd be simpler to replace the current strategy.

The new strategy relies on Java 8 features and improves throughput with less code. I'm happy to make that change, but thought it'd be worth discussing first.

📝 Description

We have a job using this plugin that intermittently hangs. I have been able to reproduce the scenario by creating a job that logs many lines, creating an OutOfMemory error on the agent. A new strategy is included here that allows the plugin to parse builds with more logs than the current approach.

Parsing time is a function of the number of lines in the file and the number of rules. To test this, I defined one rule and ran it on a job that created a set number of lines. 10% of the lines matched the rule. I expect 100 rules run against 100 lines would scale similarly to 10 rules run on 1,000 lines or one rule on 10,000 lines.

Line-Rules	2.3.0 (average seconds over 10 runs)	StreamParsingStrategy
100	0	0
1,000	0	0
10,000	0	0
100,000	0	0
1,000,000	5	3.1
10,000,000	75.4	31.9
20,000,000	Failed (OOM)	66.3
50,000,000	Failed (OOM)	220

I'm using the already-existing log statement in LogParserParser to get this data. For small log files, both are under a second.

💎 Type of change

• Bug fix (non-breaking change which fixes an issue)

🚦 How Has This Been Tested?

Please describe the tests that you ran to verify your changes. Provide instructions so we can reproduce. Please also list any relevant details for your test configuration

I extracted the current code into a ClassicParsingStrategy and introduced a new StreamParsingStrategy. The new classes have unit tests that verify behavior.

For testing large logs, I deployed Jenkins 2.361.2 on an Azure Standard D2s v3 (2 vcpus, 8 GiB memory) and an agent with one executor on another Standard D2s v3. I installed v2.3.0 from the update center to gather the baseline data. I built the plugin locally at this commit, uploaded, and set the system property for the comparison data.

I created a job that would log out 10 lines on a loop, and a rules file with one rule.

build script

echo "warn /warning/" > rules.txt
set +x

for i in $(seq 0 10)

do

echo "> Task :javadocJar $i"

echo "> Task :sourcesJar $i"

echo "> Task :assemble $i"

echo "> Task :codenarcMain $i"

echo "> Task :codenarcTest $i"

echo "> Task :compileTestKotlin NO-SOURCE $i"

echo "> Task :pluginUnderTestMetadata $i"

echo "> Task :compileTestJava NO-SOURCE $i"

echo "> Task :javadoc $i"

echo "/home/runner/work/gradle-jpi-plugin/gradle-jpi-plugin/src/main/java/shaded/hudson/util/VersionNumber.java:551: warning: no @param for idx $i"

done

🏁 Checklist:

• My code follows the style guidelines of this project
• I have performed a self-review of my own code
• I have added tests that prove my fix is effective or that my feature works
• New and existing unit tests pass locally with my changes
• I have commented my code, particularly in hard-to-understand areas

[hypery2k]

nice, I really like that one. Thx