feat(pipelineRef): Replace PipelineTrigger with PipelineRef for Spinnaker UI

[edgarulg]

Feature: Performance Optimization for Nested Pipelines in Spinnaker Orca
In Spinnaker, pipelines with many nested child pipelines (e.g., Pipeline → Pipeline → Pipeline → Pipeline) can cause significant delays in loading times, particularly for large instances. This is because the Spinnaker UI often needs to load parent executions for any pipeline that was triggered by another, which unnecessarily increases the load on the Orca execution repository.

Solution:
To address this issue, I’ve introduced an optimization using the PipelineRef feature that was previously delivered. The idea is to propagate the PipelineRef trigger instead of the full PipelineTrigger for nested pipelines. This reduces the load on Orca and improves UI performance.

How It Works:

ExecutionRepository Update:
I added a new flag, includeNestedExecutions, to the retrieve method of the ExecutionRepository interface.

Default Behavior:
Existing execution repository implementations don’t need to change.

• The default behavior remains to use the normal retrieve method.

SqlExecutionRepository Logic:
The only repository that requires an update is the SqlExecutionRepository, where the business logic for includeNestedExecutions is implemented.

• If includeNestedExecutions is true, Orca will convert any PipelineRefTrigger into a PipelineTrigger to ensure nested executions are returned.

• If the flag is false, Orca will return executions with the PipelineRefTrigger.

SPeL Compatibility:
To maintain backward compatibility with Spinnaker’s SPeL, the OrcaMessageHandler ensures that nested executions are included when evaluating any expressions. This guarantees that SPeL expressions still resolve correctly.

SPeLAutoComplete Compatibility:
The SPeLAutoComplete feature continues to work as expected. The endpoint that retrieves previous executions to feed the autocomplete is still retrieving executions with the full execution context, including nested executions. This ensures that the autocomplete functionality is unaffected by the performance improvement.

Summary:
The main idea is to minimize unnecessary load on Orca by returning a PipelineRefTrigger for external requests, while keeping full execution context for internal modules that need it. This change significantly reduces the pressure on the Orca execution repository and improves UI performance, especially in large Spinnaker instances.

This performance improvement is backward compatible, and SPeLAutoComplete continues to function as expected, with no disruption to existing Spinnaker functionality.

How to enable it

executionRepository:
  redis:
    enabled: false
  sql:
    enabled: true
    pipelineRef:
      enabled: true

Here is an example on how the Spinnaker UI looks like when a pipeline is triggered by another pipeline:

Here is another example on how the execution looks like if we inspect it in the Spinnaker UI:

Finally here is another example that probes the SPeLAutoComplete still works by converting PipelineRefTrigger to PipelineTrigger and existing SPeLs still works as expected:

[edgarulg]

I will add more tests around TaskController and endpoints.

[edgarulg]

I added all tests I have in mind for this feature. This is ready now.