Remove experimental shredder feature #57
Merged
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
Removes the
getResourceForNode
method from the shredder base class. This method was originally intended to provide a feature where the resource name could be determined from name of a node used by a job on a given day. This would allow for defining sub-resources within a given resource. It was never fully implemented and the removed method effectively returned the resource name that was set for a shredder instance (since the resource name is always required by thexdmod-shredder
command).The tests have also been improved to ensure that the resource name corresponds to the one which is set and not the resource name that may be contained in the accounting data (which is what is expected elsewhere in the ingestion process).
Motivation and Context
Since this feature was never fully implemented it provides no benefits and makes the code more complicated. Removing it simplifies the code and makes it easier to understand. It also fixes an issue where parsing the end time in SGE logs files fails.
Tests performed
Updated unit tests.
Types of changes
Checklist: