You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
The MultiSourceFileFinder is a special case of the DefaultImporterFactory which has a different behaviour than other ImporterFactories. It was created to handle when we have multiple files that need to be fetched for a single entity, ie all the files retrieved in a single source entry are will be passed as input for processing the said entity.
This is used for the alignment importer, where we have multiple different files that need to be fetched for a single alignment entity.
This is a low-lift solution and can be a little opaque for the config creator on the behaviour of how the files are processed. This should be refactored to be a finder which supports all the different cases such as source_glob, source_multi_glob, literal, etc. That will make it more intuitive.
Definition of Done
There is a new finder for this specific usecase
All the existing configs that utilize this ImporterFactory have been updated to the new finder.
The existing MultiSourceFileFinder has been sunset
The text was updated successfully, but these errors were encountered:
Motivation
The
MultiSourceFileFinder
is a special case of the DefaultImporterFactory which has a different behaviour than other ImporterFactories. It was created to handle when we have multiple files that need to be fetched for a single entity, ie all the files retrieved in a single source entry are will be passed as input for processing the said entity.This is used for the alignment importer, where we have multiple different files that need to be fetched for a single alignment entity.
This is a low-lift solution and can be a little opaque for the config creator on the behaviour of how the files are processed. This should be refactored to be a finder which supports all the different cases such as source_glob, source_multi_glob, literal, etc. That will make it more intuitive.
Definition of Done
MultiSourceFileFinder
has been sunsetThe text was updated successfully, but these errors were encountered: