PR: Implement support for spectral data lazy loading and reshape cache. #840
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR implements support for lazy loading of spectral data so that importing Colour is significantly faster, i.e. ~= 1.5sec vs
~5.5sec for 0.3.16 on my old Macbook Pro 2014.
Note: Some of the import improvements are also pertaining to some changes I made in the
colour.plotting.tm3018
module.The shipped spectral data is not instantiated until first used after full package import. The implementation uses the
colour.utilities.LazyCaseInsensitiveMapping
class that initially stores callables that are replaced by the data when an item is being requested:A typical 0.3.16 dataset is as follows:
it is now defined as follows:
In order to be able to compose datasets during import and avoid instantiation
to spectral data, the process is prevented by the
colour.__disable_lazy_load__
attribute that is only set after the package is fully imported at the very end ofcolour.__init__
.This new mechanism also means that we have to update the signature of all the
objects that use a default spectral distribution or multi-spectral distribution
as a default argument otherwise, they are defined with the
partial
as a default argument which is unsuitable.It is worth noting that this does not affect packages using
Colour
because it isColour
import time "magic" only.I took that opportunity to introduce high-level wrappers, i.e.
colour.colorimetry.reshape_sd
andcolour.colorimetry.reshape_msds
, that handle alignement, extrapolation, interpolation and trimming while avoiding to have to make a copy systematically. The other benefit is that we can also cache the operation, anda subsequent call for the same spectral data with exact same reshaping arguments will use the cached spectral data instead of regenerating, thus better performance.
A typical 0.3.16 definition is as follows:
It is now as follows: