Lazy loading

Lazy Loading of Libraries

Various RAVEN features can use different libraries. Some of these libraries can take noticeable time to load. If they are only used for some inputs, they should be lazily loaded so they are not loaded for inputs that do not use them.

Methods of Lazily loading

Use lazyImporterUtils

The lazyImporterUtils can be used to do lazy imports. First the module is imported. It will be imported the first time that any attributes of the module are used.

import utils.lazyImporterUtils
tf =  utils.lazyImporterUtils.import_module_lazy_renamed("tf", globals(), "tensorflow")
...
self.availLayer['dense'] = tf.keras.layers.Dense

If renaming the module is not desired then the shorter form can be used:

import utils.lazyImporterUtils
tensorflow =  utils.lazyImporterUtils.import_module_lazy("tensorflow", globals())

Load module in function

If the module is only used in one function, then the module can just be loaded that function.

def someFunc():
  import sklearn.neighbors
  nr =  sklearn.neighbors.KNeighborsRegressor(...)

How to load plugins lazily

PluginFactory does not load the plugin immediately. In order to load the plugin, PluginFactory.finishLoadPlugin needs to be called.

So something like:

#found something.stuff in input file
PluginFactory.finishLoadPlugin("something")
#then can check for something.stuff

Modules that always must be lazily loaded

These are modules that are time consuming to load and only sometimes needed, so they should only be loaded when the input requires them.

• tensorflow
• statsmodels
• scikit-learn (sklearn)

Modules that can be directly loaded

These are modules that practically every RAVEN input will need, so they can just be loaded directly.

• numpy
• pandas
• xarray