Memory usage of example modules

[moyner]

This was discussed on Slack, making an issue in case others run into the same problem.

When Documenter builds examples it creates a local module for each named example:

Documenter.jl/src/expander_pipeline.jl

Line 768 in 1dce985

# The sandboxed module -- either a new one or a cached one from this page.

The issue is that these modules persist both after all blocks of an example has been passed and also after makedocs finishes. If the examples only make use of a trivial amount of memory, this is no issue, but if examples allocate significant amounts of memory (e.g. some large arrays or loading of test data) the memory is lost. For free GitHub runners this can lead to jobs being cancelled, as the build process gets OOM killed.

In my case, my big examples were all generated through Literate.jl, so the solution was to add the following at the end of each example based on a Discourse discussion:

function clear_module!(M::Module)        # hide
    for name ∈ names(M, all=true)        # hide
        if !isconst(M, name)             # hide
            @eval M $name = $nothing     # hide
        end                              # hide
    end                                  # hide
end                                      # hide
clear_module!(@__MODULE__)               # hide
GC.gc();                                 # hide

Ideal behavior would be that Documenter could parse the order of named example blocks, figure out when they are no longer needed for the build and make sure that the variables in the module are then available to GC.