File crawler does not scale for deeply nested directories

[aeisenbarth]

We have AnnData files stored together with other data which inherently use a deep folder hierarchy (Zarr). The file crawler successfully skips files that are not *.h5ad, but directories not containing h5ad are still rendered. This produces a huge tree.

Solutions

• A possible solution would be to integrate a proper file browser web component that uses step-by-step navigation (progressive disclosure).
• Keeping the existing file crawler, it would help to also filter directories which have no h5ad as leaf.
  (See Filter directories without h5ad files #77)

[alokito]

This is deployed to pypi in v0.3.11, please open a new ticket if there are any remaining issues.