Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

Compare factors in differential expression #185

Merged
merged 9 commits into from
Dec 28, 2019
Merged

Conversation

eweitz
Copy link
Owner

@eweitz eweitz commented Dec 28, 2019

This enables comparing all pairs of experimental factors in differential expression (DE) datasets.

Previously in #184, only two factors could be compared, e.g. "space flight" and "ground control". This was inadequate for DE datasets with more than two factors, e.g. "space flight", "ground control", and "vivarium control".

See the updated "Differential expression" example:
ideogram_compare_factors_differential_expression_2019-12-27 2019-12-27 20_33_40

For datasets with many factors, naively encoding all possible pairwise combinations in the same Ideogram.js annotation JSON file causes very large file sizes. The number of pairwise comparisons in a dataset with n factors is given by n! / (2! * (n - 2)!).

Left unoptimized, for 20 factors this could yield 30-50 MB JSON files and load times of 10-20 seconds with average U.S. bandwidth -- an unacceptable user experience.

To address that, these changes also split DE datasets with n factors into n JSON files. So instead of a 20-factor DE dataset requiring a file size of 190x, it requires a file size 20x. This graphs shows the relative rates of growth of the old and new data structures.

ideogram_de_space_complexity_before_and_after_optimization_2019-12-27

@coveralls
Copy link

Coverage Status

Coverage increased (+1.0%) to 90.55% when pulling 97e8a00 on deg-compare-factors into aab4ffb on master.

@eweitz eweitz merged commit a6097fd into master Dec 28, 2019
@eweitz eweitz deleted the deg-compare-factors branch February 29, 2020 11:31
# for free to join this conversation on GitHub. Already have an account? # to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants