Authors | Haibao Tang (tanghaibao) |
DV Klopfenstein (dvklopfenstein) | |
Brent Pedersen (brentp) | |
Fidel Ramirez (fidelram) | |
Aurelien Naldi (aurelien-naldi) | |
Patrick Flick (patflick) | |
Jeff Yunes (yunesj) | |
Kenta Sato (bicycle1885) | |
Chris Mungall (cmungall) | |
Greg Stupp (stuppie) | |
David DeTomaso (deto) | |
Olga Botvinnik (olgabot) | |
tanghaibao@gmail.com | |
License | BSD |
Tip
GOATOOLS is now published in Scientific Reports!
Klopfenstein DV, ... Tang H (2018) GOATOOLS: A Python library for Gene Ontology analyses Scientific reports
- GO Grouping: Visualize the major findings in a gene ontology enrichment analysis (GOEA) more easily with grouping. A detailed description of GOATOOLS GO grouping is found in the manuscript.
- Compare GO lists:
Compare two or more lists
of GO IDs using
compare_gos.py
, which can be used with or without grouping. - Stochastic GOEA simulations: One of the findings resulting from our simulations is: Larger study sizes result in higher GOEA sensitivity, meaning fewer truly significant observations go unreported. The code for the stochastic GOEA simulations described in the paper is found here
This package contains a Python library to
-
Process over- and under-representation of certain GO terms, based on Fisher's exact test. With numerous multiple correction routines including locally implemented routines for Bonferroni, Sidak, Holm, and false discovery rate. Also included are multiple test corrections from statsmodels: FDR Benjamini/Hochberg, FDR Benjamini/Yekutieli, Holm-Sidak, Simes-Hochberg, Hommel, FDR 2-stage Benjamini-Hochberg, FDR 2-stage Benjamini-Krieger-Yekutieli, FDR adaptive Gavrilov-Benjamini-Sarkar, Bonferroni, Sidak, and Holm.
-
Process the obo-formatted file from Gene Ontology website. The data structure is a directed acyclic graph (DAG) that allows easy traversal from leaf to root.
-
Read GO Association files:
- GAF (GO Annotation File)
- GPAD (Gene Product Association Data)
- NCBI's gene2go file
- id2gos format. See example
-
Print decendants count and/or information content for a list of GO terms
-
Get parents or ancestors for a GO term with or without optional relationships, including Print details about a GO ID's parents
-
Compare two or more lists of GO IDs
-
Group GO terms for easier viewing
-
Map GO terms (or protein products with multiple associations to GO terms) to GOslim terms (analog to the map2slim.pl script supplied by geneontology.org)
Make sure your Python version >= 3.7, and download an
.obo
file of the most current
GO:
wget http://current.geneontology.org/ontology/go-basic.obo
or .obo
file for the most current GO
Slim terms (e.g.
generic GOslim) :
wget http://current.geneontology.org/ontology/subsets/goslim_generic.obo
pip install goatools
To install the development version:
pip install git+git://github.com/tanghaibao/goatools.git
conda install -c bioconda goatools
When installing via PyPI or Bioconda as described above, all dependencies are automatically downloaded. Alternatively, you can manually install:
-
For statistical testing of GO enrichment:
scipy.stats.fisher_exact
statsmodels
(optional) for access to a variety of statistical tests for GOEA
-
To plot the ontology lineage, install one of these two options:
- Graphviz, for graph visualization.
- pygraphviz, Python binding for communicating with Graphviz:
- pydot, a Python interface to Graphviz's Dot language.
run.sh
contains example cases, which calls the utility scripts in the
scripts
folder.
See examples in find_enrichment
The find_enrichment.py
takes as arguments files
containing:
- gene names in a study
- gene names in population (or other study if
--compare
is specified) - an association file that maps a gene name to a GO category.
Please look at tests/data
folder to see examples on how to make these
files. when ready, the command looks like:
python scripts/find_enrichment.py --pval=0.05 --indent data/study \
data/population data/association
and can filter on the significance of (e)nrichment or (p)urification. it can report various multiple testing corrected p-values as well as the false discovery rate.
The e
in the "Enrichment" column means "enriched" - the concentration
of GO term in the study group is significantly higher than those in
the population. The "p" stands for "purified" - significantly lower
concentration of the GO term in the study group than in the population.
Important note: by default, find_enrichment.py
propagates counts
to all the parents of a GO term. As a result, users may find terms in
the output that are not present in their association
file. Use
--no_propagate_counts
to disable this behavior.
wr_hier.py
: Given a GO ID, write the hierarchy below (default) or above (--up
) the given GO.
go_plot.py
:- Plots user-specified GO term(s) up to root
- Multiple user-specified GOs
- User-defined colors
- Plot relationships (
-r
) - Optionally plot children of user-specfied GO terms
plot_go_term.py
can plot the lineage of a certain GO term, by:
python scripts/plot_go_term.py --term=GO:0008135
This command will plot the following image.
Sometimes people like to stylize the graph themselves, use option
--gml
to generate a GML output which can then be used in an external
graph editing software like Cytoscape. The
following image is produced by importing the GML file into Cytoscape
using yFile orthogonal layout and solid VizMapping. Note that the GML
reader plugin may need to be
downloaded and installed in the plugins
folder of Cytoscape:
python scripts/plot_go_term.py --term=GO:0008135 --gml
See map_to_slim.py
for usage. As arguments it takes the gene ontology
files:
- the current gene ontology file
go-basic.obo
- the GOslim file to be used (e.g.
goslim_generic.obo
or any other GOslim file)
The script either maps one GO term to its GOslim terms, or protein products with multiple associations to all its GOslim terms.
To determine the GOslim terms for a single GO term, you can use the following command:
python scripts/map_to_slim.py --term=GO:0008135 go-basic.obo goslim_generic.obo
To determine the GOslim terms for protein products with multiple associations:
python scripts/map_to_slim.py --association_file=data/association go-basic.obo goslim_generic.obo
Where the association
file has the same format as used for
find_enrichment.py
.
The implementation is similar to map2slim.
For calculating uncorrected p-values, we use SciPy:
We have implemented several significance tests:
bonferroni
, bonferroni correctionsidak
, sidak correctionholm
, hold correctionfdr
, false discovery rate (fdr) implementation using resampling
Additional methods are available if statsmodels
is installed:
sm_bonferroni
, bonferroni one-step correctionsm_sidak
, sidak one-step correctionsm_holm-sidak
, holm-sidak step-down method using Sidak adjustmentssm_holm
, holm step-down method using Bonferroni adjustmentssimes-hochberg
, simes-hochberg step-up method (independent)hommel
, hommel closed method based on Simes tests (non-negative)fdr_bh
, fdr correction with Benjamini/Hochberg (non-negative)fdr_by
, fdr correction with Benjamini/Yekutieli (negative)fdr_tsbh
, two stage fdr correction (non-negative)fdr_tsbky
, two stage fdr correction (non-negative)fdr_gbs
, fdr adaptive Gavrilov-Benjamini-Sarkar
In total 15 tests are available, which can be selected using option
--method
. Please note that the default FDR (fdr
) uses a resampling
strategy which may lead to slightly different q-values between runs.
iPython Notebooks
goea_nbt3102 human phenotype ontologies
semantic_similarity semantic_similarity_wang
Prior to submitting your pull request, please add a test which verifies your code, and run:
make test
Items that we know we need include:
-
Add code coverage runs
-
Edit tests in the
makefile
under the comment -
Help setting up documentation. We are using Sphinx and Python docstrings to create documentation. For documentation practice, use make targets:
make mkdocs_practice
To remove practice documentation:
make rmdocs_practice
Once you are happy with the documentation do:
make gh-pages
Copyright (C) 2010-2021, Haibao Tang et al. All rights reserved.