awesome-ai

Awesome list for all things AI, ML and deep learning

Table of Contents

• awesome-ai
  • Concepts
    • Neural Machine Translation - attention mechanisms
    • Transformer
    • Universal Transformers
  • node2vec
  • Topic: graph analytics
  • Tooling
    • general purpose
    • probalistic inference
    • graph
  • Tutorials
  • Research papers / blog articles
    • NLP: methods
    • NLP: applications
    • Transformer applications
  • Awesome lists
    • General
    • Frameworks / ecosystems
    • Awesome papers
    • NLP
    • Graph theory
    • Knowledge graph
    • Notebooks

Concepts

Neural Machine Translation - attention mechanisms

Visualizing A Neural Machine Translation Model (Mechanics of Seq2seq Models With Attention)

https://jalammar.github.io/visualizing-neural-machine-translation-mechanics-of-seq2seq-models-with-attention/

Transformer

A model that uses attention to boost the speed with which these models can be trained

http://jalammar.github.io/illustrated-transformer/

Universal Transformers

The Universal Transformer is an extension to the Transformer models which combines the parallelizability and global receptive field of the Transformer model with the recurrent inductive bias of RNNs, which seems to be better suited to a range of algorithmic and natural language understanding sequence-to-sequence problems.

http://mostafadehghani.com/2019/05/05/universal-transformers/

node2vec

An algorithmic framework for learning useful representation from highly structured objects such as graphs.

http://sujitpal.blogspot.com/2019/05/node2vec-graph-embeddings-for-neurips.html?utm_source=feedburner&utm_medium=email&utm_campaign=Feed%3A+SalmonRun+%28Salmon+Run%29

Topic: graph analytics

• Benchmark of popular graph/network packages in python/r

Tooling

general purpose

probalistic inference

graph

• dgl: Python package built to ease deep learning on graph, on top of existing DL frameworks.
• cdlib: Community Discovery Library - (for networkx and igraph)
• Embedding-Vis
• OpenNE

web resources

• clocate.com; search conferences

Tutorials

• fastai: practical deep learning for coders
• practicalAI
• Machine-Learning-Tutorials

Research papers / blog articles

NLP: methods

• Parikh, A.P., Täckström, O., Das, D. and Uszkoreit, J., 2016. A decomposable attention model for natural language inference. arXiv preprint arXiv:1606.01933.
• Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł. and Polosukhin, I., 2017. Attention is all you need. In Advances in neural information processing systems (pp. 5998-6008).
• Devlin, J., Chang, M.W., Lee, K. and Toutanova, K., 2018. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805.
• The illustrated GPT-2 (Visulaizing Transformer Language Models)
• Transformers from scratch
• Searching for Answer Candidate Passages with Solr and Anserini
• Subramanian et al., 2019. On extractive and abstractive neural document summarization with transformer language models

NLP: applications

• Wang et al., 2018. Ontology Alignment in the Biomedical Domain Using Entity Definitions and Context
• Wang et al., 2019., Extracting evidence of supplement-drug interactions from literature

Transformer applications

• Bert2Tag: BERT key phrase tagging

Awesome lists

General

• awesome-deep-learning
• awesome-deep-learning-resources

Frameworks / ecosystems

• awesome-pytorch-list
• the-incredible-pytorch
• awesome-ternsorflow
• keras-resources
• DataSciencePython
• DataScienceR

Awesome papers

• awesome-deep-learning-papers
• awesome-decision-tree-papers
• awesome-gradient-boosting-papers
• must read papers on graph neural network
• deep learning biology
• awesome-deepbio

NLP

• nlp-roadmap
• NLP-progress
• awesome-nlp
• awesome-bert
• awesome-word2vec
• awesome-embedding-models
• NCBI BioNLP Research Group (PI: Zhiyong Lu)
• awesome-bert
• thunlp awesome paper repos
  • neural relation extraction (NRE)
  • machine reading comprehension
  • network representation learning (NRL) / network embedding (NE)
  • graph neural networks (GNN)
  • knowledge representation learning (KRL) / knowledge embedding (KE)
• the super duper NLP repo

Graph theory

• awesome-community-detection
• awesome-graph-classification
• awesome-network-analysis
• awesome-network-embedding
• graph-based-deep-learning-literature
• LiteratureDL4Graph

Knowledge graph

• shaoxiongji/awesome-knowledge-graph
• totogo/awesome-knowledge-graph
• BrambleXu/knowledge-graph-learning
• husthuke/awesome-knowledge-graph (in Chinese)

Notebooks

• NLP graphs