Machine Learning Knowledge Graph

Machine learning emerges from the intersection of many fields of study. Important concepts in these areas are related in many ways. The aim with this graph is to highlight the connections between those concepts and, hopefully, help us navigate this complex idea space. Currently, the graph has 206 nodes and 278 edges.

The concepts were classified in 5 categories:

• Mathematics
• Statistics
• Machine Learning
• Optimization
• Artificial Intelligence

A category called "Other" was added to list important related research areas. Some concepts lie on the intersection of fields and are hard to classify. An effort was made to put them where they are used more frequently. The topics covered on the graph are listed below.

Curriculum

• Mathematics
  • Set theory
    • Empty set
    • Finite and infinite sets
    • Operations on sets
      • Complement
      • Union
      • Intersection
    • Sigma-algebra
  • Algebra
    • Linear Algebra
      • Matrix transformation
      • Eigenstuff
      • Matrix decomposition
        • Singular Value Decomposition
        • Non-negative Matrix Factorization
    • Abstract Algebra
  • Calculus
    • Limits
    • Derivatives
      • Partial derivatives
        • Gradient
    • Integrals
    • Taylor series
      • Maclaurin series
    • Fourrier series
      • Fourrier transform
        • Laplace transform
  • Topology
    • Algebraic topology
      • Manifolds
• Optimization
  • Combinatorial Optimization
    • Branch and Bound
  • Convex Optimization
    • Linear Programming
      • Simplex
  • Iterative methods
    • Newton's method
    • Gradient descent
    • Expectation Maximization
      • Baum-Welch algorithm
  • Heuristics
    • Evolutionary algorithms
• Probability
  • Sample Space
  • Kolmogorov axioms
  • Cox's theorem
  • Relative frequency and probability
  • Counting methods
    • Multiplication rule
    • Permutation
    • Combination and Binomial coefficient
    • Arrangement
  • Conditional probability
  • Bayes' Theorem
    • Posterior probability distribution
  • Random Variables
    • Algebra of random variables
    • Expected value
    • Variance
    • Distributions
      • Exponential family
        • Normal distribution
        • Bernoulli distribution
      • Moment-generating function
        • Characteristic function
      • Multivariate distributions
        • Joint distribution
        • Marginal distribution
        • Conditional distribution
  • Probability inequalities
    • Chebyshev's inequality
    • Bernstein inequalities
      • Chernoff bound
      • Hoeffding's inequality
• Statistics
  • Sampling distribution
  • Law of large numbers
  • Central Limit Theorem
  • Resampling
    • Jacknife
    • Bootstrap
  • Monte Carlo method
  • Likelihood function
  • Random Field
    • Stochastic process
      • Time-series analysis
    • Markov Chain
  • Inference
    • Hypothesis testing
      • ANOVA
    • Survival analysis
      • Non-parametric
        • Kaplan–Meier
        • Nelson-Aalen
      • Parametric
        • Cox regression
    • Properties of estimators
      • Quantified properties
        • Error
          • Mean squared error
        • Bias and Variance
          • Unbiased estimator
            • Minimum-variance unbiased estimator (MVUE)
            • Cramér-Rao bound
        • Bias-variance tradeoff
      • Behavioral properties
        • Asymptotic properties
          • Asymptotic normality
          • Consistency
          • Efficiency
        • Robustness
          • M-estimators
    • Multivariate analysis
      • Covariance matrix
      • Dimensionality reduction
        • Feature selection
          • Filter methods
          • Wrapper methods
          • Embedded methods
        • Feature extraction
          • Linear
            • Principal Component Analysis
            • Linear Discriminant Analysis
          • Nonlinear
            • t-SNE
            • UMAP
      • Factor Analysis
    • Mixture models
      • Method of moments
      • Spectral method
    • Parametric inference
      • Regression
        • Linear regression
        • Quantile regression
        • Autoregressive models
        • Generalized Linear Models
          • Logistic regression
          • Multinomial regression
    • Bayesian Inference
      • Sampling Bayesian Methods
        • MCMC
          • Hamiltonian Monte Carlo
      • Approximate Bayesian Methods
        • Variational inference
        • Integrated Nested Laplace Approximation
      • Maximum a posteriori estimation
    • Probabilistic Graphical Models
      • Bayesian Networks
        • Hidden Markov Models
      • Markov Random Field
        • Boltzmann machine
      • Latent Dirichlet Allocation
      • Conditional Random Field
    • Nonparametric inference
      • Additive models
        • Generalized additive models
      • Kernel density estimation
    • Generative and discriminative models
• Machine Learning
  • Statistical Learning Theory
    • Vapnik-Chervonenkis theory
    • Hypothesis set
      • Inductive bias
        • No free lunch theorem
    • Loss function
    • Regularization
      • LASSO
      • Ridge
      • Elastic Net
      • Early stopping
      • Dropout
  • Cross-validation
    • Hyperparameter optimization
    • Automated Machine Learning
  • k-NN
  • Naive Bayes
  • Support Vector Machines
    • Kernel trick
  • Decision trees
    • Random Forest
  • Neural Networks
    • Training
      • Backpropagation
      • Activation function
        • Sigmoid
        • Softmax
        • Tanh
        • ReLU
    • Architecture
      • Feedforward networks
        • Perceptron
        • Multilayer perceptron
          • Convolutional Neural Networks
            • Deep Q-Learning
            • Temporal Convolutional Networks
        • Autoencoder
          • Variational autoencoder
      • Recurrent networks
        • LSTM
        • Hopfield networks
      • Restricted Boltzmann machine
        • Deep Belief Network
  • Adversarial Machine Learning
    • Generative Adversarial Networks
  • Ensemble
    • Bagging
    • Boosting
    • Stacking
  • Meta-learning
  • Sequence models
• Artificial Intelligence
  • Symbolic AI
    • Logic-based AI
      • Automated reasoning
  • Search Problems
    • A* search algorithm
    • Decision Theory
      • Game Theory
        • Zero-sum game
          • Minimax
        • Non-zero-sum game
  • Cybernetics
    • Computer vision
    • Robotics
    • Natural Language Processing
      • Language model
        • Unigram model
      • Topic model
        • Text classification
          • Sentiment analysis
          • Word representation
            • Bag-of-words
            • Word embedding
              • Word2vec
              • Latent Semantic Analysis
      • Natural Languange Understanding
        • Speech recognition
        • Question answering AI
        • Text summarization
        • Machine translation
      • Information Retrieval (IR)
        • Probabilistic IR models
        • Information filtering system
          • Recommender system
            • Collaborative filtering
            • Content-based filtering
            • Hybrid recommender systems
      • Turing test
• Other
  • Complexity Theory
  • Statistical physics
    • Hamiltonian mechanics
    • Ising model
  • Information Theory
    • Entropy
    • Kullback–Leibler divergence
    • Signal processing
      • Kalman filter

FAQ

• Why make a distinction between Machine Learning and Artificial Intelligence?

I followed the approach explored by Russel and Norvig [1]. In that sense, Artificial Intelligence is a broader field that encompasses Machine Learning.

References

[1] Russell, S. J., & Norvig, P. (2016). Artificial intelligence: a modern approach. Malaysia; Pearson Education Limited,.