Topics in Machine Learning

There are various ways to categorize machine learning methods but a common way to think about it is via the following categories, i.e. supervised learning, semi-supervised learning and unsupervised learning. Note that there is another distinction, whether your output is a real-valued number or some sort of classification. Classification can also contain probability measure as in, ‘input X is of class A with a probability of 80%’. Other ways to think of machine learning involve categorizing the methods used by the models or the general idea that the model is using. For now I will use the former method to categorize and list machine learning algorithms. Lastly please note that Artificial Intelligence is the overarching theme, inside of that is machine learning and that deep learning is considered to be a subset of machine learning.

• Supervised Learning - use labeled data to train your algorithm
  • Data Processing and Preliminaries
    • Data Splitting/Partitioning
    • K-fold cross validation
    • Data pre-processing
    • Other topics
  • Bayes, MLE and MAP Preliminaries
  • Linear Regression using Ordinary Least Squares and the Normal Equation,
  • Linear Regression with Regularization
    • Ridge (l2) Regression
    • Lasso(l1) Regression
  • Polynomial regression
  • Logistic Regression Classification
  • Naive Bayes Classification
  • Decision Trees (CART)
  • Support Vector Machine (SVM)
  • Ensemble Methods
    • Random Forest
    • Boosting using AdaBoost
  • Sample machine learning on a house sale price data set
• Semi-supervised Learning - many times acquiring labeled data is either expensive or impractical. So you use the labeled data to train your algorithm as you would with supervised learning. And you use the unlabeled data to possibly infer more information and therefore increase the predictive strength of your model.
  • More on Semi-Supervised Learning
  • Mixture models
  • Expectation Maximization
• Unsupervised Learning - tries to find some sort of structure in the data. Depends heavily on similarity (ex: distance) and/or dissimilarity measures (ex: inverse distance)
  • More on Unsupervised Learning
  • K-Means to find subsets in the data
  • PCA to reduce dimensionality
• Deep Learning - A method that improves on traditional machine learning methods, Deep Learning makes use of neural networks that essentially give the algorithm the ability to process the data in a non-linear fashion. This often results in a performance that beats traditional methods although this comes at the expense of complexity and needing large amounts of data.
  • Introduction to Deep Learning using Tensorflow