Types of ML approaches

ML has as essential attribute 'learning' that is acquiring knowledge or skills from experience, at a broad-level it could be classified into supervised, unsupervised, reinforced, hybrid and transfer learning. But there are sub-categories, which are distinct enough to be mentioned separately.

1. Supervised Learning where the output variable (the one you want to predict) is labeled in the training dataset (data used to build the Machine Learning model). Techniques include Decision Trees, Random Forests, Support Vector Machines, Bayesian Classifier etc. For instance, predicting whether a given email is SPAM or not, given sample emails with the labels whether they are SPAM or not, falls within Supervised learning. In this approach, Models are fit on training data comprised of inputs and outputs and used to make predictions on test sets where only the inputs are provided and the outputs from the model are compared to the withheld target variables and used to estimate the skill of the model. There are two main types of supervised learning problems: they are classification that involves predicting a class label and regression that involves predicting a numerical value.

   1. Classification: Supervised learning problem that involves predicting a class label.

   2. Regression: Supervised learning problem that involves predicting a numerical label.

2. Unsupervised Learning where the training dataset does not contain the output variable. The objective is to group the similar data together instead of predicting any specific value. Clustering, Density estimation, Dimensionality Reduction, Principal component analysis and Anomaly Detection are some of the Unsupervised Learning techniques. For instance, grouping the customers based on their purchasing pattern.

3. Reinforcement Learning: Unlike traditional Machine Learning techniques, Reinforcement Learning focuses on finding a balance between Exploration (of unknown new territory) and Exploitation (of current knowledge). It monitors the response of the actions taken through trial and error and measures the response against a reward. The goal is to take such actions for the new data so that the long-term reward is maximized. Let’s say that you are in an unknown terrain, and each time you step on a rock, you get negative reward whereas each time you find a coin, you get a positive reward. In traditional Machine Learning, at each step, you would greedily take such an action whose immediate reward is maximum even though there might be another path for which the overall reward is more. In Reinforcement Learning, after every few steps, you take a less greedy step to explore the full terrain. After much exploration and exploitation, you would know the best way to walk through the terrain so as to maximize your total reward.

4. Hybrid learning has several sub-categories. Semi-supervised learning is more real world problems, where the labeled data is fairly sparse as compared to overall training data. This approach is combinatorial in which unlabeled data could be clustered using unsupervised approach and then synthetic or real labels could be used to generate a larger set of labeled data for supervised learning. Speech synthesis or web-content classification fall under this category. Self-supervised algorithms are used as a pretext part for solving a supervised learning problem. A prime example of such an approach is an autoencoder, which creates a compressed representation of the input. Autoencoders are trained on the input in such a way that using its decoder it can generate the same output as the input. Once trained, the decoder is discarded and the encoder is used as needed to create compact representations of input. An example problem for this approach could be generating a colored picture from its grey-scale input. Another example of self-supervised learning is GAN. GAN models are trained indirectly via a separate discriminator model that classifies examples of photos from the domain as real or fake (generated), the result of which is fed back to update the GAN model and encourage it to generate more realistic photos on the next iteration.

5. Ensemble learning is an approach where two or more modes are fit on the same data and the predictions from each model are combined. The objective of ensemble learning is to achieve better performance with the ensemble of models as compared to any individual model. This involves both deciding how to create models used in the ensemble and how to best combine the predictions from the ensemble members.