Unsupervised Machine Learning: Algorithms, Types & Example

What is Unsupervised Learning?

Unsupervised Learning is a machine learning technique in which the user does not need to supervise the model. Instead, it allows the model to work on its own to discover patterns and information that were previously undetected. It mainly deals with unlabeled data.

Unsupervised Learning Algorithms

Unsupervised Learning Algorithms allow users to perform more complex processing tasks compared to supervised learning. Unsupervised learning can, however, be more unpredictable than methods trained against known answers. Unsupervised learning algorithms include clustering, anomaly detection, dimensionality reduction and self-organising neural networks.

Example of Unsupervised Machine Learning

Let us take an example of Unsupervised Learning for a baby and her family dog. The first picture shows the pet the baby already recognises.

She knows and identifies this dog. A few weeks later a family friend brings along a dog and tries to play with the baby. That second, unfamiliar dog is shown below.

The baby has not seen this dog earlier. But she recognises that many features (2 ears, eyes, walking on 4 legs) are like her pet dog. She identifies the new animal as a dog. This is unsupervised learning, where you are not taught but you learn from the data (in this case data about a dog). Had this been supervised learning, the family friend would have told the baby that it is a dog, as shown in the above Unsupervised Learning example.

Why Unsupervised Learning?

Here are the prime reasons for using Unsupervised Learning in Machine Learning:

• Unsupervised machine learning finds all kinds of unknown patterns in data.
• Unsupervised methods help you to find features which can be useful for categorization.
• It can run on data as it arrives, so incoming records are analyzed and grouped without waiting for a human to label them first.
• It is easier to get unlabeled data from a computer than labeled data, which needs manual intervention.

Clustering Types of Unsupervised Learning Algorithms

Unsupervised learning problems are further grouped into clustering, association and dimensionality reduction problems. Clustering groups similar records, association finds items that appear together, and dimensionality reduction compresses many features into a few.

Clustering

Clustering is an important concept when it comes to unsupervised learning. It mainly deals with finding a structure or pattern in a collection of uncategorized data. Unsupervised Learning Clustering algorithms will process your data and find natural clusters (groups) if they exist in the data. You can also modify how many clusters your algorithms should identify. It allows you to adjust the granularity of these groups. The diagram below shows scattered records resolved into distinct groups.

There are different types of clustering you can utilize:

Exclusive (partitioning)

In this clustering method, data are grouped in such a way that one record can belong to one cluster only.

Example: K-means

Agglomerative

In this clustering technique, every record starts as its own cluster. The iterative unions between the two nearest clusters reduce the number of clusters.

Example: Hierarchical clustering

Overlapping

In this technique, fuzzy sets are used to cluster data. Each point may belong to two or more clusters with separate degrees of membership.

Here, data will be associated with an appropriate membership value. Example: Fuzzy C-Means

Probabilistic

This technique uses a probability distribution to create the clusters.

Example: The following keywords

• “man’s shoe.”
• “women’s shoe.”
• “women’s glove.”
• “man’s glove.”

can be clustered into two categories, “shoe” and “glove,” or “man” and “women.”

Clustering Types

Following are the algorithms most often met in unsupervised Machine Learning. The first two group records, the last three reduce dimensions rather than form clusters, and K-NN is listed because it is often confused with K-means.

• Hierarchical clustering — clustering
• K-means clustering — clustering
• K-NN (k nearest neighbors) — a supervised classifier, not a clustering method
• Principal Component Analysis — dimensionality reduction
• Singular Value Decomposition — dimensionality reduction
• Independent Component Analysis — dimensionality reduction

Hierarchical Clustering

Hierarchical clustering is an algorithm which builds a hierarchy of clusters. It begins with all the data assigned to a cluster of its own. Here, two close clusters are going to be merged into the same cluster. This algorithm ends when there is only one cluster left. It defines two ideas worth naming separately.

Agglomerative clustering

This bottom-up form of hierarchical clustering does not require the number of clusters K as an input. The agglomeration process starts by forming each record as a single cluster.

This method uses some distance measure and reduces the number of clusters (one in each iteration) by a merging process. Lastly, we have one big cluster containing all the objects, and the analyst cuts the tree at the height that gives a sensible number of groups.

Dendrogram

In the Dendrogram clustering method, each level will represent a possible cluster. The height of the dendrogram shows the level of similarity between two joined clusters. The closer to the bottom of the process they are, the more similar the clusters; choosing the cut that defines the final groups is not automatic and is mostly subjective.

K-means Clustering

K-means is an iterative clustering algorithm which refines the grouping at every iteration. Initially, the desired number of clusters is selected. In this clustering method, you need to cluster the data points into k groups. A larger k means smaller groups with more granularity; a lower k means larger groups with less granularity.

The output of the algorithm is a group of “labels.” It assigns each data point to one of the k groups. In k-means clustering, each group is defined by creating a centroid for that group. The centroids are like the heart of the cluster, which captures the points closest to them and adds them to the cluster.

K- Nearest neighbors

K-nearest neighbour is the simplest of all machine learning classifiers. It differs from other machine learning techniques in that it does not produce a model. It is a simple algorithm which stores all available cases and classifies new instances based on a similarity measure. Because it needs labeled cases to classify against, K-NN is a supervised method; it appears here only because its distance-based logic resembles clustering.

It works very well when there is a meaningful distance between examples. The learning speed is slow when the training set is large, and the distance calculation is nontrivial.

Principal Components Analysis

Principal Component Analysis takes a high-dimensional space and selects a new basis, keeping only its most important scores. Each direction in this basis is known as a principal component. The subset you keep constitutes a new space which is small in size compared to the original space. It maintains as much of the complexity of the data as possible.

Association

Association rules allow you to establish associations amongst data objects inside large databases. This unsupervised technique is about discovering interesting relationships between variables in large databases, and it is a staple of data mining. For example, people that buy a new home are most likely to buy new furniture.

Other Examples:

• A subgroup of cancer patients grouped by their gene expression measurements
• Groups of shoppers based on their browsing and purchasing histories
• Movies grouped by the ratings given by viewers

Supervised vs. Unsupervised Machine Learning

Here is the main difference between Supervised vs. Unsupervised Learning:

Parameters	Supervised machine learning technique	Unsupervised machine learning technique
Input Data	Algorithms are trained using labeled data.	Algorithms are used against data which is not labeled
Computational Complexity	Supervised learning is a simpler method.	Unsupervised learning is computationally complex
Accuracy	Accuracy can be measured directly against known labels.	Accuracy cannot be measured directly; results need interpretation
Typical output	A prediction for each new record	Groups, rules or compressed features

Applications of Unsupervised Machine Learning

Some applications of Unsupervised Learning Techniques are:

• Clustering automatically splits the dataset into groups based on their similarities
• Anomaly detection can discover unusual data points in your dataset. It is useful for finding fraudulent transactions
• Association mining identifies sets of items which often occur together in your dataset
• Latent variable models are widely used for data preprocessing, such as reducing the number of features in a dataset or decomposing the dataset into multiple components

Disadvantages of Unsupervised Learning

• You cannot get precise information regarding data sorting, because the data used in unsupervised learning is unlabeled and its true grouping is not known
• Less accuracy of the results, because the input data is not known and not labeled by people in advance. This means that the machine is required to do this itself.
• The spectral classes do not always correspond to informational classes.
• The user needs to spend time interpreting and labelling the classes that follow from the classification.
• Spectral properties of classes can also change over time, so you cannot keep the same class information while moving from one image to another.

FAQs