Changing Index into One-Hot Encoding with Pytorch
In this post we cover how to use torch.nn.functional to encode a integer label (e.g. 4 or 0 or 6) into a one-hot encoded tensor (e.g. [0, 0, 0, 1, 0, 0] or [1, 0, 0, 0, 0] or [0, 0, 0, 0, 1])
Why Use One-Hot Encoder
In many classification tasks the use of one-hot encoding of labels is needed by some loss functions, this is especially true when there is more than 2 classes.
The benefit of one-hot encoding is that the neural network can have an output neuron for each class, when a Softmax is applied just before the output results in an output tensor where all values add up to 1.0, the output number for each class is often read as likelihood for that class.
Example
An example could be a neural network trained (or being trained) to predict 3 classes such as cat, dog and cow is given a photo of a cow; this might result in an output tensor that looks like this [.05, 0.5, .90] therefore we might describe this as having 5% likelihood of being cat or dog and 90% of being a cow.
The ground truth tensor for this case might look like [0, 0, 1] and the tensor of the difference between the predictions and ground truth could be described as [.05, .05, .1]
Reproducing Code
This code was created and tested using Environment E037
import torch
def one_hot_encode_demo(num_classes = 2, idx_len = 10):
print('Randomly generating index.....\n')
idx = torch.Tensor(idx_len).random_(0, num_classes)
print('The index is now', idx)
print('\nCreating one-hot encoded tensor....')
one_hot = torch.nn.functional.one_hot(idx.to(torch.int64), num_classes)
print('\nThe one-hot encoded index is now:')
print(one_hot)
Now we create a very simple example with 2 classes to demonstrate the basic concept
one_hot_encode_demo(num_classes = 2)
Now we create a more complex example with 10 classes
one_hot_encode_demo(num_classes = 10)
As a final example we create a tensor of 3 classes and a length of 25
one_hot_encode_demo(num_classes = 3, idx_len = 25)