Visualizing Confusion Matrix for MNIST with PyTorch and Tensorboard

Author

Posted Nov 6, 2024

Reads 1.1K

Person Facing a Big Screen with Numbers
Credit: pexels.com, Person Facing a Big Screen with Numbers

In PyTorch, we can use the `nn.CrossEntropyLoss()` function to calculate the loss between our model's output and the true labels. This function returns the cross-entropy loss, which is a measure of the difference between our model's predictions and the actual labels.

The MNIST dataset consists of 60,000 training images and 10,000 testing images, each with 784 features (28x28 grayscale images). We can use the `torchvision.datasets.MNIST` function to load the dataset.

To visualize the confusion matrix, we'll use the `torchvision.transforms` module to normalize the images. We'll also use the `matplotlib` library to create the confusion matrix plot.

The confusion matrix will show us the number of true positives, true negatives, false positives, and false negatives for each class in the MNIST dataset. This will give us a better understanding of our model's performance on the test set.

Calculating Confusion Matrix

Calculating the confusion matrix is a crucial step in evaluating the performance of our MNIST model.

Credit: youtube.com, How to Build and Interpret Confusion Matrix Using Python & Sklearn

We will define a function that calculates the confusion matrix.

In PyTorch, we can use the torchmetrics library to calculate the confusion matrix.

The confusion matrix is a table that summarizes the predictions made by our model against the actual labels of the test data.

To create a confusion matrix, we need to know the true labels and the predicted labels of the test data.

MNIST with PyTorch

MNIST is a classic dataset for image classification tasks, consisting of 70,000 28x28 grayscale images of handwritten digits from 0 to 9.

The MNIST dataset is commonly used to evaluate the performance of deep learning models, including those built with PyTorch.

To work with MNIST in PyTorch, you'll need to import the necessary libraries, including torch and torchvision, and load the dataset using the MNIST class from torchvision.datasets.

The MNIST dataset is divided into training and testing sets, with 60,000 images in the training set and 10,000 images in the testing set.

Setting Up Tensorboard Logging

Credit: youtube.com, PyTorch Lightning #8 - Logging with TensorBoard

Setting up Tensorboard logging is a crucial step in monitoring your model's performance. We'll use the tensorboard callback to log the confusion matrix on epoch end.

Tensorboard is a powerful tool for visualizing your model's performance. It can be used to track metrics such as accuracy, loss, and more.

To set up tensorboard logging, you'll need to import the Tensorboard callback from PyTorch. This can be done with a simple import statement.

The tensorboard callback can be added to your model's training loop to log metrics on each epoch end. This will allow you to track your model's performance over time.

You can then use Tensorboard to visualize your model's performance, including the confusion matrix. This will give you a better understanding of how well your model is performing on the MNIST dataset.

Tensorboard with Fashion MNIST

Tensorboard with Fashion MNIST is a great tool for visualizing your model's performance.

You can use it to track your model's metrics and visualize your data, as seen in the example where a CNN classifier is trained on the Fashion MNIST dataset and a confusion matrix is set up.

Credit: youtube.com, #Python | PyTorch ConvNet ... classify FashionMNIST clothing! | #NeuralNetwork #ComputerVision

A confusion matrix is a table used to evaluate the performance of a classification model, and it's essential to use it when working with the Fashion MNIST dataset.

The Fashion MNIST dataset is a large dataset of 70,000 28x28 grayscale images of clothing items, and it's a great resource for training and testing your models.

You can refer to the Github repo link to see a real-world example of using Tensorboard with Fashion MNIST, where the author has trained a CNN classifier and set up a confusion matrix.

Returning Results Return only the revised heading

The confusion matrix is a table layout that allows visualization of the performance of an algorithm, typically a supervised learning one.

Each row of the matrix represents the instances in a predicted class, while each column represents the instances in an actual class.

The matrix is formatted such that columns are predictions and rows are targets, making it easy to see if the system is confusing two classes.

Credit: youtube.com, The Confusion Matrix in Machine Learning

You can plot the matrix, correctly assigning the label "predicted values" to the horizontal axis and "actual values" to the vertical axis.

In case of the targets y in (batch_size, …) format, target indices between 0 and num_classes only contribute to the confusion matrix and others are neglected.

If you're doing binary classification with a single output unit, you may have to transform your network output into a one-hot vector.

The metric's state can be updated using the passed batch output, and by default, this is called once for each batch.

Sources

  1. matrix (wikipedia.org)
  2. ConfusionMatrix — PyTorch-Ignite v0.5.1 Documentation (pytorch.org)
  3. confusion matrix (wikipedia.org)
  4. Colormap (matplotlib.org)
  5. Figure (matplotlib.org)
  6. MNIST (lecun.com)
  7. some extra documentation (scikit-learn.org)

Jay Matsuda

Lead Writer

Jay Matsuda is an accomplished writer and blogger who has been sharing his insights and experiences with readers for over a decade. He has a talent for crafting engaging content that resonates with audiences, whether he's writing about travel, food, or personal growth. With a deep passion for exploring new places and meeting new people, Jay brings a unique perspective to everything he writes.

Love What You Read? Stay Updated!

Join our community for insights, tips, and more.