Mastering PyTorch: A Comprehensive Guide to Deep Learning with PyTorch

Chapter 3: Building Neural Networks with PyTorch

In this chapter, we will dive into the essential concept of neural networks and how to build them using PyTorch. Neural networks are at the core of deep learning, and understanding how to construct and train these models is critical for any machine learning practitioner. We will begin by covering the fundamental components of neural networks, followed by a step-by-step guide to building a simple feedforward neural network for image classification. The goal is to understand the theory behind neural networks while applying it practically using PyTorch.

By the end of this chapter, you will have a solid understanding of how to construct, train, and evaluate basic neural networks in PyTorch. You will also be able to extend these concepts to more complex models such as Convolutional Neural Networks (CNNs) and Recurrent Neural Networks (RNNs).

3.1 Introduction to Neural Networks

A neural network is a collection of layers of interconnected neurons (also known as artificial neurons or perceptrons). The main objective of a neural network is to learn a mapping between the input and output data by adjusting the weights through a process called training.

Neural networks can be visualized as a series of layers:

1. Input Layer: Takes input data for the model.
2. Hidden Layers: Perform computations on the input data to transform it into features that the model can understand.
3. Output Layer: Produces the final prediction.

Each neuron in a layer is connected to neurons in the subsequent layer by weights. During training, these weights are updated through the process of backpropagation, which is powered by gradient descent.

Basic Terminology in Neural Networks:

• Neurons: The basic computational units of a neural network.
• Weights: Parameters that connect neurons across layers.
• Biases: Additional parameters that allow the model to adjust its output.
• Activation Function: A non-linear function applied to the output of each neuron to introduce non-linearity into the model.

Key Components of Neural Networks:

1. Layers: The layers consist of neurons that process inputs, apply weights, and produce outputs.
• Fully Connected (Dense) Layer: Each neuron is connected to every neuron in the previous layer.
2. Activation Functions: These functions are applied to the outputs of the neurons to introduce non-linearity.
• ReLU (Rectified Linear Unit): The most commonly used activation function.
• Sigmoid: Often used for binary classification.
• Softmax: Used in the output layer for multi-class classification tasks.

3.2 Building a Simple Feedforward Neural Network

In PyTorch, neural networks are constructed by subclassing torch.nn.Module and defining the architecture in the __init__ and forward() methods.

Let’s begin by building a simple fully connected neural network to classify images from the MNIST dataset (a dataset of handwritten digits).

Step 1: Load and Preprocess the MNIST Dataset

Before building the model, we need to load the MNIST dataset and preprocess it. We will use the torchvision library to load and transform the data.

Code Sample:

import torch

from torch.utils.data import DataLoader

from torchvision import datasets, transforms

# Define data transformation (convert images to tensor and normalize)

transform = transforms.Compose([transforms.ToTensor(), transforms.Normalize((0.5,), (0.5,))])

# Download and load the MNIST training and test datasets

train_dataset = datasets.MNIST('.', train=True, download=True, transform=transform)

test_dataset = datasets.MNIST('.', train=False, download=True, transform=transform)

# Load data into batches using DataLoader

train_loader = DataLoader(train_dataset, batch_size=64, shuffle=True)

test_loader = DataLoader(test_dataset, batch_size=64, shuffle=False)

Explanation:

• The transform.Compose() function is used to chain multiple transformations (converting to tensor and normalizing).
• We load the MNIST dataset using datasets.MNIST() and apply the transformation.
• We use DataLoader to load data in batches for efficient training and evaluation.

Step 2: Define the Neural Network Architecture

Next, we will define the architecture of our neural network. The network will have one hidden layer and one output layer.

Code Sample:

import torch.nn as nn

import torch.optim as optim

# Define the neural network class

class SimpleNN(nn.Module):

    def __init__(self):

        super(SimpleNN, self).__init__()

        # Input layer (28*28 flattened) -> Hidden layer (128 neurons) -> Output layer (10 classes)

        self.fc1 = nn.Linear(28*28, 128)

        self.fc2 = nn.Linear(128, 10)  # 10 output units for 10 digits

    def forward(self, x):

        x = x.view(-1, 28*28)  # Flatten the input (batch_size, 28, 28) -> (batch_size, 28*28)

        x = torch.relu(self.fc1(x))  # Apply ReLU activation to hidden layer

        x = self.fc2(x)  # Output layer (no activation function)

        return x

# Instantiate the model

model = SimpleNN()

Explanation:

• The SimpleNN class defines a feedforward neural network.
• nn.Linear() defines fully connected layers. The first layer transforms the input of size 28x28 (flattened to 784) to 128 hidden units, and the second layer produces 10 output values (for 10 classes).
• The forward() method applies ReLU activation after the first layer and produces output without an activation function in the final layer (softmax will be applied later during evaluation).

Step 3: Define the Loss Function and Optimizer

We will use CrossEntropyLoss for the loss function, which is suitable for multi-class classification tasks, and Adam for optimization.

Code Sample:

# Define loss function and optimizer

criterion = nn.CrossEntropyLoss()  # Cross-entropy loss for multi-class classification

optimizer = optim.Adam(model.parameters(), lr=0.001)  # Adam optimizer with learning rate 0.001

Explanation:

• CrossEntropyLoss combines log_softmax and nll_loss in one function, suitable for classification tasks.
• Adam optimizer is chosen for its adaptive learning rate and efficiency.

Step 4: Training the Model

Now, we’ll train the model using the training data. In each epoch, we will:

1. Perform a forward pass to calculate the output.
2. Compute the loss.
3. Perform backpropagation to compute gradients.
4. Update the model parameters using the optimizer.

Code Sample:

# Training loop

num_epochs = 5

for epoch in range(num_epochs):

    model.train()  # Set the model to training mode

    running_loss = 0.0

    for data, target in train_loader:

        optimizer.zero_grad()  # Zero the gradients from the previous step

        output = model(data)  # Forward pass

        loss = criterion(output, target)  # Calculate the loss

        loss.backward()  # Backpropagation

        optimizer.step()  # Update the model parameters

        running_loss += loss.item()

    print(f"Epoch {epoch+1}/{num_epochs}, Loss: {running_loss/len(train_loader)}")

Explanation:

• The training loop runs for 5 epochs.
• optimizer.zero_grad() clears the old gradients, loss.backward() computes the new gradients, and optimizer.step() updates the weights.

Step 5: Evaluating the Model

After training, we evaluate the model using the test dataset. The accuracy is calculated by comparing the predicted classes with the actual labels.

Code Sample:

# Evaluation loop

model.eval()  # Set the model to evaluation mode

correct = 0

total = 0

with torch.no_grad():  # Disable gradient calculation for inference

    for data, target in test_loader:

        output = model(data)  # Forward pass

        _, predicted = torch.max(output, 1)  # Get the predicted class

        total += target.size(0)  # Total number of samples

        correct += (predicted == target).sum().item()  # Count correct predictions

accuracy = 100 * correct / total

print(f'Test Accuracy: {accuracy:.2f}%')

Explanation:

• We disable gradient calculation using torch.no_grad() for efficiency during inference.
• torch.max(output, 1) returns the index of the maximum value in each row, which corresponds to the predicted class.

3.3 Common PyTorch Functions for Neural Networks

3.4 Summary

In this chapter, we learned how to:

1. Load and preprocess datasets using DataLoader.
2. Define simple feedforward neural networks in PyTorch by subclassing nn.Module.
3. Implement the forward pass using PyTorch’s nn.Linear and activation functions.
4. Train the model using backpropagation and optimization algorithms like Adam.
5. Evaluate the model on test data and calculate accuracy.

This foundational knowledge will allow you to extend your work to more advanced models like Convolutional Neural Networks (CNNs) and Recurrent Neural Networks (RNNs) in the upcoming chapters.