Understanding Convolutional Neural Networks

Convolutional Neural Networks (CNNs) are a class of deep neural networks, particularly adept at analyzing visual imagery. They are designed to automatically and adaptively learn spatial hierarchies of features from input images. CNNs have revolutionized the field of computer vision and are widely used in tasks such as Image classification, Object detection, & Segmentation.

What is Convolution?

Convolutions are a mathematical operation that plays a critical role in the functioning of CNNs. Unlike traditional fully connected neural networks, where each neuron is connected to every other neuron in the next layer, CNNs use convolutions to process data in smaller, localized regions. This makes them highly efficient for tasks involving spatial hierarchies, such as image and video analysis.

Convolution involves sliding a filter (also known as a kernel) over an input (like an image) to produce a transformed output. This process helps in identifying and enhancing certain patterns in the input data, such as edges, textures, or other significant features.

The key idea behind convolution is to extract local patterns or features from the input data. By learning appropriate filters through the training process, CNNs can automatically discover relevant features at different spatial locations in the input image. These features may include edges, textures, shapes, or more complex patterns, depending on the depth and architecture of the network.

in practice, this operation involves moving the filter across the image, and at each position, calculating the weighted sum of the filter’s elements and the corresponding input image’s elements.

(1)  

This weighted sum gives us the intensity value of the output pixel at position (𝑥, 𝑦)

Convolution in Practice

In image processing, convolution is widely used for various tasks such as edge detection, blurring, sharpening, and more. The convolution operation helps in modifying an image by applying different filters, each designed to perform specific tasks.

1- Sobel Kernels

Sobel kernels are used to detect edges in horizontal and vertical directions.

2- Laplacian kernels

3- Blurring Kernels

Filters like the Gaussian blur and Box Blur reduce image noise and detail by averaging pixel values.

Kernels Properties

The size and weights of kernels are crucial parameters in convolutional neural networks (CNNs) as they determine the patterns and features extracted from the input data. Here’s how the size and weights of kernels impact feature extraction:

1. Size of Kernels:
   • Smaller kernel sizes (e.g., 3×3) are common and allow the network to capture fine-grained details and local patterns in the input data.
   • Larger kernel sizes (e.g., 5×5 or 7×7) capture more global patterns and can help the network learn higher-level features.
   • The choice of kernel size depends on the complexity of the task and the size of the input data. For tasks where fine details are important, smaller kernels are preferred, while larger kernels may be more suitable for tasks requiring understanding of broader context.
2. Weights of Kernels:
   • The weights of kernels are learned during the training process of the CNN.
   • These weights determine the specific patterns or features that the kernel is sensitive to. For example, in an edge detection kernel, the weights might be tuned to detect horizontal, vertical, or diagonal edges.
   • The weights are adjusted through backpropagation based on the network’s loss function, allowing the network to learn the most relevant features for the task at hand.
   • Weight initialization techniques (e.g., Xavier initialization, He initialization) are used to set initial values for the kernel weights, helping in training convergence and preventing issues like vanishing or exploding gradients.

Kernels Receptive Field

The receptive field in the context of convolutional neural networks refers to the area of the input image that influences the activation of a particular feature map or neuron in the network. It helps in understanding how much context or spatial information a neuron can “see” from the input image.

There are two types of receptive fields:

Different Types of Convolutions

1- Standard Convolution:

This is the basic form of convolution where a filter/kernel slides over the input data, performing element-wise multiplication and summing to produce the output feature map. It’s used for tasks like image classification, object detection, and segmentation.

2- Dilated Convolution:

Dilated convolution introduces gaps or spaces between the kernel elements, effectively increasing the receptive field without increasing the number of parameters. It’s useful for capturing larger context in the input data while maintaining computational efficiency. Dilated convolutions are commonly used in tasks like semantic segmentation and image generation.

3- Depth-wise Separable Convolution:

This type of convolution decomposes the standard convolution into two separate operations: depthwise convolution and pointwise convolution. Depthwise convolution applies a single filter per input channel, while pointwise convolution combines the output of depthwise convolution using 1×1 filters. Depthwise separable convolution reduces the computational cost and number of parameters while preserving model performance. It’s commonly used in mobile and resource-constrained applications.

4- Transpose Convolution (Deconvolution):

Transpose convolution, also known as deconvolution, is used for upsampling or generating higher-resolution feature maps from low-resolution input. It’s commonly used in tasks like image super-resolution, semantic segmentation, and generative modeling (e.g., image generation with autoencoders or generative adversarial networks).

Building Blocks of CNNs

The architecture of a Convolutional Neural Network (CNN) consists of a sequence of layers that transform the input image into an output prediction. The typical layers in a CNN include convolutional layers, pooling layers, activation functions, fully connected layers, and sometimes normalization layers. The combination of these layers allows CNNs to automatically and adaptively learn spatial hierarchies of features from the input data.

A typical CNN architecture might follow this pattern:

1. Input Layer: Accepts the raw input image data.
2. Convolutional Layers: Apply filters to detect features in the input image.
3. Activation Layers: Introduce non-linearity to the model.
4. Pooling Layers: Reduce the spatial dimensions of the feature maps.
5. Fully Connected Layers: Integrate high-level reasoning and produce the final output.
6. Output Layer: Generates the final predictions, such as class probabilities in classification tasks.

Role of Convolutional Layers

Convolutional layers are essential for detecting and learning patterns in the input data. Each convolutional layer applies a set of filters to the input data, producing feature maps that highlight specific features such as edges, textures, and shapes. These layers are responsible for extracting local features and building up to more complex structures as the network goes deeper.

• Filters/Kernels: Learnable weights that slide over the input to detect various features.
• Receptive Field: The region of the input image that a particular filter looks at. As the network depth increases, the receptive field grows, allowing the network to capture more complex patterns.

Pooling Layers (Max Pooling, Average Pooling)

Pooling layers reduce the spatial dimensions of the feature maps, thereby decreasing the number of parameters and computational load while retaining important information. This operation helps make the model invariant to small translations in the input image.

• Max Pooling: Selects the maximum value from each patch of the feature map. This operation preserves the most prominent features and introduces some form of translation invariance.

Activation Functions (ReLU, Sigmoid, etc.)

Activation functions introduce non-linearity into the network, enabling it to learn complex patterns. Without non-linearity, the network would behave like a linear model, regardless of its depth. You can read more about Activation functions in one of our recent articles here.

Reference

• [1] Sobel Operator
• [2] deeplearningbook.org
• [3] 498_FA2019_lecture07
• [4] CS231n Convolutional Neural Networks for Visual Recognition
• [5] What are Convolutional Neural Networks? | IBM
• [6] Convolutional neural network
• [7] Micromachines | Free Full-Text | Hybrid Dilated Convolution with Multi-Scale Residual Fusion Network for Hyperspectral Image Classification