C4W1L07 One Layer of a Convolutional Net

TL;DR

One convolutional layer convolves an input volume with multiple filters, adds a separate bias to each resulting feature map, applies a non-linearity, and stacks the maps into an output volume. In the example, a 6×6×3 input and two filters produce a 4×4×2 output; ten 3×3×3 filters require 280 parameters. Read on for the dimension rules, parameter calculation, and layer notation.

Transcript

you're now ready to see how to go one layer of a convolution on your network let's go through the example you've seen in the previous video how to take a 3d volume and convolve it with say two different filters in order to get in this example two different 4x4 outputs so let's say convolving with the first filter gives this first 4x4 output and con... Read More

Key Insights

• 🔇 CNN layers involve convolving input volumes with filters to generate output volumes.
• 🖐️ Biases and non-linearities play crucial roles in shaping the output of CNN layers.
• 🔇 The number of filters in a CNN layer affects the size and complexity of the output volume.
• ✳️ Parameters in CNN layers remain fixed regardless of the input image size, reducing overfitting risks.

Questions & Answers

Q: How does one layer of a convolutional neural network work?

The layer convolves its input volume with each filter to produce feature maps. It adds a separate real-number bias to every element of each map, applies a non-linearity, and stacks the resulting maps to form the next layer’s activation volume.

Q: How does a convolutional layer relate to forward propagation in a standard neural network?

The filters play a role similar to the weights, and the convolution operation acts as the linear computation. Adding the biases produces the counterpart of Z, while applying the non-linearity produces the next activation.

Q: What output does a 6×6×3 input produce with two filters in this example?

Each filter produces a 4×4 feature map after convolution, bias addition, and application of the non-linearity. Stacking the two maps gives a 4×4×2 output activation volume.

Q: How does the number of filters determine the output volume’s depth?

Each filter generates one feature map, so the number of filters becomes the output volume’s depth. Two filters produce a 4×4×2 output in the example, while ten filters would produce a 4×4×10 output.

Q: How is bias applied in a convolutional layer?

Each filter has its own bias, represented by a real number. That same number is added to all 16 elements of the corresponding 4×4 feature map through broadcasting before the non-linearity is applied.

Q: How many parameters are in a layer with ten 3×3×3 filters?

Each 3×3×3 filter contains 27 learned values and has one bias, giving 28 parameters per filter. With ten filters, the layer therefore has 28 × 10 = 280 parameters.

Q: Why can convolutional layers use relatively few parameters on large images?

The same learned filters can detect features anywhere in an image, so their parameter count does not grow with the image dimensions. In the example, ten filters still require only 280 parameters whether the input is 1000×1000 or 5000×5000, which makes the network less prone to overfitting.

Q: What notation describes a convolutional layer’s filter size, padding, and stride?

For layer L, the filter size is denoted by F with a layer-L superscript, the padding amount by P with that superscript, and the stride by S with that superscript. A valid convolution means no padding, while a same convolution uses padding so the output retains the input’s height and width.

Summary & Key Takeaways

• Demonstrates convolutions with filters & biases in CNN layers.

• Explains how convolutional layers transform input to output volumes.

• Illustrates the computation process from one layer to the next in CNNs.