Chapter 5
Feedforward neural networks combine multiple perceptrons to complete far more challenging tasks. This chapter presents the anatomy of this simplest type of artificial neural networks. Keep scrolling!
Scroll
To this point, we have only discussed networks with a single perceptron. These perceptron algorithms are not particularly powerful, as they cannot learn complex concepts. To improve them, we need to "wire" perceptrons together so that they can form a more complex model.
"Wiring" multiple perceptrons together give us a feedforward neural network. Now, instead of only having several inputs, we now have an input layer. The layers between this first layer and the last layer are called hidden layers since they are invisible to the inputs and outputs of the network.
The last layer is called the output layer. We say that a network has n layers when there are n layers that perform computations. That is, there are n-1 hidden layers and 1 output layer.
As an example of how a neural network can solve an everyday problem, let’s say that we want to predict tomorrow’s temperature range using a network that has already been trained.
The network that we are using is a simple two-layer feedforward neural network that takes in two inputs: the maximum and minimum temperature of today.
These two inputs in the input layer are then sent to 4 perceptrons in the hidden layer. Then, the outputs of the hidden layer are fed into the 2 outputs in the output layer, which generate the network’s guess of tomorrow’ maximum and minimum temperatures.
In this network, all of the nodes in the hidden layer have a ReLU function, while the last layer is a linear layer (meaning that it has no activation function). But why do we need activation functions when we don’t need to control the range of output here? And what exactly does ReLU do?
It turns out that, if we have consecutive linear layers, the network will behave exactly the same as a network with a single linear layer. To make the extra layers count, we need to introduce non-linearity. One of the most popular choices to break linearity is ReLU.
With these activation functions, we can wire up enough layers so that the network can learn complex tasks, such as recognizing handwritten digits as showcased in Chapter 0.
Now that we have understood the basics of how feedforward neural networks work, we can get our hands a little dirty and look at how we can train a model to classify handwritten digits.
In this chapter, we have examined how the components of a feedforward neural network work together to solve hard problems. In the next chapter, we break down the feedforward neural network used in Chapter 0 to interactively recognize digits.
But what *is* a Neural Network?
Deep Learning: Feedforward Neural Network
Neural Networks - Architecture - Stanford CS
What are neural networks? - SciToons
Hidden Layer: The middle layers of a feedforward
neural network that performs intermediate computations.
Node: Used as a synonym for a perceptron that is part
of a more complex network.
ReLU: Rectified Linear Unit. A type of activation
functions commonly used to break linearity.