Chapter 2

Deep learning is currently at the heart of some of the most powerful AI systems. We are going to give you a sense of how deep learning has come to where it is today. Keep scrolling!

Scroll

To understand how deep learning has progressed, we may first look at its inspiration, the neuron. That will give us a glimpse of how AI extracts valuable sensory experiences from this chaotic world and solidifies them into algorithms that are “smart”.

A neuron is the building block of the human brain. Neurons are nerve cells that take in some stimuli from other neurons. When enough stimuli are received, these neurons fire signals (more stimuli) to other connected neurons.

Inspired by neurons, computer scientists created **perceptrons**
that take in many stimuli, or inputs, and output one value. We will examine
perceptrons in more details in the next chapter.

It turns out that perceptrons are far from enough to learn complicated concepts, but
some simple concepts, such as the logical **AND**,
can be learned by a perceptron. The model can compute the logical AND of its
inputs. This computation is visualized in the animation as drawing a line to divide
the possible outcomes of the logical AND into two groups, each consisting of only
one type of circle.

More complex concepts, such as **XOR**,
cannot be learned by a simple perceptron. This is due to the fact that it is not
linearly separable - we cannot find a straight line to divide the filled circles
from the open ones.

To solve this problem, computer scientists turn to more complicated “simulations” of
the brain. A **feedforward
neural network** is a network of perceptrons organized into layers.
These simple neural networks can represent every rule, from classifying images to
predicting income.

Simple feedforward neural networks are very nice, but they have their own problems. Smaller networks do not have the capacity to take on complex problems, but deeper networks need a lot more computation power. This time, computer scientists looked at higher level human learning mechanisms for inspirations.

One such inspiration is that we humans conceptualize fundamental building blocks of
things, and combine them to create more complex concepts. We recognize a dog as
having a nose, two ears, and four legs. This behavior inspired computer scientists
to create **Convolutional
Neural Networks** (CNN), which are constructed from reusable groups of
units that learn to recognize visual patterns like basic shapes.

The power of deep learning made some traditional machine-learning techniques go out of fashion. With enough data and computation power, deep learning is able to achieve a level of performance traditional algorithms could never hope for. With recent developments in computer hardware, and the abundance of data, we are finally able to apply deep learning to the real world.

Now that you have a general idea of the inspirations behind some of the most important concepts in deep learning, you are prepared to get into the details of how things work. In the next chapter, we will discuss perceptrons, an important building block of deep learning.

The XOR Problem in Neural Networks

Deep Learning 101 - History and Background

A Short History Of Deep Learning

**Perceptron**: An algorithm inspired by neurons. A
single-layer neural network that performs binary classification on
input data.

**Feedforward Neural Networks**: The first and simplest
type of artificial neural networks. It is composed of many perceptrons
arranged into
layers and can perform more complex tasks than perceptrons.

**Convolutional Neural; Networks**: A type of artificial
neural networks that is mainly used to analyze and classify images.