ProjectBlog: Neural Networks: what are they, types and applications

Blog: Neural Networks: what are they, types and applications

Illustration of a brain

I remember the first time I listened to the word “neural network”. I was in the second year of my bachelor degree, while two professors were talking about the final project of one of their students.

By then I had no idea what that meant, but without a doubt, the name sounded extremely well. “Neural network” is one of those words which can be said to someone who doesn’t know anything about the topic and immediately rises you to the next level of intelligence. As what business people do when they try to sell something.

However, the concept behind the word was much simpler than I thought. In fact, the complete term would be “Artificial Neural Network”, differentiating from those which are indeed not artificial. I suppose you can imagine what I’m talking about, right? Yes, the human brain contains networks of neurons and neural networks are said to be based on the same idea.

Relationship between the human brain and neural networks

The biological process that allows humans to understand and think is far beyond our current understanding. However, we do have an abstract idea about how it works.

Roughly, the human brain is composed of nerve cells called neurons. The functionality of a neuron is to receive an electrical impulse as input, process it and respond with another impulse at the output.

The magic happens when millions of neurons are connected to each other, creating a network of neurons. But, what is the origin of the electrical impulses? To understand it, we need to know how the human body acquires information:

Reception of external impulses: we catch what occurs around us using our senses. For example, we can see what’s in front of us with our eyes or we can check if the ground is hot by touching it with our hands.

  • Processing of received information: our body transforms these external stimuli into electrical signals which are propagated through our brain.
  • Conclusion: we are able to understand and make conclusions, which makes us intelligent systems.

With the arrival of the computational era, computers have been able to outperform humans in many tasks, especially in those which are based on a set of rules to follow (for example, to make complicated numerical computations). However, when the set of rules are not evident, the human has always been better than the machine. Some examples of these tasks would include:

  • Recognize the content of a recorded audio
  • Identify objects in images
  • Classify texts according to its intentions (for example, offensive texts)

The center of the matter is, if both the human and the machine are able to see the same content, why the machine is not able to succeed in these tasks? As I previously said, that’s because we don’t even know how our own brain works, and it’s impossible for us to create an abstract set of rules which work for complex scenarios.

The idea behind artificial neural networks (from now own, just neural networks) is none other but to try to simulate the behavior of the human brain:

  1. We have an input signal (for example, an image), which mimics our senses.
  2. We introduce this signal in a network of neurons which processes it, as our own brain.
  3. We obtain a final result and check if it’s correct. If not, we try to check why we failed and learn from our own mistakes.

This image represents the structure of a neural network with 4 layers and {4, 3, 3, 1} neurons in each layer:

Scheme of a neural network

The basic unit of a neural network: the neuron

Everything has been nice and easy so far, but now it’s time to face reality. I have already said the, in the human brain, neurons receive and propagate electrical impulses. In our case, as we can’t propagate electrical signals, we will have to conform with numbers.

Representation of a single neuron

To understand the global behavior of a complex neural network we need the understand the maths behind a single neuron. Let’s suppose the following scheme:

As we can see, in this simple example, we have a neuron with nx inputs, which we can represent as a vector x and one output y. Besides, we can see that each neuron is characterized by a vector w, a constant b, and a function a(·). We can compute the output of it as:


And a(·) is a non-linear function which is called activation. The output y can be used now as one of the inputs of another neuron, creating a network as bigger as we want.

Notice that, if we don’t use this non-linearity, the concatenation of several neurons can be replaced by a single neuron, which would make the use of a network totally useless.

Note: real implementations use matrices W instead of vectores w to represent several neurons in each layer, making a much more efficient computation.

Cost function and backpropagation

Once our input has propagated through the network and we have the output of the last layer, we need a way to evaluate if the result is correct or not. If know what the result is supposed to be, we can define a distance between the prediction and the expected value. This function is called cost function, and our goal is it to be as closer to 0 as possible.

Let’s suppose that the input parameters x of our network are the size of a house, it’s antiquity and its number of bathrooms. As our business is Real State and we have been doing it for 10 years, we do have a database with past selling prices so we would want to use it to predict how much can we ask for the houses we are trying to sell now.

We will use this labeled data to construct our regressor using a neural network. We will use the 3 things we know about the houses as the input and the price as the objective output. If y is the predicted price given by the network and z is the real price, we can measure the error committed as:

If the error is close to 0, then our network is doing great! If it is not, what can we do? The solution is simple and uses simple calculus: we can take the derivative of the error with respect to each parameter and push it to the closest minimum (hopefully, the global minimum).

We will get in details about backpropagation in further posts. Why don’t you subscribe in the meantime? :)

Types of neural networks

Luckily or not, there are a lot of cases where the previously showed architecture doesn’t fit the nature of the input signal.

We are using all the data we have as input, but what if the input is an image of 1000 x 1000 pixels? We would have 1.000.000 inputs! We could deal with it, but it would not be efficient and probably there are much cheaper and better solutions.

Besides, we are inserting the data as if all the inputs don’t have any type of hierarchical or temporal structure. What if we want to predict the content of a given audio recording? Again, we will have to face a huge problem of dimensionality at the input. Besides, there is a lot of redundancy in speech signals, why don’t we try to use it?

To solve these types of problems, there are two types of neural architectures which have proved to work much better:

  • Convolutional neural networks: they apply the convolution operation, using windows of a certain size and displacing through the input.
  • Recurrent neural networks: recurrent units try to use temporal information of the inputs by propagating not only horizontally, but also vertically. They are extremely used for speech and audio tasks, natural language processing and time-series analysis.

It’s worth mentioning that, in practice, modern neural network architectures use a mix of these and new modules that appear every year.

Applications of neural networks

If neural networks have had such a success has been because of the variety of fields that they can be applied to. Now, we can deal with data that we don’t fully understand and obtain results which outperform humans in most of the cases.

In the field of image processing, the best-known applications include:

  • Facial detection and recognition
  • Tagging of objects in images
  • Medical image segmentation
  • Anomaly detection in satellite images and manufacturing processes

In speech and audio:

  • Voice recognition (speech-to-text)
  • Voice synthesis (text-to-speech)
  • Musical generation
  • Speaker verification and identification

In natural language processing:

  • Machine translation
  • Sentimental analysis
  • Text summarization
  • News generation

The list can grow as much as I want, but I think that you already see the idea. Every year, neural networks prove to work well for new applications and the demand for qualified professionals is growing fast.

Will you join the AI era?

Source: Artificial Intelligence on Medium

Leave a Reply

Your email address will not be published. Required fields are marked *

Back To Top

Display your work in a bold & confident manner. Sometimes it’s easy for your creativity to stand out from the crowd.