Blog: Neural Networks aren’t a big pile of dirt
This is similar to what people might envision at the back of their minds when they first think about the idea of neural networks. It’s nothing compared to the little pile of dirt which is like the rest of the things we learn in life. It might consist of learnings from our day job, reading books, school homework, etc. but neural networks are pretty different. The big pile of dirt that’s three times the size of you is intimidating in the same way that neural networks can come across as being pretty complicated and therefore, intimidating. And that’s a fair statement. I mean look at this image of what a complicated neural network looks like.
Now don’t worry, we won’t get into anything this complicated but it can be easy to just ignore this complex image of random dots and 6 thousand different lines going in 7 thousand directions. But the truth is that you’re missing out because something that’s much less complex than this with the same concept can do a whole bunch that humans can’t do! Let me give you the low down. Neural networks aren’t a big pile of dirt.
What are neural networks?
Neural networks are a specific form of AI but really not that specific at all. At the heart of machine learning (algorithms based on patterns and inferences rather than statistics) lies neural networks and within that is deep learning. Going down the path of deep learning lies a whole bunch of other types of AI like…
In 1959, Arthur Samuel, one of the pioneers of machine learning, defined machine learning as a “field of study that gives computers the ability to learn without being explicitly programmed.”
Now, if you think about that, it seems really cool. Predicting things without explicitly telling a computer to learn!!!
There’s no way machine learning or neural networks can be compared to a pile of dirt1
The name neural networks sound a lot like neurons. And it’s for a reason. Neural networks work in a very similar way that neurons do.
Electrical impulses enter the neuron which does something with it and decides if its a nervous impulse or not through the axon. In the same way, the electrical impulse that enters the neuron is the same as data that is inputted into a neural network. Through this, calculations are made which is almost like the impulse going through the neuron and in the end, the output is created in the form of 1 or 0 in the way that the neuron decides if an impulse is a nervous impulse.
How do neural networks work?
This is what a simple neural network looks like. Take a good look because I will refer to it often.
The purpose of neural networks is to receive a set of inputs, perform a bunch of complex calculations then create an output to solve a problem.
As shown above, the inputs or data given are the two images which allow the algorithm to recognize the difference between the cat and the dog (this is called training). When the image of the dog is given to the input layer, this gets transported to the hidden layers which perform a bunch of calculations. The output layer is the last layer which leads to the prediction of the image being a dog or a cat. Take a look at the process above again and you’ll begin to understand it a bit more.
Weights and biases
Apart from the three layers: the input, hidden and output layers there are parts called weights and biases that are crucial parts of the neural network.
Neural networks can be thought of as spinning classifiers together in a web where each node (purple circle) has its own classifier and contains some kind of data (cat and dog). As a result of each classification from each node, it’s passed on to the next layer through edges (purple lines) and in the end, the output is created to solve some sort of problem.
Each node has the same classifier so if you give it the same input it will create the same output. Going back to our cat/dog example, this means that if you give the algorithm the same picture of the dog as shown in the diagram it will produce an output of classifying the image as a dog every time. However, they don’t create the same output values in the end since each set of inputs is modified by a unique set of weights and biases.
Imagine weights and biases to look like this:
- Each edge= unique weight (2 and 1)
- Each node= unique bias (-18)
When calculating the output of a node, the inputs are multiplied by weights and a bias value is added to the result. To make the training of the algorithm as accurate as possible, the weights and biases are tweaked so that the correct output is created.
Think of this like throwing a baseball. If you use the same technique and positioning of your body to throw a baseball it will probably end up going along a very similar direction. But, at the same time, it won’t always have the exact speed every time you throw the ball with this same body positioning. In the same way, technique and body positioning is like the classifier in the way that the same output will be created every time you give it the same input. But, the weights and biases make the output values different in the way that the speed of the ball will be a bit different each time you throw the baseball even if it’s with the same technique.
To go just a bit deeper as to how a neural network may create a prediction, another key factor in all neural networks is a perceptron.
Perceptrons were one of the first computational units of AI and are the simplest form of a neuron. With all the information given, they’re put into nodes, weights, and biases.
x represents the data and w represents the weights and b as the bias (don’t worry about the 1 and the bias)
The first input contains all the data which transforms it into a linear equation shown in the second function in which the data is represented and the actual answer to the problem we are solving is solved (by being graphed) but isn’t shown yet.
In the third function, the step function is applied which translates the answer to the problem as 0 or 1. As a result, yes or no is outputted where:
- yes = 1
- no = 0
Take a look at the perceptron again and you might understand it a bit better.
Now that you can understand what neural networks are, you can probably see that it’s not a piece of dirt! The algorithm containing layers, nodes, edges, weights, and biases, in the form of a perceptron creates this incredible system of predicting things for us! We used a simple example of predicting the image of a dog or a cat, and this is only the very beginning of what neural networks can do. However, more important algorithms can be created to have a more meaningful purpose like when dealing with the classification of large data sets. Maybe classifying one type of cell from another!