Blog: A Gentle Introduction to Machine Learning
Any sufficiently advanced technology is indistinguishable from magic
Welcome! This article is my first effort to dive a little deep into an advanced topic like Machine Learning. There are two reasons for me writing this article. Firstly, I found this subject super interesting and if you are curious about Machine Learning but don’t know where to start, then this article is for you. Secondly, this article is a way for me to synthesize my learning and share my knowledge with Machine Learning passionates.
I expect you have some basic knowledge of “Probability Theory” in order to follow along. Since Machine Learning is a very broad topic and I want you to enjoy reading, thus this article will mainly focus on:
- Demystify what Machine Learning is?
- The Learning Approach
- Supervised and Unsupervised Learning
- Overview of Regression, Classification & Clustering
- Two practical examples of data sets.
- Free resources to learn more
So, what is Machine Learning & Why do we need it?
“Any sufficiently advanced technology is indistinguishable from magic.”
— Arthur C. Clarke
We know that humans learn from past experiences, and machines follow instructions given by humans.
What if humans can train the machines from the past data and do what humans can do?
Well, that’s machine learning, let’s dig deep into it.
The world is filled with data, a lot of data — pictures, music, text, videos, spreadsheet. Machine Learning is the science and branch of Artificial Intelligence, in which computers are trained from data to perform a particular task, rather than by explicitly programming them.
Traditional Software Development VS Machine Learning
In the traditional software development approach, input and the algorithm is known, and you write a function to produce an output.
- Input data
- Design an algorithm by applying logic to it
- Output is produced
However, in the Machine Learning approach, you know the input and desired output, but you don’t know the algorithm that gives the output.
- Give a set of input data
- Give a set of desired output data
- Through neural networks (to be covered in another article) you get the algorithm which predicts the desired output.
Some applications of Machine Learning
Companies use Machine Learning algorithms to optimize the results for their products.
- When you go on Netflix, how do you figure out which movie you want to watch? Netflix makes a recommendation using Machine Learning.
- When you use a credit card to make a purchase, there is fraud protection on it, that’s machine learning.
- Other applications of Machine Learning are speech recognition, language translation, self-driving cars, and many others.
How does learning happen?
In most applications of machine learning: the focus is not so much on building models (“understanding the underlying process”) but more on predicting.
Same way as humans learn from experiences, the same is with machine learning, you give them lots of examples (data), and they start to figure out what’s going on. We, humans, do inductive reasoning as opposed to deductive reasoning. Let’s review these two concepts as they are very important in Machine Learning.
Induction (Specific → General) is generalizing conclusion from specific examples based on our limited experiences, here the conclusion might not be 100% accurate all the time.
- We observe the sun rises every day.
- So we predict that the sun is going to rise tomorrow as well.
- Hence, we predict the Sun will rise tomorrow as well.
Deduction (General → Specific) is the process of drawing a specific conclusion from general statements. Here the conclusion is a fact.
- All men are mortal.
- William is a Man.
- Therefore, William is Mortal.
It is important to remember that any system that learns has “Inductive Bias” or prior knowledge, and Machine Learning aims to automate the process of “Inductive inference”. A notable example of this diversity of Machine Learning is the separation of Supervised Learning and Unsupervised Learning, I won’t talk about Reinforcement Learning here.
In Supervised Learning, we are given a labeled data set and already know what our correct output should look like, taking into consideration that there is some relationship between the input and output. A self-driving car is an example of Supervised learning. Supervised problems are categorized into “Regression” and “Classification” problems.
In a “Regression” problem we are trying to predict something that has a continuous result. Example, given a picture of a person, we have to predict their age on the basis of the given picture.
In the “Classification” problem we are trying to predict results which have a discrete output. Example, given a patient with a tumor, we have to predict whether the tumor is malignant or benign.
5 Machine Learning STEPS
In order to design Machine Learning algorithm, here are the main 5 steps to follow:
- Input, also known as features: x
- Output, also known labels: y
- Target Function: f: x → y (unknown to us)
- Collect Data (x1,y1), (x2,y2),….,(xn, yn)
- Hypothesis: h: x → y
Let’s see two examples of supervised learning with both classification and regression to get a deeper understanding.
Let’s say we work at the admission office at the university and our job is to accept or reject a student. In order to make a decision, we need some inputs from the students, right? Let’s take: their grades and final year exam mark.
We are trying to predict based on the past data collected, whether Student C will be admitted or not? In order to visualize we can plot it on the graph.
In the image above we have first separated with a line the data from good and bad. This line is going to be our model. The blue dots represent the data points students who got accepted and red dots students who got rejected based on their test mark and grades. Looking at the diagram we can observe that students above the line get accepted while the student under the line gets rejected. You might be thinking that there are a couple of red dots above the line and a couple of blue dots below the line, yes you are right, the model makes a couple of mistakes, that is why it is probability and not 100% certain. So it is safe to predict, if a point is above the line then the student is accepted. Student C with Grade(6) and Test Mark (7) is above the line, thus we can assume with some confidence that Student C is most likely to get accepted. This is a classification problem because the output is discrete (Accepted, Rejected).
Example2: Regression (Little Advanced)
Let’s take an example of House Price Prediction. Your friend owns a house and says it is 750 sq/feet and he is hoping to sell his house and want to know for how much he can sell it. You know Machine Learning, and hence you can help him to predict the price using your Machine Learning knowledge and skills. Let’s see how to do it.
Our goal is, given a training set, to learn a function h: X → Y so that h(x) is a “good” predictor for the corresponding value of y. For historical reasons, this function h is called a hypothesis. The hypothesis takes the input and gives out an estimated value, so it maps from x → y.
We can use a “linear “ line (pink line in the image below) to fit into the data set, and based on that it looks like maybe he can sell it for $150,000. However, there might be a better learning algorithm than linear. Instead of fitting the straight line into the data, we can maybe try “quadratic” to this function (blue line in the image below). Then, by making a prediction on it, it looks like maybe he can sell it for $200,000. Now it comes a question of choosing which one we should use? There is no fair in picking, whichever gives our friend the best price. This is a regression problem because the answer can be continuous.
In Unsupervised Learning, we have little or no idea what our results should look like. Hence, the purpose is to find hidden structure in data. The core idea to understand Unsupervised learning is “Clustering”.
Clustering: Take a collection of 50 different people, and find a way to automatically group these people into groups. Maybe you want to group them into male and female, hence you can decide how would you do that. You can perhaps divide them by facial hair, skin tone, clothes, etc. In the end, you can say thee are 27 male and 23 females.
- Feature: The input(s) to our model
- Examples: An input/output pair used for training
- Labels: The output of the model
- Layer: A collection of nodes connected together within a neural network
- Model: The representation of your neural network
- Any system that learns has “Inductive Bias” or “prior knowledge”.
- Machine Learning aims at automating the process of “inductive inference”
- We need to have an idea of what we are looking for (inductive bias)
- Since Machine Learning is a form of inductive reference, we can never be fully confident about the outcomes.
- In most applications of Machine Learning, the focus is not so much on building models, but more on predicting.
This article is just a small taste about the Machine Learning field, there is much more in it and there are tons of online resources available to enhance your knowledge. Below I’ve provided 5 resources that I personally found very helpful.
Some Free Useful Resources to Learn More:
- Google’s AI Adventures
- Udacity’s Machine Learning Course
- Machine Learning Made Fun & Interesting
- Caltech’s Machine Learning Course
- Andrew Ng’s Coursera Course
This article was my first attempt to dive deep into a complex topic like Machine Learning. I will write more articles to cover the math behind Machine Learning. I hope I did a decent job in explaining and you found this article insightful.
Stay Curious & Keep Learning :)