Blog: Episode I : What is Natural Language Processing?
Hey everyone! This article is one of many articles that will help you to understand natural language processing (NLP), the various techniques and algorithms used to analyse text data. I am going to start off with a very brief introduction to NLP and its various applications.
Firstly, NLP is NOT machine learning (ML). NLP is one branch of Artificial Intelligence (AI) and ML is another branch. More accurately, NLP lies at the intersection of AI and linguistics. Now that we have some terminology out of the way lets go ahead and discover NLP.
I’m sure most of you have at least heard of Google translate or if you have used it, you know how it works.
You give it a sentence in one language it gives you the same sentence in any language of your choice. This is the short and sweet definition of NLP. It is the ability of a machine to understand “Natural Language”, and what I mean by “Natural Language” is the way we communicate everyday.
Wouldn’t it be cool if a machine could understand your words and emotions and reply just like any other human (Eg: Jarvis from the Iron man movie)?
Why is it difficult?
However, natural language processing is difficult because language has many rules. Making a machine understand these rules is not easy. For example, sarcasm can be very hard to detect as someone can say “Great!” both sarcastically and genuinely.
To understand human language, a machine needs to understand both the syntax and semantics of a language along with other aspects(phonetics, phonology, morphology etc.)
So, where do we use NLP?
Information retrieval: Obtaining relevant documents and web pages for a given search query. Eg: Search engines
Machine translation: Translating one language to another
Question Answering: Imagine you have a very large document. A Question Answering system will analyse the corpus (the text), extract useful information from it and will be ready to answer any questions you may have about the content. This is very difficult among the many NLP problems and a lot of research is being carried out to address this task.
Sentiment analysis: Given a review about a particular movie, product or book, the machine will categorise whether the review is good, bad or neutral. This can be very helpful for companies as they can analyse their customers’ needs more effectively.
Text summarization: Summarise a given corpus of text.
These are few of the applications of NLP.
I hope now you have an idea of what NLP is, why is it difficult and what are its applications. I know this is a very brief introduction, and the main question is yet to be answered: How does it work?
Hold your horses people, Rome wasn’t built in a day.
Soon (in the later articles), you will discover many algorithms and techniques used to analyse text data. As we go along you will get to see NLP in action (I mean in code) and I will also provide links to other helpful resources at the appropriate time.
The next time while typing an email, if you get suggestions on your next words or talk to your digital assistant (Alexa, Siri, Google assistant etc.), remember that that’s Natural Language Processing for you.
Are you excited about NLP now?
Thanks for being patient and coming to the end. See you in the next Episode!
Repository to track the progress in Natural Language Processing (NLP), including the datasets and the current…nlpprogress.com
Natural Language Processing, or NLP for short, is broadly defined as the automatic manipulation of natural language…machinelearningmastery.com
We’ve trained a large-scale unsupervised language model which generates coherent paragraphs of text, achieves…openai.com