Pratik Bhavsar

Jun 9

NLP has grown into a field of wide applications ranging from chatbots to news generation. I started my journey into NLP with standard courses but a majority of my practical learnings came from personal blogs of researchers. This metablog is a collection of my favourite blogs and courses I go through every once in a while to refresh myself.

As they say “Finding the right learning material is half the work.”

Basics

• Intro to NLP
• Text Mining
• Coursera sequence models
• CS224n — Natural Language Processing with Deep Learning — 2019
• Spacy crash course
• Google crash course

Embeddings

Word2Vec

• Jalammar
• Lilian
• McCormik — Part 1
• McCormik — Part 1

GLOVE

ELMO

• ELMO by mlexplained
• ELMO by Edward Ma

BERT

• Attention by Denny
• Attention by Lilian
• Transformer by Jalammar
• Transformer by Harvard NLP
• BERT by Jalammar
• BERT by mlexplained
• Łukasz Kaiser’s talk

Embeddings summary by Lilian Weng

Transfer learning

Slides from NAACL

Ruder on Multi-task learning

ULMFiT (Universal Language Model For Fine Tuning)

• RNN theory
• Seq2Seq theory
• Neural machine translation(NMT)
• AWD-LSTM
• ULMFiT

NN concepts

• Improving NNs
• Why momentum works?
• Learning-rate-tuning
• Cyclical Learning Rates

Normalisation

• Why batch normalisation works
• Weight and layer normalisation
• Overview of normalisation techniques

4 Sequence Encoding Blocks You Must Know Besides RNN/LSTM

Conversational AI

• Neural Approaches review paper

Denny’s

• DL for chatbots Part 1
• DL for chatbots Part 2

Han’s

• Question answering systems Part 1
• Question answering systems Part 2
• Neural Information Retrieval

Rasa

• Bot levels
• Level 3 bot
• 13 rules for chatbot design
• Rasa NLU in Depth: Part 1 — Intent Classification
• Rasa NLU in Depth: Part 2 — Entity Recognition
• Rasa NLU in Depth: Part 3 — Hyperparameter Tuning

Huggingface

• A Transfer Learning Approach for Neural Network Based Conversational Agents

ML pipeline

• Full Stack Deep Learning
• Rules of ML by Google
• Deploying ML models

Airbnb

• Zipline — Data management
• Bighead — ML infra

Uber

• Michelangelo
• Big Data Infrastructure

Databricks Mlflow

Libraries

• Custom training in Spacy

Random

• Topic modelling by Nanonets
• How to detect fake text?
• An Embarrassingly Simple Approach for Transfer Learning
• Machine Learning Fairness

Papers and code

State of the art NLP papers

NLPprogress