Legacy NLP & Stat Resources

• • Natural Language Processing (Almost) from Scratch
  • Foundations of statistical natural language processing
  • [Springer]A Bayesian/Information Theoretic Model of Learning to Learn via Multiple Task Sampling
  • Math needed for Deep Learning

Outline

DL NLP Papers

• Year 2020
  
  • Deep Learning 2019
  • Prospects on Deep Learning in 2020

• Year 2013
  
  • On the difficulty of training Recurrent Neural Networks

• Year 2015
  
  • Teaching Machines to Read and Comprehend
  • Effective Approaches to Attention-based Neural Machine Translation
  • Conditional Random Fields as Recurrent Neural Networks

• Year 2016
  
  • Neural Architectures for Named Entity Recognition
  • Exploring the Limits of Language Modeling

• Year 2017
  
  • Attention Is All You Need

• Year 2018
  
  • [SOTA] BERT
  • Phrase-Based & Neural Unsupervised Machine Translation
  • [DeepMind] Conditional Neural Processes
  • [DeepMind] Fast Parametric Learning with Activation Memorization
  • [DeepMind] Automatic Goal Generation for Reinforcement Learning Agents
  • [DeepMind] Successor Features for Transfer in Reinforcement Learning
  • [DeepMind] Measuring abstract reasoning in neural networks

• Year 2019
  
  • [SOTA] BERT
  • TRANSFORMER-XL: ATTENTIVE LANGUAGE MODELS BEYOND A FIXED-LENGTH CONTEXT
  • Assessing BERT’s Syntactic Abilities
  • Analysis Methods in Neural Language Processing: A Survey
  • Identifying and Correcting Label Bias in Machine Learning

April 4 2019

• • Neural Machine Translation of Rare Words with Subword Units
  • Subword Regularization: Improving Neural Network Translation Models with Multiple Subword Candidates

Burning Hot Ones

• • LOOKING FOR ELMO’S FRIENDS: SENTENCE-LEVEL PRETRAINING BEYOND LANGUAGE MODELING
  GAN Mania
  • Keeping up with the GANs

My research focus : Language Modelling

• • [SOTA] BERT
  • ULM FiT
  • ELMo
  • AWD LSTM
  • Multi-Task Learning
  • Attention is all you need
  • Bi-DAF
  • GLoMo
  • QANet
  • QRNN
  • Char-CNN
  • Morphological Embeddings
  • Incorporating Latent Meanings of Morphological Compositions to Enhance Word Embeddings
  • DiSAN
  • Character-Level Language Modeling with Deeper Self-Attention
  • A Framework for Learning Predictive Structures from Multiple Tasks and Unlabeled Data
  • LOOKING FOR ELMO’S FRIENDS: SENTENCE-LEVEL PRETRAINING BEYOND LANGUAGE MODELING
  • TRANSFORMER-XL: ATTENTIVE LANGUAGE MODELS BEYOND A FIXED-LENGTH CONTEXT
  • On the State of the Art of Evaluation in Neural Language Models
  • Assessing BERT’s Syntactic Abilities

Great Blogs

• • A3C RL Policy Gradient Medium Article
  • BERT, ELMo and Co.
  • BERT Google Blog
  • ULMFiT Analitics Vidya Blog
  • Reddit - Link to BERT resources
  • BERT explained
  • Allen Ins. tutorials on Elmo
  • Rudder's [Amazing Guy for NLP] Intro to Multi-task Learning
  • CMU ML Blog on Parallel HPO
  • Progress in NLP
  • Optimization algorithms : Overview
  • Best Practices of DL in NLP
  • Selecting data for Transfer Learning
  • Approximating Softmax in Word Embeddings
  • [GAN] PAPER READING ROAD MAP

Great Githubs and Data Sources

• • Datasets by Percy Liang
  • SQUAD
  • SNLI Dataset
  • WMT
  • WordNet
  • Pre-trained Word Embeddings : Glove Stanford , Spacy Large , Google wrd2vec, Fast Text
  • Stanford NLP Reading Group
  • Awesome deep Learning Papers
  • Deep Learning Papers Git 2
  • NLP Deep Learning Papers Git 1
  • GAN Zoo
  • [Google] Attention and Augmented Recurrent Neural Networks
  • [Google] Sequence Modelling with CTC
  • [SEAS] The Annotated Transformer
  • Generalized Language Models

Things to remember

• • Phrase Structure Rules
  • Dependency Grammar