Language Models (Classical)

Topics

Notes

Linked

Seq2Seq Language Models (Classical) Language Model Sampling
Beam Search

Greedy decoding misses high-probability sequences. Maintain top-k partial hypotheses at each step for better approximate search.

Reinforcement Learning Language Models (Classical)
RLHF - Reinforcement Learning with Human Feedback

Specifying a reward function for complex tasks like language generation is intractable. Learn a reward model from human preferences and optimize with RL.

BERT Inverse Reinforcement Learning Policy Gradient PPO - Proximal Policy Optimization TRPO - Trust-Region Policy Optimization Transformers
Natural Language Processing Deep Learning Language Models (Classical)
BERT

Language models only use left context, missing bidirectional understanding. Mask random tokens and train to predict them using full context.

Transformers