Latent Reasoning

Topics

Notes

Linked

Latent Reasoning Transformers
Deep Supervision with Recursion

Train a model with recursion without large costs of BPTT

Hierarchical Reasoning Model (HRM) Tiny Reasoning Model (TRM) Transformers
Transformers Latent Reasoning
Tiny Reasoning Model (TRM)

Small transformers lack multi-step reasoning ability. Use latent recurrence in hidden states instead of explicit chain-of-thought tokens.

Transformers Deep Supervision with Recursion Hierarchical Reasoning Model (HRM) Multi-Network Training with Moving Average Target
Latent Reasoning Transformers
Hierarchical Reasoning Model (HRM)

Flat latent reasoning has limited depth. Add hierarchical structure to latent recurrence for deeper reasoning in transformers.

Expectation Maximization Backpropagation Through Time (BPTT) Deep Supervision with Recursion Transformers Temporal Difference Learning Tiny Reasoning Model (TRM)