Robonaissance
Subscribe
Sign in
Home
Notes
🧠 Intelligence Foundations
🧭 Agentic Intelligence
🤖 Embodied Intelligence
🌍 World Models
Archive
About
The Journey of RL
Latest
Top
Discussions
The Journey of RL, Part 12: Learning What to Optimize
Six open problems define the next decade. None of them is “better policy optimization.”
Jul 8
•
Hugo
5
2
3
The Journey of RL, Part 11: The Verifiable Turn
The escape from the Goodhart collapse was to find problems where reward could be checked rather than learned, and to discover, in passing, that…
Jul 7
•
Hugo
3
5
The Journey of RL, Part 10: When Scale Replaced Signal
Faced with the failure of reward modeling, the field’s loudest response was lateral: stop trying to fix reward, fix the data and the compute instead.
Jul 4
•
Hugo
3
2
The Journey of RL, Part 9: The Goodhart Collapse
A learnable proxy for a value is not the value. The crisis was not in the algorithms. It was in the framing.
Jul 2
•
Hugo
4
4
The Journey of RL, Part 8: The Human Turn
RLHF was not the addition of human feedback to reinforcement learning. It was the moment reward stopped being given and started being learned.
Jun 30
•
Hugo
6
4
The Journey of RL, Part 7: The First Cracks
Three different attempts to rescue reward. Three different admissions that reward was the problem.
Jun 26
•
Hugo
5
7
The Journey of RL, Part 6: The Brain in the Loop
Reinforcement learning and neuroscience read each other for forty years. The conversation reveals that reward in biology was never the simple scalar RL…
Jun 25
•
Hugo
7
2
The Journey of RL, Part 5: The Internal Model
Self-play and world models converge on a single idea. When an agent has a model of the thing it is optimizing against, the reward signal can come from…
Jun 23
•
Hugo
6
1
4
The Journey of RL, Part 4: The Exploration Question
Before an agent can optimize anything, it has to figure out what is worth measuring. Exploration is the part of reinforcement learning that admits the…
Jun 19
•
Hugo
6
5
The Journey of RL, Part 3: Policy or Value
The Great Schism. Two routes to the same destination, until they were not.
Jun 17
•
Hugo
10
2
6
The Journey of RL, Part 2: The Value Hypothesis
Q-Learning and Its Discontents. The compression that made an entire field possible, and the trap it concealed.
Jun 9
•
Hugo
11
5
The Journey of RL, Part 1: Before the Equation
How machines learned what to optimize, beginning in 1898 with a stopwatch and the borrowed assumption that has shaped reinforcement learning ever since.
May 26
•
Hugo
19
3
14
This site requires JavaScript to run correctly. Please
turn on JavaScript
or unblock scripts
