Robonaissance
Subscribe
Sign in
Home
Foundations
Thinking
World Models
Agents
Robotics
Machine Minds
Archive
Leaderboard
About
The Journey of RL
Latest
Top
Discussions
The Journey of RL
Reward sounds like an axiom. It was always an argument. A puzzle box and a verified reward turn out to be the same argument. Twelve parts, one argument.
Sep 21
•
Hugo
9
4
The Journey of RL, Part 12: Learning What to Optimize
Six open problems define the next decade. None of them is “better policy optimization.”
Jul 8
•
Hugo
7
2
3
The Journey of RL, Part 11: The Verifiable Turn
The escape from the Goodhart collapse was to find problems where reward could be checked rather than learned, and to discover, in passing, that…
Jul 7
•
Hugo
4
5
The Journey of RL, Part 10: When Scale Replaced Signal
Faced with the failure of reward modeling, the field’s loudest response was lateral: stop trying to fix reward, fix the data and the compute instead.
Jul 4
•
Hugo
4
2
The Journey of RL, Part 9: The Goodhart Collapse
A learnable proxy for a value is not the value. The crisis was not in the algorithms. It was in the framing.
Jul 2
•
Hugo
5
4
The Journey of RL, Part 8: The Human Turn
RLHF was not the addition of human feedback to reinforcement learning. It was the moment reward stopped being given and started being learned.
Jun 30
•
Hugo
7
4
The Journey of RL, Part 7: The First Cracks
Three different attempts to rescue reward. Three different admissions that reward was the problem.
Jun 26
•
Hugo
6
7
The Journey of RL, Part 6: The Brain in the Loop
Reinforcement learning and neuroscience read each other for forty years. The conversation reveals that reward in biology was never the simple scalar RL…
Jun 25
•
Hugo
7
2
The Journey of RL, Part 5: The Internal Model
Self-play and world models converge on a single idea. When an agent has a model of the thing it is optimizing against, the reward signal can come from…
Jun 23
•
Hugo
6
1
4
The Journey of RL, Part 4: The Exploration Question
Before an agent can optimize anything, it has to figure out what is worth measuring. Exploration is the part of reinforcement learning that admits the…
Jun 19
•
Hugo
6
5
The Journey of RL, Part 3: Policy or Value
The Great Schism. Two routes to the same destination, until they were not.
Jun 17
•
Hugo
10
2
6
The Journey of RL, Part 2: The Value Hypothesis
Q-Learning and Its Discontents. The compression that made an entire field possible, and the trap it concealed.
Jun 9
•
Hugo
11
5
This site requires JavaScript to run correctly. Please
turn on JavaScript
or unblock scripts
