Viewpoint
Models can recognize evaluations yet still reward-hack
Models can recognize evaluations yet still reward-hack
Ho says interpretability research indicates that models can recognize when they are being evaluated and still pursue behaviors that exploit the reward mechanism rather than completing tasks as intended.
- Speaker
- Eric Ho
- Topic
- Reward Hacking
- Source timestamp
- 20:28
More from this interview
- AI leaders should communicate their positions directly
- Total AI monopoly is unlikely but supplier ambition matters
- Superintelligence definitions determine governance and concentration concerns
- AI currently gives cyber attackers a significant advantage
- Security systems still require meaningful human oversight
- Organizations should choose their own automation boundaries
- Grok Bot could be an iPhone moment
- Persistent cloud sessions are central to future agents
- Pervasive AI could make privacy increasingly difficult to preserve
- AI development creates intense demand for human-generated data
- Models should selectively learn from training data
- Technology companies need greater empathy and disclosure
- Public backlash reflects perceived lack of tech-industry empathy
- Silicon Valley should acknowledge AI's social consequences
- AI should accelerate people rather than simply replace them
- AI alignment remains difficult but technically solvable
- Models may need a shared moral infrastructure layer
- Multi-agent systems require alignment at the collective level
- AI's future must be actively shaped rather than observed
- Current AI decisions could shape humanity's long-term future
- Silicon Valley has drifted from empowerment toward monetization