An Alien Mind
Points and comments are a snapshot, not live.
OpenAI warns AI progress toward recursive self-improvement threatens safety if alignment doesn't keep pace.
OpenAI researcher describes 2023 experiments that convinced them reasoning models could scale toward superhuman intelligence. The essay argues AI is 'grown more than designed,' making its behavior hard to understand or predict. It distinguishes goal alignment (following instructions) from value alignment (holding human principles). Key monitoring tool, chain-of-thought observation, is weakening as models get smarter and more complex. The author advocates combining faster AI development for defense with voluntary slowdowns and government-mandated safety standards, warning that progress in alignment may not outpace general intelligence gains.
What commenters are saying
Most comments dismiss the essay as marketing fluff from an OpenAI desperate for funding and regulatory capture. A top commenter calls it 'absolute trash marketing drivel,' pointing to Kurzweil's missed predictions as evidence of hype. Another argues slowing down is impossible in a zero-trust game, while others counter that racing is exactly the dangerous mindset the essay critiques. Some praise a new model, Astra, as genuinely more capable and concise than predecessors, though skepticism about OpenAI's motives dominates the thread.