评论
打印
And the name of this god was Moloch. Bit of a bummer, as stories go.
这尊神的名字叫摩洛克。故事讲下去就有点扫兴了。
But you can see why it's an apt metaphor, because sometimes we get so lost in winning the game right in front of us, we lose sight of the bigger picture and sacrifice too much in our pursuit of victory.
但是你会明白为什么这是一个恰当的比喻，因为有时我们太过沉迷于赢得眼前的比赛，而在追求胜利时忽视了大局，牺牲了太多。
So just like these guys were sacrificing their children for power, those influencers are sacrificing their happiness for likes.
就像这些人为了权力牺牲孩子一样，那些网红为了点赞牺牲了快乐。
Those news editors are sacrificing their integrity for clicks, and polluters are sacrificing the biosphere for profit.
那些新闻编辑为了点击量牺牲了自己的诚信，污染者则为了利润牺牲了生物圈。
In all these examples, the short-term incentives of the games themselves are pushing, they're tempting their players to sacrifice more and more of their future, trapping them in a death spiral where they all lose in the end.
在这些例子中，游戏本身的短期激励措施是在鼓动，引诱玩家牺牲越来越多的未来，陷入死亡螺旋中，最终全盘皆输。
That's Moloch's trap. The mechanism of unhealthy competition.
那是摩洛克的陷阱。不良竞争的机制。
And the same is now happening in the AI industry.
AI 行业中也正在上演。
We're all aware of the race that's heating up between companies right now over who can score the most compute, who can get the biggest funding round or get the top talent.
我们都知道公司间的竞争正热火朝天，竞争最高算力、最高融资、最优人才。
Well, as more and more companies enter this race, the greater the pressure for everyone to go as fast as possible and sacrifice other important stuff like safety testing.
随着越来越多公司加入竞争，每个人都面临着更大的压力，迫使他们尽可能快地动起来，牺牲安全测试等重要事项。
This has all the hallmarks of a Moloch trap.
这就是摩洛克陷阱的所有特征。
Because, like, imagine you're a CEO who, you know, in your heart of hearts, believes that your team is the best to be able to safely build extremely powerful AI.
因为想象一下你是一位 CEO，你内心深处相信你的团队是能够安全地打造极其强大的 AI 的最佳人选。
Well, if you go too slowly, then you run the risk of other, much less cautious teams getting there first and deploying their systems before you can.
如果你动作太慢，那你就会面临其他不那么谨慎的团队先行一步，在你之前部署他们的系统。
So that in turn pushes you to be more reckless yourself.
因此，这反过来又会促使你自己更冲动行事。
And given how many different experts and researchers, both within these companies but also completely independent ones, have been warning us about the extreme risks of rushed AI, this approach is absolutely mad.
有多少专家和研究人员，包括这些公司的职员，还有完全独立的人士，一直在警告我们警惕仓促发展的 AI 带来的极端风险，这种做法简直太疯狂了。
Plus, almost all AI companies are beholden to satisfying their investors, a short-term incentive which, over time, will inevitably start to conflict with any benevolent mission.
此外，几乎所有 AI 公司都必须满足投资者的要求，这种短期激励措施随着时间的推移，将不可避免地开始与任何“圣母”使命发生冲突。
And this wouldn't be a big deal if this was really just toasters we're talking about here.
如果我们在这里谈论的真的只是面包机，那也没什么大不了的。
But AI, and especially AGI, is set to be a bigger paradigm shift than the agricultural or industrial revolutions.
但是，AI，尤其是 AGI（通用人工智能），必将带来比农业或工业革命更大的范式转变。
A moment in time so pivotal, it's deserving of reverence and reflection, not something to be reduced to a corporate rat race of who can score the most daily active users.
这是个关键时刻，需要敬意和反思，不能简化成企业间你死我活的竞争，比拼谁能获取更多的日活跃用户。
I'm not saying I know what the right trade-off between acceleration and safety is, but I do know that we'll never find out what that right trade-off is if we let Moloch dictate it for us.
我并不是说我知道加速发展和安全之间权衡取舍的正确答案，但我确实知道，如果我们受制于摩洛克，我们将永远找不到正确的权衡取舍。
