Transcript
Meta 的人工智能战略与组织挑战
(00:00:00)
The episode introduces Dylan Patel, an expert in AI, and begins by discussing Meta's AI endeavors, specifically focusing on the performance and delays of their LLMs, including LLaMA. Patel notes that Behemoth might not be released due to training problems, while Maverick is decent but surpassed by other models. One model was a rushed response to DeepSeek and suffered from sparsity issues, with tokens not routing to certain experts, indicating training flaws. The conversation shifts to the organizational challenges within Meta, suggesting that despite having talent and compute, the lack of a strong technical leader to choose the best ideas hinders progress. Patel emphasizes the importance of "taste" in AI research, highlighting that even with brilliant researchers, poor decision-making can lead to wasted effort on bad research paths.
本集介绍了人工智能专家 Dylan Patel，并首先讨论了 Meta 在人工智能方面的努力，特别关注其大型语言模型（LLM）的性能和延迟问题，包括 LLaMA。Patel 指出，Behemoth 模型可能因训练问题而不会发布，而 Maverick 虽然尚可，但已被其他模型超越。其中一个模型是为了仓促应对 DeepSeek 而推出的，存在稀疏性问题，即令牌（token）未能路由到某些专家（expert），这表明训练存在缺陷。对话随后转向 Meta 内部的组织挑战，指出尽管拥有人才和计算资源，但缺乏一位强有力的技术领导者来抉择最佳创意，阻碍了进展。Patel 强调了在人工智能研究中“品味”的重要性，并指出即使拥有杰出的研究人员，糟糕的决策也可能导致在错误的研究路径上浪费精力。
(00:00) Matthew Berman:
Super intelligence reaching it first who you picking and why open AI He's the guy the chip industry reads before making a move meet Dylan Patel He's a quick thinker with a depth and breadth of knowledge. That is unrivaled in AI, you know for one scale AI is like a.
要率先达到超级智能，你会选谁，为什么？OpenAI。他就是芯片行业在采取行动前会参考的人物。来认识一下 Dylan Patel。他思维敏捷，知识渊博，在人工智能领域无人能及。你知道，对于某种规模的人工智能来说，它就像一个……
(00:18) Dylan Patel:
It's kind of cooked.
它基本没戏了。
(00:19) Matthew Berman:
And today, Dylan's answering the tough questions. What went wrong with GPT4.5? In general, it's not that useful and it's too slow.
今天，Dylan 将回答一些棘手的问题。GPT-4.5 出了什么问题？总的来说，它没那么有用，而且太慢了。
(00:28) Dylan Patel:
You zoom back, right? It's like, if you believe super intelligence is the only thing that matters, then you need to chase it. Otherwise, you're a loser. It's not the money. It's more the power.
你把眼光放长远，对吧？就好比，如果你相信超级智能是唯一重要的事情，那么你就必须去追逐它。否则，你就是个失败者。重要的不是钱，而是权力。
(00:35) Matthew Berman:
What do you think is going on at Apple?
你觉得苹果公司内部发生了什么？
(00:37) Dylan Patel:
They hate NVIDIA, maybe for reasonable reasons. Models are just like pansies about like giving me data.
他们讨厌英伟达，也许有合理的理由。模型在提供数据这方面扭扭捏捏的。
(00:42) Matthew Berman:
You're using O3 day to day, even though there's, you know, it takes so much time to actually get your response back. The model I go to the most is either 50% of white-collar jobs could disappear.
你日常在使用 O3，尽管你知道，它要花很长时间才能返回响应。我最常用的模型是……要么就是 50% 的白领工作可能会消失。
(00:52) Dylan Patel:
Generally, people work less than ever before. The average amount of hours worked 50 years ago was way higher. And then eventually, there just won't be humans in the loop, right?
总的来说，人们现在的工作时间比以往任何时候都少。50 年前的平均工作时长要高得多。然后最终，就不会再有人类参与其中了，对吧？
(01:00) Matthew Berman:
I don't believe that.
我不相信。
(01:01) Dylan Patel:
It's an art form to some extent. What is worth researching and what's not? GPUs that ended up breaking, it was called BumpGate. It was a very interesting thing. You don't or you do? No, I don't. Okay, so this is a very fun story, right?
在某种程度上，这是一种艺术形式。什么值得研究，什么不值得？那些最终坏掉的 GPU，被称为“颠簸门”（BumpGate）。这是件非常有趣的事。你听说过还是没听说过？不，我没有。好的，那这会是个很有趣的故事，对吧？
(01:15) Matthew Berman:
All right, Dylan, thank you so much for joining me today. I'm really excited to talk to you. I've seen you do a number of talks. I've seen you do a number of interviews. We're going to talk about a whole bunch of things. The first thing I want to talk about is Meta. Let's start with LLM4. I know it's been a little while in the AI world since that kind of released, but there was a ton of anticipation. It was good, not great. It wasn't kind of world changing at the moment. And then they delayed Behemoth. What do you think is going on there?
好的，Dylan，非常感谢你今天能来。我很高兴能和你交谈。我看过你的几次演讲和采访。我们今天要聊很多事情。首先我想谈谈 Meta。让我们从 LLM4 开始吧。我知道在人工智能领域，它发布已经有一段时间了，但当时人们的期望非常高。它还不错，但算不上卓越。当时它并没有改变世界。然后他们又推迟了 Behemoth 的发布。你觉得这是怎么回事？
(01:41) Dylan Patel:
Yeah, so I mean, It's funny, there's like three different models and they're all quite different. So Behemoth got delayed. I actually think they might not ever release it. I'm not sure. There's a lot of problems with it, the way they trained it, some of the decisions they made don't pan out. And then there's Maverick and Scout, right? And so one of those models is actually decent. It's pretty good. It wasn't the best on release, but it was comparable to the best Chinese model on release. But then, you know, Alibaba came out with a new model. DeepSeek came out with a new model, so I was like, okay, it's worse. The other one was objectively just bad.
是的，我的意思是，这很有趣，他们大概有三个不同的模型，而且都非常不一样。所以 Behemoth 被推迟了。我其实觉得他们可能永远不会发布它了。我不确定。它有很多问题，比如他们的训练方式，他们做的一些决策结果并不理想。然后还有 Maverick 和 Scout，对吧？其中一个模型其实还不错。它挺好的。发布时不是最好的，但可以和当时最好的中国模型相媲美。但是后来，你知道，阿里巴巴推出了新模型，DeepSeek 也推出了新模型，所以它就显得更差了。另一个模型客观上讲就是很糟糕。
(02:15)
I know for a fact they trained it as a response to DeepSeek, trying to use more of the elements of DeepSeek's architecture, but they didn't do it properly. It was just a rush job and it really messed up because they went really hard on the sparsity on the MOE. But funnily enough, if you actually look at the model, it oftentimes won't even route tokens to certain experts, so it was like a waste of training. Basically, in between every layer, the router can route to whatever expert it wants to, and it learns which expert to route to, and each expert learns its own independent things, and it's really not something observable by people, but what you can see is tokens,
我确切地知道，他们训练这个模型是为了应对 DeepSeek，试图更多地采用 DeepSeek 的架构元素，但他们没有做好。这只是一个仓促赶工的项目，结果搞砸了，因为他们在混合专家模型（MoE）的稀疏性上做得太过火了。但有趣的是，如果你真的去观察这个模型，它常常甚至不会将令牌（token）路由到某些专家（expert），所以这就像是浪费了训练资源。基本上，在每一层之间，路由器（router）可以把它路由到它想路由到的任何专家那里，它会学习该路由到哪个专家，每个专家也学习自己独立的东西，这其实不是人们能直接观察到的，但你能看到的是令牌，
(02:53)
which experts do they route to, when they go through the model, And it's just like some of them just didn't get routed to. So it's like you have a bunch of empty experts that are not doing stuff. So there's clearly something wrong with training.
当它们通过模型时，被路由到了哪些专家那里。结果就是，有些专家根本没有被路由到。所以就像你有一堆空闲的专家什么事也没做。所以训练过程明显出了问题。
(03:05) Matthew Berman:
Is it like an expertise thing internally? I mean, they have to have some of the best people in the world. And we're going to get to some of their hiring efforts as of late. But like, what? What? Like, why? Why haven't they been able to really do it?
这是内部专业能力的问题吗？我的意思是，他们肯定拥有世界上最顶尖的一批人才。我们稍后会谈到他们最近的一些招聘动作。但是，到底是什么？为什么？为什么他们就是做不成呢？
(03:16) Dylan Patel:
I think it's a combination and confluence of things, right? Like, yes, they have tons of talent. They have tons of compute. But the organization of people is always like the most challenging thing. Which ideas are actually the best? Who is the technical leader choosing the best ideas, right? It's like if you have a bunch of great researchers, that's awesome. But then if you put like product managers on top of them, and then there's no technical lead who's like evaluating what to choose, then you have a lot of problems, right? Open AI, right? Yeah, Sam is a great leader and he gets all the resources, but the technical leader is Greg Brockman, right? And Greg Brockman is choosing a lot of stuff.
我认为这是多种因素共同作用的结果，对吧？是的，他们有大量的人才，有大量的算力。但人员的组织管理始终是最具挑战性的事情。哪些想法才是真正最好的？由哪位技术领导者来选择最好的想法，对吧？就好比你有一群杰出的研究人员，这很棒。但如果你在他们上面安排了产品经理，然后又没有一个技术负责人来评估该选择什么，那你就会遇到很多问题，对吧？就拿 OpenAI 来说，是的，Sam 是一位伟大的领导者，他能争取到所有资源，但技术领导者是 Greg Brockman，对吧？很多事情都是 Greg Brockman 在做选择。
(03:51)
And there's a lot of other folks, right? Like Mark Chan and others who are like the technical leaders who are like really deciding the like, you know, technically, which route do we go down? Because a researcher is going to have their research, they're going to think their research is the best. Who's evaluating everyone's research and then deciding that idea is great. Let's use that one. That one sucks. Let's not use that one. It's just really difficult. So when you end up with researchers not having a leader who is technical and can choose, and really, you know, choose the right things, you end up with, great, we did have all the right ideas.
还有很多其他人，对吧？比如 Mark Chan 和其他人，他们是技术领导者，真正决定着，你知道，从技术上讲，我们该走哪条路？因为一个研究员会有自己的研究，他们会认为自己的研究是最好的。谁来评估所有人的研究，然后决定这个想法很棒，我们用这个；那个想法很烂，我们不用那个。这真的非常困难。所以，当你的研究人员没有一个懂技术、能做选择、能真正选对方向的领导者时，你最终会发现，很好，我们确实有过所有正确的想法。
(04:23)
But part of AI research is that you have all the wrong ideas, too. And you learn from them, and you have the right ideas, and you choose them, right? And now, what if what happens if you're choosing of them is really bad, and actually, you choose some wrong ideas? And then you go up the, you know, the branch of sort of research, right? Like you've chosen this bad idea. This is something we're going to do. Let's go further down. And then now you're like, oh, branching off of this bad idea. There's a lot more research because, you know, it's like, well, we're not going to go back and undo the decision we made. Right. So everyone's like, oh, we made that decision.
但人工智能研究的一部分就是你也会有很多错误的想法。你从中学习，然后你有了正确的想法，并选择它们，对吧？但现在，如果你做出的选择非常糟糕，实际上选择了一些错误的想法，会发生什么呢？然后你就沿着那个，你知道，研究的分支路径走下去了，对吧？比如你选了一个坏主意。这是我们要做的事。让我们继续深入。然后现在你就会发现，哦，从这个坏主意又分支出了更多的研究，因为，你知道，大家会觉得，好吧，我们总不能回头去推翻已经做出的决定。对。所以每个人都说，哦，我们已经做了那个决定。
(04:54)
OK, let's see what's researchable from here. And so you end up with like this like potentiality of like great researchers wasting their time on bad paths. Right. And there's sort of this like The thing that researchers talk about, which is taste, right, which is very funny, right? You think like these are like nerds who won like the International Math Olympiad and like that's their like, you know, claim to fame. But when they were like a teenager, then they got a job at OpenAI or whatever at 19 or Meta or whatever. But there's actually a lot of taste involved, right? It's an art form to some extent. What is worth researching and what's not?
好吧，让我们看看从这里开始有什么可以研究的。于是你最终就可能让杰出的研究人员在错误的道路上浪费时间。对。研究人员之间会谈论一种东西，叫做“品味”（taste），对吧，这很有趣，对吧？你以为他们就是些书呆子，赢过国际数学奥林匹克竞赛，这就是他们，你知道，引以为傲的成就。但当他们还是青少年的时候，可能 19 岁就在 OpenAI 或 Meta 之类的地方找到了工作。但实际上这里面涉及很多“品味”，对吧？在某种程度上，这是一种艺术。什么值得研究，什么不值得？
(05:28)
And it's an art form of choosing what's the best because you're making all these ideas down here on the scale here and then all of a sudden you're like, yeah, let's now, you know, great. Those experiments were all done with like 100 GPUs. Awesome. Now let's make a run with 100,000 GPUs with that idea. It's like, well, things don't just translate perfectly. So there's a lot of taste and intuition here. It's not that they don't have good researchers. It's that like who's choosing the taste, right, is difficult, right?
选择什么是最好的，本身就是一种艺术。因为你在这里小规模地验证了所有这些想法，然后突然之间你说，是的，我们现在，你知道，太棒了。那些实验都是用 100 个 GPU 完成的。太好了。现在让我们用 10 万个 GPU 来运行那个想法。但问题是，事情并不会完美地等比例放大。所以这里需要大量的品味和直觉。并不是说他们没有好的研究人员，而是说由谁来把握这个“品味”，这很难，对吧？
(05:53)
You know, it's like, you don't care about movie critic reviews, you care about Rotten Tomatoes, you know, audience score, perhaps, right?
你知道，这就像，你不在乎影评人的评论，你可能更在乎烂番茄的观众评分，对吧？
(05:58)
And it's like, it's like, who's the critic that you're listening to, though, right? And that's, that's, it's challenging to, you know, even if you have great people to actually have good stuff come out, because of organizational issues, because the right people aren't at the right spot and decision makers. And maybe the wrong person gets to like be political and have their idea and research path put into the model, when it's not necessarily a good idea.
但问题是，你到底在听哪个评论家的意见呢，对吧？这就是挑战所在，你知道，即使你拥有优秀的人才，也很难真正产出好的东西，就是因为组织问题，因为对的人没有在对的位置上成为决策者。也许错误的人通过一些政治手腕，把他们的想法和研究路径放进了模型里，而那未必是个好主意。
Meta 对 ScaleAI 的收购以及向超级智能的转变
(00:06:23)
The discussion centers on Meta's acquisition of ScaleAI, primarily for Alexander Wang and his team, as ScaleAI's core business is declining. Patel suggests that Meta's interest lies in Wang's leadership for their super intelligence efforts. This acquisition signifies a strategic shift for Zuckerberg, who previously downplayed AGI but now prioritizes super intelligence. Patel explains that the term AGI has become meaningless, prompting the rebranding to "super intelligence," a move pioneered by Ilya Sutskever. The conversation touches on Zuckerberg's failed attempts to acquire SSI, Thinking Machines, and Perplexity. Patel emphasizes that individuals joining Meta are motivated by the power to influence AI development within a massive company, allowing them to implement their AI visions across billions of users.
讨论的焦点是 Meta 对 ScaleAI 的收购，主要是为了 Alexander Wang 和他的团队，因为 ScaleAI 的核心业务正在衰退。Patel 认为，Meta 的兴趣在于 Wang 在其超级智能项目中的领导能力。这次收购标志着扎克伯格的战略转变，他之前对 AGI 持轻视态度，但现在却将超级智能放在首位。Patel 解释说，AGI 这个词已经变得毫无意义，因此促使了向“超级智能”的品牌重塑，这一举动由 Ilya Sutskever 开创。对话还提到了扎克伯格收购 SSI、Thinking Machines 和 Perplexity 的失败尝试。Patel 强调，加入 Meta 的人是受到在大公司内影响人工智能发展的权力的激励，这让他们能够将自己的人工智能愿景实施到数十亿用户身上。
(06:19) Matthew Berman:
Okay, well, let's continue down the path of who is making decisions. Obviously, Zuck, last week, there was a lot of news about him giving $100 million offers. I mean, Sam Altman literally said it. They acquired ScaleAI, seemingly for Alexander Wang and his team. He's in founder mode. What does the ScaleAI acquisition actually give Meta? First, let's start there.
好的，那我们继续沿着谁在做决策这个话题聊。很明显，扎克伯格，上周有很多关于他开出 1 亿美元 offer 的新闻。我的意思是，Sam Altman 亲口说的。他们收购了 ScaleAI，似乎是为了 Alexander Wang 和他的团队。他现在处于创始人模式。ScaleAI 的收购到底能给 Meta 带来什么？我们先从这里开始吧。
(06:41) Dylan Patel:
Yes, I think, you know, for one, ScaleAI is like.
是的，我认为，你知道，首先，ScaleAI 就像...
(06:45) Matthew Berman:
It's kind of cooked right now as a company because everybody's canceling their contracts.
作为一家公司，它现在差不多完蛋了，因为大家都在取消和他们的合同。
(06:51) Dylan Patel:
Google's backing out. I think they're going to spend on the order I've heard like $250 million this year with them and they're backing out. Obviously, they've spent a lot of money and there's stuff they can't back out of, but that's going to go down a lot. OpenAI allegedly cut the external slack connection, so there's no slack between scale and OpenAI anymore.
谷歌正在退出。我听说他们今年原本要和他们合作一个价值约 2.5 亿美元的订单，但现在他们要退出了。很明显，他们已经花了很多钱，有些东西是退不掉的，但这笔金额会大幅下降。据说 OpenAI 切断了外部的 Slack 连接，所以 Scale 和 OpenAI 之间不再有 Slack 通道了。
(07:10) Matthew Berman:
It's the ultimate breakup between companies.
这是公司之间的终极分手。
(07:12) Dylan Patel:
Yeah, so obviously these companies are like, I don't want Meta to know what I'm doing with my data, because that's one of the unique aspects of models. It's like, what do you want with your custom data? So clearly scale is not, you know, Meta didn't buy scale for scale, right? They bought Scale for the purposes of having Alex and his few best colleagues. There's a few other folks at Scale who are really awesome as well, and they bought them to bring them over, right? Now the question is sort of like, is the data that Scale has good? Is knowing all the paths of sort of data labeling that all these other companies were doing good? Sure. But more importantly, it's like, We want to get, you know,
是的，所以很明显这些公司会想，我不想让 Meta 知道我用我的数据在做什么，因为这是模型的一个独特之处。这就像是，你想用你的自定义数据做什么？所以很明显，Scale 不是……你知道，Meta 收购 Scale 并不是为了 Scale 本身，对吧？他们收购 Scale 是为了得到 Alex 和他最优秀的几个同事。Scale 还有其他一些非常出色的人才，他们收购公司就是为了把这些人挖过来，对吧？现在的问题有点像是，Scale 拥有的数据好吗？了解所有这些其他公司进行数据标注的各种路径，这有好处吗？当然有。但更重要的是，他们想的是，我们想要得到，你知道的……
(07:56)
someone to help us lead this super intelligence effort, right? And Alex is a, you know, he's the same age as me. He's 28 or 29-ish. Yeah, I think he's around that age. Dependently successful in every way, shape or form, right? People can hate on him if they want, but he's obviously very successful, especially when he convinces Mark Zuckerberg, who's not an irrational person. Very smart to buy his company, right? Like, you know, it's like, and their company was doing, you know, nearly a billion of revenue and is like, let's chase super intelligence, right? Which is very different, right?
一个能帮助我们领导这个超级智能项目的人，对吧？Alex 呢，你知道，他和我同龄。他大概 28 或 29 岁左右。是的，我想他大概是那个年纪。在任何方面都取得了成功，对吧？人们可以随心所欲地讨厌他，但他显然非常成功，尤其是当他说服了马克·扎克伯格——一个并非不理性、非常聪明的人——去收购他的公司，对吧？就像，你知道，他们的公司当时有近十亿美元的收入，然后他却说，我们去追逐超级智能吧，对吧？这非常不同，对吧？
(08:25)
If you go look at Zuckerberg's interviews, even a handful of months ago, he wasn't chasing super intelligence, right? He was chasing like AI is good and great, but like AGI is not a thing that is going to happen soon. This is a big shift in strategy in that he's basically saying, yeah, yeah, super intelligence is all that matters. We're on the path there, I believe now. What can I do to catch up because I'm behind?
如果你去看扎克伯格的采访，即使是几个月前，他也没有在追逐超级智能，对吧？他当时追求的是，人工智能很好很强大，但像 AGI 这种东西短期内不会发生。这是一个重大的战略转变，他基本上是在说，是的，是的，超级智能才是一切。我现在相信，我们正走在这条路上。我落后了，我能做些什么来追赶？
(08:47) Matthew Berman:
It seems like the narrative throughout all of these major companies is now super intelligence, even when it was AGI just a month ago. Why the transition, by the way?
似乎所有这些大公司的叙事现在都变成了“超级智能”，尽管就在一个月前还是“AGI”。顺便问一下，为什么会有这种转变？
(08:57) Dylan Patel:
The word AGI has no meaning anymore.
AGI 这个词已经没有意义了。
(08:59) Matthew Berman:
It's amorphous, yeah.
是的，它很模糊。
(09:01) Dylan Patel:
You can look at an anthropic researcher in the face and be like, what does AGI mean? And they literally think it just means an automated software developer. And it's like, that's not artificial general intelligence, but that's what they think, right? And like a lot of researchers across the ecosystem.
你可以当面问一个 Anthropic 的研究员，AGI 是什么意思？他们真的认为那只是指一个自动化的软件开发者。这就好比，那不是通用人工智能，但他们就是这么想的，对吧？整个生态系统中的很多研究员都这样。
Ilya saw this, you know, Ilya saw everything first, obviously, Ilya Sestover. And he started his company, Safe Super Intelligence, right, SSI. And I think that started the rebranding and now like, you know, many months later, it's like almost a year now, a year later, I think it's like nine months to a year later, everyone's like, oh, super intelligence is a thing.
Ilya 看到了这一点，你知道，Ilya 总能先看到一切，很明显，Ilya Sutskever。然后他创办了他的公司，“安全超级智能”，对吧，SSI。我认为这开启了品牌重塑，然后现在，你知道，很多个月过去了，差不多一年了，一年后，我想大概是九个月到一年后，所有人都觉得，哦，超级智能是个正经事了。
(09:35)
So another research direction that Ilya got first, whether it was like, Whether it was like pre-training scaling or like, you know, the original like vision networks, right? Pre-training scaling, reasoning, right? All these things that he sort of had the idea at least, if not first, among the first and worked on it a lot. You know, sort of Ilya's got this one too, which is the rebranding. So maybe he's got marketing too.
所以这是 Ilya 又一个抢先的研究方向，无论是像预训练扩展，还是像，你知道的，最初的视觉网络，对吧？预训练扩展、推理，对吧？所有这些东西，他就算不是第一个，也是最早提出想法并为此付出了大量努力的人之一。你知道，Ilya 在品牌重塑这件事上也抢了先机。所以也许他也懂市场营销。
(09:59) Matthew Berman:
Yeah, well, Zuck, at least rumored, tried to acquire SSI and was rebuffed by Ilya, right? I wanted to also ask you about Daniel Gross and Nat Freeman. I think it's rumors, maybe confirmed at this point, but it seems like Zuck is trying to hire them as well. What did those two folks give Zuck?
是的，嗯，扎克伯格，至少有传言说，他试图收购 SSI，但被 Ilya 拒绝了，对吧？我还想问问你关于 Daniel Gross 和 Nat Freeman 的事。我想这还是传闻，也许现在已经证实了，但看起来扎克伯格也想聘请他们。这两个人能给扎克伯格带来什么？
(10:17) Dylan Patel:
Zuck tried to buy SSI. He also tried to buy thinking machines, like these are rumors. He also tried to buy Perplexity. These are all in some of the media, right? He tried to buy all of these companies, but specifically like some of the rumors that have been floated around is basically that like Mark tried to buy SSI. Ilya obviously said no because he is like committed to super intelligence and straight shotting it, right? Not like worried on products and he's probably not even that money focused, right? He's mostly focused on like building it, right? A true believer in all respects and regards, right? So obviously he probably was like no.
扎克伯格试图收购 SSI。他还试图收购 Thinking Machines，这些都是传闻。他还试图收购 Perplexity。这些都在一些媒体报道里，对吧？他试图收购所有这些公司，但具体来说，一些流传的谣言基本上是说马克试图收购 SSI。Ilya 显然拒绝了，因为他致力于超级智能并要直奔目标，对吧？他不像别人那样担心产品，而且他可能也不是那么看重钱，对吧？他主要专注于构建它，对吧？在各方面都是一个真正的信徒，对吧？所以很明显他可能会说不。
(10:54)
I don't know what the makeup of equity is there but Ilya's He's probably got strong enough votership and ownership to be like, no. And if the rumors are true about Daniel Gross, then like Daniel Gross probably was the one wanting. The acquisition, right? He's like, yeah, this is awesome, right? Yeah, another founder. And he comes from, you know, not an AI research background, although he is technical to a degree. But like, you know, it's like, he, you know, he had his venture fund with Nat and he then he founded SSI with Ilya and he probably wanted the acquisition. And then it's like, well, I was pushing for an acquisition and it didn't happen.
我不知道那里的股权结构是怎样的，但 Ilya 可能有足够强的投票权和所有权来说不。如果关于 Daniel Gross 的传闻是真的，那么 Daniel Gross 可能才是想要被收购的那个人，对吧？他会觉得，是的，这太棒了，对吧？是的，另一个创始人。而且他，你知道，并非来自人工智能研究背景，尽管他有一定技术水平。但是，你知道，他和 Nat 有自己的风险基金，然后他和 Ilya 一起创立了 SSI，他可能想要被收购。然后情况就变成了，嗯，我当时在推动收购，但没有成功。
(11:31)
And, you know, I'm just guessing like, you know, If he's going at all, I don't actually know if he's going at all. It would make sense that that's a chasm and split and he's going. I think generally when you look at really a lot of people who are very successful, it's not the money. It is the money always, but it's more the power. And if you ask like, you know, anyone going to Meta, a lot of them will obviously be going for money, but a lot of them are going because now they have control over the AI path for, you know, a trillion dollar plus company and They're right there talking to Zuck and they can convince one person who has full voting rights over the entire company.
而且，你知道，我只是猜测，你知道，如果他真的要走的话——我其实不知道他到底会不会走——那这就说得通了，这是一种分歧和分裂，所以他要离开。我认为总的来说，当你观察很多非常成功的人时，会发现驱动他们的不只是钱。钱固然重要，但更重要的是权力。如果你问，你知道，任何去 Meta 的人，很多人显然是为了钱，但也有很多人去是因为他们现在可以控制一个万亿美元以上公司的 AI 发展路径，他们可以直接和扎克伯格对话，并且他们可以去说服那个对整个公司拥有完全投票权的人。
(12:18)
There's a lot of power there and they can implement across billions of users whatever AI technology they want using the engine of Facebook, whether it be infrastructure or researchers or product to push whatever AI product you want. That would make a lot of sense to me for an Alex Wang or a Nat Friedman or a Daniel Gross who are They are much more product people, right? Like Nat doing GitHub Copilot, he's a product person, right? He's not an AI researcher, although he knows a lot about AI research, he's a product person, right? And same applies to sort of like Alex, like obviously he's very well versed with the research,
那里有巨大的权力，他们可以利用 Facebook 的引擎——无论是基础设施、研究人员还是产品——在数十亿用户中实施他们想要的任何 AI 技术，来推广他们想要的任何 AI 产品。对于像 Alex Wang、Nat Friedman 或 Daniel Gross 这样的人来说，这就非常有吸引力了，因为他们更偏向于产品型人才，对吧？比如 Nat 做了 GitHub Copilot，他就是一个产品人，对吧？他不是 AI 研究员，尽管他对 AI 研究了解很多，但他是一个产品人，对吧？同样的情况也适用于 Alex，很明显他对研究非常精通，
(12:56)
but his super skill set is product and people and like convincing people and organization probably not as much the research. That's sort of the angle there is like they've got like all this power to do a lot at Meta.
但他的超强技能在于产品、人事以及说服人和组织，可能研究方面相对没那么突出。所以角度就在于，他们在 Meta 拥有了巨大的权力去做很多事情。
Meta 的招聘策略、微软与 OpenAI 的关系以及对超级智能的追逐
(00:13:10)
The discussion explores Meta's strategy of offering substantial bonuses to retain top AI researchers, questioning whether money alone can foster a strong culture. Patel argues that the pursuit of super intelligence is now a primary motivator, driving companies to acquire talent and teams. He then analyzes the complex relationship between Microsoft and OpenAI, highlighting the unique deal structure involving revenue shares, profit guarantees, and IP rights until AGI. Patel points out potential risks for OpenAI, including Microsoft's access to their IP and the ambiguity surrounding the definition of AGI. He notes that OpenAI has diversified its compute resources, partnering with Oracle and others, after initially being exclusive to Microsoft. Patel emphasizes that OpenAI's need for continuous funding makes these complex deals challenging to navigate.
讨论探讨了 Meta 通过提供巨额奖金来留住顶尖 AI 研究人员的策略，并质疑仅靠金钱能否培养出强大的文化。Patel 认为，对超级智能的追求如今已成为主要驱动力，促使各家公司收购人才和团队。随后他分析了微软与 OpenAI 之间复杂的关系，指出其独特的交易结构涵盖收入分成、利润保底以及在实现 AGI 之前的知识产权归属。Patel 指出 OpenAI 面临的潜在风险，包括微软对其知识产权的可访问性以及“AGI”定义的模糊性。他指出，OpenAI 继最初专属微软之后，已将算力资源多元化，与 Oracle 等公司合作。Patel 强调，OpenAI 对持续资金的需求使这些复杂交易难以驾驭。
(13:10) Matthew Berman:
Sam Altman also mentioned that Meta has been giving \$100 million bonus offers to their top researchers. Apparently, none of the top researchers have left. I want to ask, is that a successful strategy just to like throw money at the problem, get the best people in? It feels like maybe the cultural element would be lacking there where, you know, give as much shit as you want to open AI and say, I'm all in. But there are a lot of true believers there in what they're doing. Is that enough to just throw money and get the best researchers where that culture is going to be built?
Sam Altman 还提到 Meta 向他们的顶级研究人员提供了一亿美元的奖金。据说没有顶尖研究人员离职。我想问，仅仅向问题砸钱以吸引最佳人才是否是一种成功的策略？感觉这样可能缺少文化元素——你可以随意批评 OpenAI 并声称自己全身投入，但那边有很多真正的信徒相信他们正在做的事。单靠砸钱并吸引最优秀的研究人员，就足以建立那种文化吗？
(13:42) Dylan Patel:
You zoom back, right? It's like, if you believe super intelligence is the only thing that matters, then you need to chase it. Otherwise, you're a loser, right? And Mark Zuckerberg certainly doesn't want to be a loser. And he thinks he can build it. He can build Super Intelligence too, right? So then questions like, okay, well, what do you do? Well, then you go and try and acquire the best teams out there, right? Thinking machines, right? All these ex-OpenAI people, but also there's some other folks from, you know, Character AI, GDM, Meta, etc. All these great researchers and people and same with SSI. It's Ilya and the people he's recruited.
你把视角拉远看看，如果你相信超级智能是唯一重要的事情，那你就必须去追逐它，否则你就是失败者，对吧？马克·扎克伯格当然不想成为失败者，他认为自己也能打造超级智能，对吧？那么问题来了，你该怎么做？那就去尝试收购外面最优秀的团队，对吧？Thinking Machines——那些前 OpenAI 的人，还包括 Character AI、GDM、Meta 等的研究员，全都是顶尖人才；SSI 也是一样，是 Ilya 和他招募的那批人。
(14:14)
Trying to recruit people from these companies or trying to buy these companies. That didn't work out. So now you go with Alex who's tremendously connected and can help you build the team and now you gotta go get the team. Now, what's the difference between acquiring SSI where there's way less than 100 employees. I think there's less than even 50 employees at SSI. And for \$30 billion, and like, okay, we just paid hundreds of millions of dollars per researcher. And, you know, 10 billion plus for Ilya, right? Like, that's sort of what you just did. And it's like, well, then you're sort of doing the same thing, right?
你试着从这些公司挖人，或者干脆收购这些公司，但都没有成功。所以你选择了 Alex——他人脉极广，可以帮你组建团队，而你现在就得去招人了。那收购 SSI 有什么不同？那里的员工不到 100 人，我觉得甚至不到 50 人，却要支付 300 亿美元，相当于每位研究人员花几亿美元，还有给 Ilya 的 100 亿美元以上，对吧？你其实就在做同样的事情。
(14:49)
As far as Sam is saying that no top researchers have gone, I don't believe that's accurate. I think initially the top researchers definitely did say no. The best researchers, the best people. And you said \$100 million. I've heard a number for someone over a billion actually for one person at OpenAI. But anyways, it's a ridiculous amount of money, but it's like, well, it's the same thing as buying one of these companies, right? Thinking machines at SSI don't have a product. You're buying them for the people.
至于 Sam 说没有顶尖研究人员离开，我认为这并不准确。我觉得最初确实有顶尖研究人员拒绝了。最好的研究人员，最优秀的人才。你说是一亿美元，我听说 OpenAI 有人报价甚至超过十亿美元。但无论如何，这都是荒唐的天价，但本质上跟收购那些公司是一样的，对吧？SSI 的 Thinking Machines 没有产品，你买的就是那群人才。
(15:19) Matthew Berman:
If Super Intelligence is the end-all, be-all, A hundred million dollars, even a billion dollars is really a drop in the bucket compared to one Meta's market cap currently and also the total addressable market of artificial intelligence. I want to talk about Microsoft and OpenAI's relationship a little bit. We're well past the honeymoon phase it seems. It definitely seems to be a.
如果超级智能是终极目标，那么一亿美元、甚至十亿美元，相较于 Meta 当前的市值以及人工智能的整体可开拓市场，都只是九牛一毛。我想稍微谈谈微软和 OpenAI 之间的关系。看来我们已经远远度过了蜜月期，这段关系显然正陷入某种……。
(15:44) Dylan Patel:
This is now a therapy show.
现在这成了心理治疗节目。
(15:47) Matthew Berman:
Yeah, absolutely.
是的，完全正确。
(15:48) Dylan Patel:
Tell me about your feelings, Sam and Satya.
说说你们的感受吧，Sam 和 Satya。
(15:51) Matthew Berman:
Well, this is therapy, right? These are two people and they have a relationship and it does seem to be folding a bit. OpenAI's ambitions seem to have no bounds. Is Microsoft thinking right now, they want to restructure the deal, OpenAI does, Microsoft really has no reason to but like what do you think is going on at like what do you think about the dynamics of this relationship going forward?
嗯，这就是心理治疗，对吧？这两家公司之间存在一种关系，而这段关系似乎正在出现裂痕。OpenAI 的雄心似乎没有边界。微软现在在想什么？OpenAI 想要重谈协议，但微软其实没有理由这么做。你觉得眼下到底发生了什么？你怎么看这段关系未来的走向？
(16:14) Dylan Patel:
Like OpenAI would not be where they are without Microsoft and and Microsoft signed a deal where they get tremendous power. It's a weirdass deal because like OpenAI wanted to be a nonprofit and they cared about AGI but at the same time they Had to give up a lot to get the money. But at the same time, Microsoft didn't want to run into antitrust stuff, so they like structured this deal really weird, right? Which is like, there's like revenue shares, and there's like profit guarantees, and there's like all these different things, but nowhere is it like, oh yeah, you own X percent of the company, right? I think it's like, they get like, it's off the top of my head,
要知道，没有 Microsoft，OpenAI 绝不可能走到今天，而微软签下了一份让他们拥有巨大权力的协议。这份协议非常古怪，因为 OpenAI 想做非营利组织，关注 AGI，但为了筹钱他们不得不做出巨大让步。与此同时，微软又不想踩到反垄断红线，于是把协议设计得离奇复杂：有收入分成，有利润保底，还有各种条款，但没有任何一条是“你拥有公司 X% 的股权”。据我印象……
(16:48)
but I think it's like 20% revenue share, 49 or 51% like, It's like profit share up until some cap and then there's like Microsoft has the IP rights of all of OpenAI IP until AGI. And it's like all of these things are just like nebulous as hell, right? It's like, I think the profit cap might be like 10x again. I'm like going off the top of my head. It's been a while since I looked at it, but it's like if Microsoft gave roughly \$10 billion and OpenAI has, it's a profit cap of 10x, it's like, well like what incentive does Microsoft have to renegotiate now If they get \$100 billion of profit from OpenAI, and until then OpenAI has to give them all their profit, or half of their profit, right?
我记得大概是 20% 的营收分成，49% 或 51% 的利润分成直到某个上限；此外，在实现 AGI 之前，微软拥有 OpenAI 全部知识产权。这些条款都极其模糊。我记得利润上限大约是 10 倍——如果微软投入了 100 亿美元，利润上限 10 倍，那么微软就能从 OpenAI 获得 1000 亿美元的利润。在那之前，OpenAI 必须把全部或一半利润交给微软。微软还有什么动机去重新谈判？
(17:30)
And they get this 20% rev share. And they have access to all of OpenAI's IP. Until AGI, but what is the definition of AGI? Theoretically, OpenAI's board gets to decide when OpenAI hits AGI, but then if that happens, Microsoft will just sue the shit out of them, and Microsoft has more lawyers than God. So it's like this just crazy-ass deal. I think there's a few really worrisome things in there for OpenAI. One of the main things they got removed already, because Microsoft was really scared, I think, about antitrust aspects of this. Which was that OpenAI had to exclusively use Microsoft for compute. They backed off of this last year and it got announced with the Stargate deal this year, right?
他们还能拿到 20% 的营收分成，并且可以访问 OpenAI 的所有 IP，直到 AGI 实现——可是什么才算 AGI？理论上是 OpenAI 的董事会来决定，但真到那时，微软肯定会把他们告个底朝天，微软的律师多得吓人。这份协议简直疯狂。我觉得里面有几条对 OpenAI 特别令人担忧的条款，其中一条去年就删掉了——因为微软很害怕反垄断风险——那就是 OpenAI 必须专门使用微软的算力。去年他们放弃了这条，今年和 Stargate 项目一起对外宣布了。
(18:15)
Which was that OpenAI's gonna go to Oracle and SoftBank and Crusoe and the Middle East to build their Stargate clusters, right? Their next generation data centers. They're still getting a bunch from Microsoft, of course, but a bunch from Stargate. And so, you know, are from Oracle primarily, but the others as well. Whereas before it was that OpenAI could not do that without going directly to Microsoft, right? OpenAI tried to go to CoreWeave initially, but then Microsoft sort of inserted themselves in the relationship like, no, you're exclusively using us. So a lot of GPUs get rented from CoreWeave to Microsoft to OpenAI. But then this like exclusivity ended.
也就是说，OpenAI 打算去找 Oracle、SoftBank、Crusoe 以及中东资金来建设他们的 Stargate 集群——下一代数据中心。当然，他们还是会从微软那边拿来不少资源，但 Stargate 会提供一大部分，主要来自 Oracle，也有其他方。以前 OpenAI 要这么做必须先去微软报备；一开始 OpenAI 想直接找 CoreWeave，结果微软硬生生插进来，要求他们“只能用我们”。于是大量 GPU 从 CoreWeave 租给微软，再到 OpenAI。如今这种排他性条款已经结束。
(18:55)
And now like CoreWeave has big deals signed with OpenAI and Oracle has big deals signed with OpenAI.
现在，CoreWeave 和 OpenAI 签了大额合同，Oracle 也与 OpenAI 签了大额合同。
(19:00) Matthew Berman:
What did Microsoft get? In that exchange where they're going to give up the exclusivity, did they get anything or was it reported that they got anything in exchange for that? Usually it's not just like, okay, cool, we'll give that up.
微软得到了什么？在放弃排他性的交换中，他们有没有获得什么，或者有报道称他们得到了什么作为回报？通常情况下不可能只是“好吧，我们就放弃了”。
(19:10) Dylan Patel:
What's been reported is just that they gave up the exclusivity and in return all they have is a first right of refusal. Anytime OpenAI goes and tries to get a contract for compute, Microsoft can provide that same compute at the same price.
据报道，他们只是放弃了排他性，作为回报仅获得了“优先权”：任何时候 OpenAI 试图签订算力合同，微软都可以以同样的价格提供相同的算力。
(19:24) Matthew Berman:
Reduce risk from antitrust.
降低反垄断风险。
(19:27) Dylan Patel:
Yeah, like antitrust is one of the biggest considerations there, but there's other considerations of course, but like antitrust being one of the big ones because like being the exclusive compute provider to OpenAI is a little iffy. And from OpenAI's perspective, They were just really annoyed that Microsoft was way slower than they needed them to be, right? They just couldn't get all the compute they needed. They couldn't get all the data center capacity, etc. Core, even Oracle, are moving much faster. But even they are not as fast and so OpenAI is turning to other folks as well, right?
是的，反垄断确实是最重要的考量之一，但当然也有其他因素；反垄断尤其关键，因为成为 OpenAI 的唯一算力供应商有点站不住脚。从 OpenAI 的角度看，他们对微软的反应速度远低于需求深感不满——他们拿不到所需的全部算力，也拿不到足够的数据中心容量。CoreWeave 甚至 Oracle 的动作都快得多，但仍然不够快，因此 OpenAI 也在转向其他合作方。
(19:58)
There's that butting of heads there, but nowadays like the real challenging thing here is like, Microsoft has the Monorepro. It has the OpenAI IP. They have rights to it all. They can do whatever they want with it. Now, whether it's like Microsoft playing nice and not doing stuff with it or being somewhat incompetent and not being able to leverage it and mostly just like looking through it, whatever the reason is, Microsoft You know, despite having access, hasn't done a ton, but the possibilities are endless, right? Like of what could be done. Then the other thing is like if you're truly like AGI or now super intelligence-pilled,
双方因此摩擦不断，但如今真正棘手的是：微软掌握着单一代码库（Monorepo），拥有 OpenAI 的全部知识产权，对其享有使用权，可以随心所欲。无论微软是“友好”而不动用这些资产，还是有点力不从心而无法充分利用、仅仅浏览一下，原因如何都好，微软虽然拥有访问权，却并未做太多，但潜在操作空间几乎无限。再者，如果你是真正的 AGI/超级智能信徒——
(20:36)
you have all the IP up until a super intelligence is, let's just say achieved. But that would imply that like the day before super intelligence is achieved, you have all of the IP and then it gets cut off. But you have all the IP up until right there. So it's like one day of work. Maybe it's hard and it takes, 10 days of work instead of one. Or maybe you achieved super intelligence, but it takes some time to get to the deliberations and agreement that you've achieved super intelligence slash all the evidence that you've achieved super intelligence. But like you've claimed it at this date, but like the model that is super intelligent is here. Like you've made it here.
在“超级智能达成”之前，你拥有所有 IP。也就是说，假设超级智能实现的前一天，你仍拥有全部 IP，然后突然被切断。但在那之前，你都可以使用这些 IP。可能只需一天，也可能艰难一些，需要十天。不排除你实现了超级智能，但要花时间讨论、认定并收集证据证明它已经达到超级智能。你宣称是在某天达成的，可模型其实已经在这里了，是你做出来的。
(21:09)
Microsoft has access to it, right? So sort of that's the real big risk or sort of to the super AGI-pilled Folks, the profit share and all this is like very clean and difficult. And most people don't care that much during when they're investing in open AI. It is challenging to get every investor in the world to be like, yeah, you're crazy ass structure, nonprofit for profit, all this sort of stuff. Okay, that's fine. Oh, Microsoft has rights to all of your profit for a long time and all your IP. So theoretically, you could be worthless if they decide to just like take some of your best researchers and implement everything themselves. Oh, wow, right?
微软对此依旧享有访问权，对吗？这就是最大的风险所在。对那些沉迷 AGI 的人来说，利润分成等条款又干净又严苛。投资 OpenAI 时，大多数人并不在意这些细节；要让全球每一位投资者都接受这种疯狂的架构——非营利与营利兼有——非常困难。好吧，微软长期拥有你全部利润和全部 IP 的权利，所以理论上如果他们挖走你最优秀的研究员并自行实现一切，你就可能一文不值。多可怕！
(21:44)
Like these sorts of things scare investors and Sam said himself, OpenAI is going to be the most capital intensive startup in the history of humanity. The valuation is going to keep soaring because of what they're building. OpenAI has no plans to produce profit anytime soon. They've been around for so long and they're doing like \$10 billion of revenue and they're still not going to do profit for another five years. And by then their projections of revenue are like well north of like there are hundreds of billions if not trillion dollars of revenue that they expect themselves to have before they ever turn a profit.
这些情况让投资者胆寒。Sam 自己也说过，OpenAI 将是人类历史上最烧钱的初创公司；鉴于他们的项目，估值将持续飙升。OpenAI 近期完全没有盈利计划：虽然已经存在多年，年收入约 100 亿美元，但未来五年仍不会盈利。届时他们预计收入会远超数千亿美元，甚至达到万亿美元级别，然后才可能盈利。
(22:18)
And so that whole way through they're gonna be losing money and they need to keep raising money and they need to be able to convince everyone who's an investor in the world and like these things are dirty, right? Like they're not clean and easy to understand.
因此，在整个过程中他们都将持续亏损，不断融资，并且必须说服全球所有潜在投资者；而这些条款又复杂难解，远非干净利落、容易理解。
The Failure of GPT-4.5 and the Importance of Data in AI Training
GPT-4.5 的失败与数据在 AI 训练中的重要性
(00:22:29)
The conversation shifts to GPT-4.5 (Orion), which was ultimately deprecated. Patel explains that it was a bet on full-scale pre-training but proved too slow and expensive. Despite being smarter, it wasn't as useful as other models. The model suffered from over-parameterization, memorizing data instead of generalizing effectively. A bug in the training code further complicated the process. Patel highlights the Chinchilla paper, which emphasizes the optimal ratio of parameters to tokens in a model. GPT-4.5 failed because it didn't have enough data relative to its parameters. The success of reasoning-based models, like Strawberry, demonstrates the importance of generating high-quality data, contrasting with the bad data often found in synthetic datasets.
讨论转向 GPT-4.5（代号 Orion），该模型最终被弃用。Patel 解释说，这是一场对全面预训练的赌注，但结果发现模型过慢且成本高昂。虽然更“聪明”，却不如其他模型实用。模型过度参数化，倾向于记忆数据而非有效泛化；训练代码中的一个漏洞进一步加剧了问题。Patel 提及 Chinchilla 论文，该论文强调模型参数与训练标记数的最佳比例。GPT-4.5 之所以失败，是因为其参数规模远超可用数据量。以 Strawberry 为代表的推理型模型成功表明，生成高质量数据至关重要，这与合成数据集中常见的低质量数据形成鲜明对比。
(22:29) Matthew Berman:
Okay, so you talked a little bit about compute capacity and specifically with Azure being able to go to CoreWeave and elsewhere. I want to talk specifically about 4.5, GPT4.5. It was deprecated, I believe, last week. This was a massive model.
好的，你刚才谈到算力容量，特别是 Azure 可以转向 CoreWeave 等其他供应商。我想具体聊聊 4.5，也就是 GPT-4.5。我记得它上周被弃用了。这是一个庞大的模型。
(22:45) Dylan Patel:
Was it really?
真的吗？
(22:46) Matthew Berman:
It wasn't. Oh, I don't know.
并没有吗？哦，我不确定。
(22:47) Dylan Patel:
I thought it was still available in chat. That's why I was just curious.
我以为它在聊天中仍可用，所以我才好奇。
(22:50) Matthew Berman:
Oh, maybe they just announced the deprecation, but it was imminent.
哦，也许他们只是宣布即将弃用，但这已迫在眉睫。
(22:55) Dylan Patel:
No, it's still there. But yeah, they announced it. Okay. No, no, they've talked about like no use. There's very little usage of it. So that makes sense.
不，它仍在。但对，他们宣布了。好吧，他们说几乎没人用，对其使用量极低，这就说得通了。
(23:02) Matthew Berman:
Was the model too big? Was it too costly to run?
模型是不是太大？运行成本是否过高？
(23:05) Dylan Patel:
What went wrong with GPT4.5? Orion, as it was like internally called, what they hoped would be GPT5. They made that bet like in early 24, right? They started training it in early 24. It was a bet on full scale, right? Full scale pre-training. We're just going to take all the data. We're going to make this ridiculously big model and we're going to train it. It is much smarter than 4.0 and 4.1 to be completely clear. I've said it's the first model to make me laugh because it's actually funny. But in general, it's not that useful and it's too slow. It's too expensive versus other models, right? Like O3 is just better. They went pure on the pre-training scaling. Data doesn't scale, right?
GPT-4.5 出了什么问题？它在内部被称为 Orion，本来他们希望它会成为 GPT-5。他们大约在 24 年初就押注了这一点，对吧？他们在 24 年初开始训练。那是一场全规模的赌注，对吧？全面的预训练。我们要拿走所有数据，造一个荒唐地庞大的模型并把它训练出来。明确地说，它比 4.0 和 4.1 聪明得多。我说过，这是第一个能把我逗笑的模型，因为它真的很有趣。但总体而言，它并不那么有用，而且速度太慢、成本太高，不如其他模型，比如 O3 就更好。他们完完全全押注在预训练的扩展上。然而数据并不会随之扩展，对吧？
(23:46)
So they weren't able to get a ton of data. So without data scaling so fast, right, they have like this model that's really, really big, trained on all this compute. But you have this issue called over-parameterization. Generally in machine learning, if you build a neural network and you feed it some data, it will tend to memorize first. And then it will generalize, right? IE, it'll know that if I ever say the quick brown fox jumped over, it would just know the next token is always lazy, right? It isn't until you've trained it on a lot more data that it learns what quick brown fox even means, what lazy dog is, right? It doesn't actually build the world model, it generalizes.
因此他们无法获得大量数据。数据增速跟不上时，他们就得到这样一个非常非常大、用巨大算力训练出来的模型。但由此产生了过度参数化的问题。通常在机器学习中，如果你构建一个神经网络并喂给它一些数据，它首先会倾向于记忆，然后才会泛化。也就是说，例如当我说 “the quick brown fox jumped over”，它就知道下一个 token 一定是 “lazy”。只有在你用更多数据训练后，它才会学会 quick brown fox 的含义以及 lazy dog 是什么。它不会真正建立世界模型，只是泛化。
(24:25)
And to some extent, GPT4.5 Orion was so large and so overparameterized that it memorized a lot. Actually, when it initially started training, I know people at OpenAI were so excited that they were like, oh my god, it's already crushing the benchmarks and we're barely into training.
在某种程度上，GPT-4.5 Orion 太大且过度参数化，以至于记住了大量内容。实际上，当它刚开始训练时，我知道 OpenAI 的一些人非常兴奋，他们说：天哪，我们才刚开始训练，它就已经碾压基准了。
(24:42) Matthew Berman:
Because some of the checkpoints were just so good.
因为其中一些检查点实在太好了。
(24:44) Dylan Patel:
Right, initially. But that's because it just memorized so much. But then it stopped improving. It was just memorized for a long time and it didn't generalize. It finally did generalize, right? Because it was such a big complicated run, they actually had to bug in it for months during the training. Training is usually a handful of months or less, right? It's usually less. And they had a bug in the training code for a couple months that was like a very tiny bug that like was messing up the training. It's funny like when they finally found it, it was like a bug within PyTorch that like OpenAI had like had found and fixed and they submitted the patch,
没错，起初是这样。但那只是因为它记住的东西太多了。随后模型停止提升，长时间停留在记忆阶段而无法泛化。最终它确实泛化了，但因为这一轮训练极其庞大而复杂，过程中代码里有一个 bug 持续了数月。通常训练只需几个月甚至更短，可他们的训练代码里有一个极小的错误却把整个训练搞砸了。很有趣的是，当他们终于找到问题时，发现是 PyTorch 里的一个 bug——OpenAI 发现并修复了它，还提交了补丁。
(25:22)
there's like 20 people at OpenAI who reacted to the bug, fixed reaction with like emojis right on GitHub. Another thing is like they had to restart training, you know, from checkpoints a lot. It's so big, so complicated, so many things can go wrong, right? And so from an infrastructure perspective, just corralling that many resources and putting them together and having it trained, And having it trained stably was really, really difficult. But from another flip side, it's just like, even if the infrastructure and code and everything like that was pristine, you still have this problem of data, you know, you're sort of like, everyone points to the chinchilla paper from 22, I think.
OpenAI 大约有 20 个人在 GitHub 上对这个 bug 的修复做出反应，用表情符号庆祝。另一方面，他们经常不得不用检查点重新开始训练；模型如此巨大且复杂，出错的地方太多。从基础设施角度看，调度如此多的资源并让训练稳定运行极为困难。即便基础设施与代码都无懈可击，数据问题依旧存在——大家都会提到 2022 年的 Chinchilla 论文，我想。
(25:59)
In 2022, Google released a paper called Chinchilla DeepMind. And what it basically said is like for a model, what's the optimal ratio of parameters to tokens? And this only applied to dense models with the exact architecture of Chinchilla model. But it was like, oh, if I have like X flops, I should have this many parameters, this many tokens, right? It's a scaling law, right? Obviously, as you make it bigger, And you apply more flops, the model gets better. But how much data should I add? How much more parameters should I add, right? Now, obviously, so over time, you know, people's architectures change, the exact observations of chinchilla aren't accurate,
2022 年，谷歌发布了一篇名为 Chinchilla DeepMind 的论文，核心观点是：模型的参数量与训练标记数之间存在一个最佳比值。这只适用于拥有 Chinchilla 架构的稠密模型。论文指出，如果我有 X 次浮点运算，就应配备多少参数和多少训练标记。这是一条扩展定律：显然，模型越大、计算越多，效果越好。但我应该增加多少数据？再增多少参数？当然，随着时间推移，架构变化，Chinchilla 的精确结论已不再准确。
(26:35)
right, which is that like, roughly, it's 20 tokens per parameter that you want of data that you're training versus parameters in the model, roughly, there's actually a curve and everything, it's more complicated than that. But like that observation is not like identical. But what it is, is that like, as you add compute, you want to add more data and parameters at a certain ratio, Or along a certain curve of like, you know, there's a formula, basically, in an ideal world. And they didn't go there, right? They had to go to way more parameters versus tokens. But this was all early 24 when they started training. You know, all these trials, tribulations, they finally get there.
论文大致指出，每个参数应配约 20 个训练标记，当然实际情况是一条曲线，复杂得多，并非绝对一致。但核心思想是：随着算力增加，数据和参数应按某种比例同步扩展，沿着某条理想曲线前进。而他们并未遵循这条曲线，参数量远超标记量。这些事都发生在 24 年初训练开启之时；经历种种曲折，他们最终走到了今天。
(27:08)
And I don't remember when they released 4.5. It was last year, right?
我不记得他们什么时候发布 4.5 了，是去年，对吧？
(27:11) Matthew Berman:
Yeah.
是的。
(27:11) Dylan Patel:
Yeah. But they finally released the model, you know, many months after they start training, after they finish training, pre-training, and then they try to do RL and all this stuff. But in the meantime, different teams at OpenAI figure out something magical, which is the reasoning stuff, the strawberry.
是的。不过在开始训练、完成预训练并尝试强化学习等流程后的好几个月，他们最终发布了这款模型。与此同时，OpenAI 内部的其他团队发现了一件神奇的事——也就是“草莓”式推理方法。
(27:26) Matthew Berman:
So it was like while they've already invested all this, while they're in process of training this massive model, they realized, okay, for a much lower cost, we can get so much more efficiency, so much higher quality out of a model because of reasoning.
也就是说，在他们已经投入巨大资源、正训练这款庞大模型的过程中，他们意识到：通过推理机制，以远低的成本就能让模型获得更高效率、更高质量。
(27:39) Dylan Patel:
And if you really like try and boil down reasoning to first principles, you're giving the model a lot more data. Where are you getting this data from is you're generating it. And how are you generating the data? Well, you're creating these verifiable domains where the model generates data and you throw away all the data where it doesn't get to the right answer. Right, where it doesn't verify that that math problem or that code or that unit test was good. So in a sense, it's kind of like, you know, like, you know, looking backwards, obviously, I didn't have the intuition then, but like, looking backwards, the intuition makes a lot of sense that like,
如果真要把推理追溯到第一性原理，你其实是在给模型提供更多数据。这些数据从哪来？是你生成的。你如何生成数据？你构建可验证的场景，让模型生成数据，并丢弃所有得不出正确答案的记录——也就是无法验证数学题、代码或单元测试正确性的输出。所以，从某种意义上说，回过头来看（当时我并没有这种直觉，但现在回望），这种直觉确实非常合理：
Apple's AI Challenges and the Debate Over On-Device AI
苹果的 AI 挑战与设备端 AI 之争
(00:28:13)
The discussion turns to Apple's AI efforts, noting that they are behind in the AI race. Patel attributes this to Apple's conservative nature, difficulty attracting AI talent, and historical aversion to NVIDIA. He recounts the "BumpGate" incident, where faulty NVIDIA GPUs led to a strained relationship. The conversation then explores the concept of on-device AI, with Patel expressing skepticism. He argues that consumers prioritize free and convenient cloud-based AI over the security benefits of on-device AI. He also questions the latency advantages of on-device AI, noting that many valuable AI applications require cloud connectivity to access data and perform complex tasks.
讨论转向苹果的 AI 布局，并指出苹果在 AI 竞赛中已落后。Patel 将此归因于苹果的保守文化、难以吸引 AI 人才，以及其对 NVIDIA 的长期排斥。他回顾了因 NVIDIA GPU 故障引发的“BumpGate”事件，导致双方关系紧张。随后对话探讨了设备端 AI 的概念，Patel 表示怀疑。他认为消费者更看重免费且便捷的云端 AI，而非设备端 AI 带来的安全优势。他也质疑设备端 AI 的时延优势，指出许多高价值 AI 应用仍需云连接来访问数据并执行复杂任务。
(28:06) Dylan Patel:
well, 4.5 failed, because it didn't have enough data. Also, it was just very complicated and difficult on a scaling perspective, infrastructure wise. And there were tons of problems and challenges there. But also, they shouldn't have enough data. And now like this breakthrough that happened from a different team is generating more data. And that data is good, right? Like a lot of the synthetic data stuff is like bad data. But like the magic of strawberry, you know, of reasoning is that the data is good, the data that you're generating. So it's really like from a first principles basis makes a lot of sense that data is the wall. You know, just adding more parameters doesn't do anything.
好吧，4.5 失败是因为数据不够。此外，从扩展和基础设施角度来看，它非常复杂、难以运维，问题和挑战一大堆。根本原因还是数据不足。现在来自另一团队的突破在生成更多数据，而且这些数据质量很好——与许多低质的合成数据不同。“草莓”式推理的神奇之处就在于生成的数据本身很优质。从第一性原理来看，这充分说明“数据才是壁垒”，光增加参数并无意义。
(28:42) Matthew Berman:
I want to talk about Apple for a second. I'm sure you have some thoughts on that. Apple is clearly behind. We're not getting much in the way of public models, leaks, anything about knowing what they're doing. What do you think is going on at Apple? Do you think they just made a misstep?They kind of were late to the game. Why aren't they acquiring companies? What is happening internally, if you had to guess?
我想聊一下苹果。我相信你对此有些看法。苹果显然落后了，我们几乎看不到公开模型、泄漏或任何能了解他们动作的消息。你觉得苹果内部发生了什么？他们是否走错了步？他们似乎入局较晚，为何不收购公司？如果让你猜，内部到底在发生什么？
(29:05) Dylan Patel:
Yes, I think Apple is like very very conservative company. They've acquired companies in the past, but they never done really big acquisitions.
是的，我认为苹果是一家非常非常保守的公司。过去他们也收购过公司，但从未做过真正的大型并购。
(29:14) Matthew Berman:
Beats was the biggest one. Yeah, it's a headphone company.
Beats 是最大的那次收购。对，它只是一家耳机公司。
(29:16) Dylan Patel:
Right, but generally their acquisitions have been really small. They do buy a lot of companies. They just buy really, really small companies early. They identify it. Maybe it's a failing startup or it's, you know, whatever it is. They buy these startups that haven't achieved product market fit and aren't like super sexy. As far as like Apple, they've always had problems attracting. In terms of AI researchers, AI researchers like to blab. They like to post and publish their research. And Apple's always been a secretive company. They actually changed that policies to where their AI researchers are allowed to publish. But at the end of the day, they're still a secretive company.
没错，但总的来说，他们的收购规模都非常小。苹果确实收购了很多公司，但通常是在早期就收购极其小型的企业。他们会识别目标，也许是一家行将失败的初创公司，或是其他什么类型的公司；这些创业公司尚未实现产品与市场的匹配，也谈不上有多“性感”。就苹果而言，他们一直难以吸引人才。对于 AI 研究人员来说，他们喜欢高谈阔论，喜欢发布并发表研究成果，而苹果一贯是一家高度保密的公司。虽然苹果后来修改政策，允许旗下 AI 研究人员发表论文，但归根结底，苹果依旧是一家保密公司。
(29:53)
They're still like an old antiquated company. It's like Meta only was able to hire a bunch of researchers and talent because they had a bunch of ML talent already. They've always been a leader in AI. They had this PyTorch team as well. And then they committed to open sourcing a lot.
从某种角度看，苹果依旧是家老派而过时的公司。Meta 之所以能招到一批研究人员和人才，是因为他们本来就拥有大量机器学习人才，一直在 AI 领域处于领先地位，并且拥有自己的 PyTorch 团队；随后 Meta 致力于大量开源。
(30:09) Matthew Berman:
For a while now, they've been open sourcing, yeah.
是的，他们已经开源有一段时间了。
(30:11) Dylan Patel:
Besides that, who's been able to acquire AI talent? The DeepMind to OpenAI shift, OpenAI being the rival to DeepMind and that whole thing, and a lot of great researchers coming together to form it. And then the Anthropic Splinter Group, And then like Thinking Machine Splinter Group from OpenAI and SSI Splinter Group from OpenAI, right? It's like what companies have actually been able to acquire talent that didn't already have AI talent. It's like Google DeepMind is just like the biggest name in the game and they've always had the highest inflows of AI researchers and PhDs. And then there's like OpenAI and Anthropic who are sort of, and Thinking Machines and SSI, right? It's all OpenAI.
除此之外，还有谁真正招揽到了 AI 人才？从 DeepMind 到 OpenAI 的转变——OpenAI 作为 DeepMind 的竞争对手，聚集了众多顶尖研究人员；随后出现的 Anthropic 分支团队，以及来自 OpenAI 的 Thinking Machine 和 SSI 等分支团队。事实上，能够在原本不具备 AI 人才的情况下吸引顶尖人才的公司寥寥无几。谷歌 DeepMind 依旧是业内最大牌，一直吸引着最多的 AI 研究人员和博士；紧随其后的就是 OpenAI、Anthropic、Thinking Machines、SSI——本质上都与 OpenAI 密切相关。
(30:48)
It's hard to get talent to come to you. Now, Anthropic has such a strong culture that they're able to get people, OpenAI is being the leader. Meta, you know, I sort of talked through, it's like, how is Apple gonna attract these best researchers? They're not, right? They're gonna get, you know, not the best researchers. And so it's really challenging for them to be competitive, right? And then there's the whole like, they have a stigma against, they hate NVIDIA. And like maybe for reasonable reasons, you know, NVIDIA threatened to sue them over some patents at one point.NVIDIA sold them GPUs that ended up breaking. It was called BumpGate. It was a very interesting thing.
要让人才主动投奔你非常困难。Anthropic 拥有强大的文化吸引力，因此能招到人；OpenAI 作为领头羊自然更具号召力。至于 Meta，我之前也讨论过。问题是：苹果要如何吸引最顶尖的研究人员？他们做不到——能招来的并非最优秀的人才，这使得他们在竞争中十分艰难。此外，苹果对 NVIDIA 存在偏见，甚至可以说“讨厌” NVIDIA，可能也有合理原因：NVIDIA 曾因专利问题威胁起诉苹果，还曾向苹果出售最终出现故障的 GPU，引发了所谓的 “BumpGate” 事件，非常有意思。
(31:28) Matthew Berman:
I don't remember that.
我不记得那件事了。
(31:29) Dylan Patel:
You don't or you do?
你是不记得，还是记得？
(31:30) Matthew Berman:
No, I don't.
不，我不记得。
(31:31) Dylan Patel:
Oh, okay. So this is a very fun story, right? One generation of NVIDIA's GPUs. I'm going to butcher the exact reason because it's been a while since I read about it.
哦，好的。这可是一段有趣的故事。那是某一代 NVIDIA GPU 的问题。我可能会记错细节，因为我已经很久没读到相关资料了。
(31:37) Matthew Berman:
How many years ago was this?
这是多少年前的事？
(31:39) Dylan Patel:
This was like, this is probably like, 2015, if not earlier. There's a generation of NVIDIA GPUs for laptops, right? And chips have solder balls on the bottom, right? That connect their IO pins to the motherboard and, you know, to the CPU power, et cetera. Somewhere along the chain, supply chain, all the companies, Dell, HPE, Apple, Lenovo, they blamed NVIDIA as far as I understand, but vice versa. NVIDIA said it wasn't their fault. I'm not gonna prescribe blame, but like the solder balls would not like, they were like not, Good enough, right? And so when the temperatures swung up and down, coefficient of thermal expansion, right, different materials expand and shrink at different rates.
大概在 2015 年，或许更早。有一代面向笔记本电脑的 NVIDIA GPU。芯片底部有焊球，将 IO 引脚连接到主板和 CPU 供电等。供应链中的各家公司——戴尔、HPE、苹果、联想等——据我所知都把责任归咎于 NVIDIA；反过来 NVIDIA 则表示并非自己的问题。我不去判定责任，但那些焊球质量确实不过关。当温度上下波动时，由于热膨胀系数不同，不同材料的膨胀和收缩速率也不同。
(32:20)
The chip versus the solder balls versus the PCB would expand and shrink at different rates. And what ended up happening is because of that different rate of expansion, the solder balls connecting the chip and the board would crack. They would, it was called buf gate. And now the connection is severed. The connection between the chip and the board. So it's called BumpGate. And I think Apple wanted compensation from NVIDIA. I think NVIDIA was like, no. There's this whole thing. Apple really hates NVIDIA because of that and because of this threatening, when NVIDIA was trying to get into mobile chips, because they tried to get into mobile chips for a time period and they failed.
芯片、焊球和 PCB 的膨胀收缩速率不同，最终导致焊球与芯片和主板之间的连接开裂，这就是所谓的 BumpGate。连接断裂后，芯片与主板失去联系。我记得苹果因此向 NVIDIA 索赔，但 NVIDIA 拒绝了。再加上 NVIDIA 当年尝试进军移动芯片失败，却又用 GPU 专利威胁所有厂商，这两件事让苹果非常痛恨 NVIDIA。
(32:57)
But at one point, they tried to sue everyone over GPU patents in mobile. And so between those two things, Apple really doesn't like NVIDIA. And so Apple doesn't really buy much NVIDIA hardware.
当时 NVIDIA 确实试图用移动 GPU 专利起诉所有厂商。结合这两件事，苹果非常不喜欢 NVIDIA，因此几乎不采购 NVIDIA 硬件。
(33:10) Matthew Berman:
They don't really need to anymore.
他们现在也确实不太需要了。
(33:11) Dylan Patel:
Well, they don't need to in the laptops, of course, but like even in data centers. It's like, well, again, if I'm a researcher, first of all, I'm going to go where the talent is, where I have my culture fit, where the money is. And even in places that have a ton of compute and good researchers, Meta still has to offer crazy money to get people to come over. It's like Apple, one, is not going to offer that crazy money. And also they don't even have compute. And then for inference to serve users, they run it on Mac chips and data centers. It's very bizarre. And it's like, I don't want to deal with all that stuff. I want to build the best models, right? It's challenging for Apple.
笔记本当然不需要 NVIDIA，但即便在数据中心也是如此。作为研究人员，我首先会去有顶尖人才、文化契合和高薪的地方。即使拥有大量算力和优秀研究员的 Meta，也得开出疯狂的薪酬来挖人。苹果一方面不会给出那样的高薪，另一方面他们甚至缺乏足够的算力。为了给用户做推理服务，他们在数据中心使用 Mac 芯片，这非常怪异。我不想处理这些繁琐问题，我只想训练最好的模型——对苹果来说，这确实困难重重。
(33:43) Matthew Berman:
Okay, I want to ask you one last question about Apple. They are very big on on-device AI. And I actually really like that approach. Security, latency. What's your take on on-device AI pushing AI to the edge versus having it in the cloud? Is it somewhere in the middle? What do you think?
好的，我想再问你最后一个关于苹果的问题。他们非常重视设备端 AI，而我其实很喜欢这种做法——安全性和延迟方面的优势。你怎么看待把 AI 推向终端而不是放在云端？还是说会采取折中方案？你怎么想？
(34:01) Dylan Patel:
So I think there's... I'm generally an on-device AI bear. I don't like it that much. But personally, I think security is awesome. But I know human psychology, like free is better than – free with ads is better than security. No one actually cares about security. They say they do. The number of people who actually make decisions based on security are very little. I would like privacy and security, of course.
嗯，说实话，我总体对设备端 AI 持悲观态度，没那么喜欢它。但就个人而言，我觉得安全性确实很棒。不过我了解人类心理：免费更香——哪怕带广告的免费也比安全更受欢迎。实际上没人真正关心安全，虽然都声称自己关心；真正以安全为主要决策依据的人极少。当然，我本人还是希望拥有隐私和安全。
(34:26) Matthew Berman:
Wait, but you said, you know, it's, you like free, but you're not, you're not, that's not analogous to on-device AI, right?
等等，可是你说你喜欢免费，但这和设备端 AI 并不等同，对吧？
(34:32) Dylan Patel:
No, no. So like, Meta will offer in the cloud for free. And OpenAI has a free tier. You know, Google has a free tier.
不，不。比如 Meta 会在云端免费提供服务，OpenAI 也有免费层，Google 同样如此。
(34:39) Matthew Berman:
And it's going to be better than free as in running it on your own device.
而且云端的免费服务会比在自己设备上跑免费模型效果更好，对吧？
(34:42) Dylan Patel:
Right, right. And that's a big challenge with that is that On device is that you're limited by the hardware, right? And so how fast the model can inference is really based upon your memory bandwidth of the chip. And okay, if I want to increase the memory bandwidth of the chip, I spend, you know, 50 more dollars of hardware, I pass on the cost to the customer, it's 100 more dollars for the iPhone. Great, with 100 bucks, I could have like, I could have like 100 million tokens, right? And it's like, I'm not consuming 100 million tokens. Or better yet, 100 bucks, I just save it and Meta will give me the model for free on WhatsApp and Instagram.And OpenAI will give it free on ChatGPT.
没错，这正是设备端 AI 的大难题：它受制于硬件。模型推理速度实际上取决于芯片的内存带宽。假如我想提升带宽，就得多花 50 美元硬件成本，最终让顾客多付 100 美元买 iPhone。可是用这 100 美元，我可以获取大约 1 亿个 token 的云端推理额度，而我根本用不了那么多。更妙的是，这 100 美元我干脆省下——Meta 会在 WhatsApp 和 Instagram 上免费给我模型，OpenAI 在 ChatGPT 上也免费提供。
(35:19)
And Google will give it free on Google, right? It's like, it's really challenging from that perspective. And then lastly, I don't agree with the latency standpoint, right? I think there's certain use cases where transformers make sense for latency. Super tiny next word prediction on your keyboard or a spelling. But the AI workloads that are the most valuable to you and I are search a restaurant at this time and go find it from a personal standpoint.
Google 也会在自家服务里免费提供，对吧？从这个角度看，设备端 AI 真的很难打。最后我也不同意所谓的延迟优势。我承认某些场景下变换器模型的延迟很关键，比如键盘上的下一个词预测或拼写校正，但对你我最有价值的 AI 工作负载是“帮我在这个时间点找一家餐厅”这类任务。
(35:46) Matthew Berman:
Or access to my Gmail, my calendar, that's all in the cloud anyways.
再比如访问我的 Gmail 或日历——反正这些都在云端。
(35:49) Dylan Patel:
Right. Within business, there's tons of use cases, but my data's all in the cloud anyways. For personal, you and I, it's like search restaurant, go through Google Maps, make all these calls, right? Go through my calendar, go through my email. All this data's in the cloud anyways, A. B, if it's more of an agentic workflow in terms of like, Yeah, you know, I'm really feeling Italian and find a restaurant that's between you and I in location. And, you know, like we're thinking about Italian, but make sure they have gluten free options because he's gluten free. You know, find me a restaurant with a reservation at 7 p.m. tonight. Like this is a deep research query. And then you get a response.
没错。在企业场景中有大量用例，但我的数据反正都在云端。至于个人用户，比如你我想找餐馆：在 Google 地图上搜索、拨电话，对吧？再查看我的日历、邮件——这些数据同样都在云端。其次，如果是更偏代理式的流程，比如：我现在想吃意大利菜，找一家位于你我之间的餐馆，而且要有无麸质选项，因为他得无麸质；再帮我订今晚 7 点的座位——这是一项深入检索请求，然后你才会得到结果。
(36:25)
It's like, well, that took minutes. Or like, you know, we envision the future where the AI books flights for us. It's like this is not a like book the flight. OK, it's book. It's like book the flight. It's researching. It's finding stuff. And it comes back. But it's going through the web. It's going through cloud, right? Where is the necessity for it to be on device? And because of the hardware constraints, even if it is a streaming tokens thing, your phone cannot run LLAMA-7B as fast as I can query a server, run LLAMA-7B and transmit the tokens back to myself, right? And no one wants to run LLAMA-7B. They want to run, you know, GPT-4.5 or 4.1 or O3 or Claude Opus or whatever, right?
结果可能要好几分钟。再比如我们设想未来让 AI 替我们订机票——这可不是简单的“订机票”动作，而是要搜索、筛选、查找信息，然后再返回结果。但整个过程都在通过网络、走云端，对吧？为什么非得在设备端完成？受限于硬件，即便只是流式输出 token，你的手机运行 LLAMA-7B 的速度也比不上我去服务器调用 LLAMA-7B 再把 token 传回来的速度。而且没人想跑 LLAMA-7B，大家想用的是 GPT-4.5、4.1、O3 或 Claude Opus 之类的优质模型，对吧？
(37:07)
They want to use a good model, right? And those models can't possibly run on device. So it's a really difficult place for the use cases there with integrated all my data, but it's in the cloud anyways. And it's like, how much of my data does Meta have? Does Google have? Does Microsoft have? Let me plug into all those. Or in the way Anthropic is doing it, they've done this MCP stuff and they're connecting in. You can connect your Google Drive to Anthropic, right? And it's like, oh, wait, even if I don't have my data with Anthropic, they're still able to connect to it if I give them the rights to. So it's like, where's the benefit of on-device AI truly from a use-case standpoint?
大家都想用好模型，而这些模型根本不可能在设备端运行。所以对于那些需要整合我全部数据的用例来说，设备端 AI 处境非常尴尬——反正数据都在云端。Meta 掌握我多少数据？Google、Microsoft 又掌握多少？干脆全部接入吧。Anthropic 也采取类似做法，推出 MCP，可以连接外部数据；你可以把 Google Drive 接入 Anthropic。这样即使我的数据不存放在 Anthropic，只要我授权，他们仍能访问。那么从实际用例角度看，设备端 AI 究竟有什么优势呢？
The Future of AI: Cloud vs. On-Device, and the Competition Between NVIDIA and AMD
人工智能的未来：云端与设备端之争，以及 NVIDIA 与 AMD 的竞争
(00:37:42)
Patel continues his critique of on-device AI, suggesting that it will be limited to low-value applications due to hardware constraints. He envisions a future where wearables perform basic tasks locally, while more complex reasoning occurs in the cloud. The conversation transitions to the competition between NVIDIA and AMD in the AI chip market. Patel acknowledges that AMD is trying hard and their hardware has some advantages, but their software stack lags significantly behind NVIDIA's. He notes that NVIDIA's Blackwell chip is objectively superior and that NVIDIA's ecosystem, including CUDA and inference libraries, provides a better user experience.
Patel 继续批评设备端 AI，指出受硬件限制，它只能用于低价值场景。他设想未来可穿戴设备将在本地执行基础任务，而更复杂的推理将在云端完成。话题随后转向 NVIDIA 与 AMD 在 AI 芯片市场的竞争。Patel 承认 AMD 努力追赶，硬件也有一些优势，但其软件栈显著落后于 NVIDIA。他指出 NVIDIA 的 Blackwell 芯片在客观性能上更强，加之 CUDA 及推理库等生态系统，整体用户体验更佳。
(37:42) Dylan Patel:
There's certainly one from a security standpoint, but the actual use case is like.
从安全角度看确实有一点优势，但真正的使用场景是……
(37:46) Matthew Berman:
Yeah, I think there's probably an argument for a little bit of both, and it probably does skew in terms of the total workload towards cloud, but I think there's an argument for doing at least a portion of the workload on device, anything that you're interacting with the device on. You mentioned typing ahead, and that makes a lot of sense.
是的，我认为两者兼用有其道理，总体工作负载可能偏向云端，但至少有一部分任务应当在设备端完成，尤其是那些直接与设备交互的操作。你提到的键盘预测就很合理。
(38:05) Dylan Patel:
I do think AI will make its way on device. I think it'll just be very low value AI where the cost structure is just so low. I don't think people should design hardware on phones for AI that's going to make it more expensive. If you're going to keep the phone the same price point, add AI capabilities, great. But if you're going to increase the price point, I don't think consumers will do it. How AI on device really will make sense is like, for example, a wearable, an earpiece or smart glasses. And there you're doing small bits and pieces locally, right? Image recognition, hand tracking, but the actual reasoning and thinking is happening in the cloud, right?
我认为 AI 终究会在设备端落地，但只会是价值很低、成本结构极低的 AI。手机硬件若因 AI 而涨价并不划算；如果维持原价同时加入 AI 功能，那很好，但若提高售价，消费者不会买账。设备端 AI 真正合理的形态可能是可穿戴设备，例如耳机或智能眼镜，本地完成图像识别、手势追踪等小任务，而真正的推理和思考仍在云端完成。
(38:46)
And that's sort of the view that sort of like a lot of these wearables are pushing. I think there will be Some AI on devices, obviously everyone's going to try. It's not like Samsung and Apple and like all these companies are going to sit on their hands. They're going to try stuff. I just think the stuff that's actually going to drive user adoption and revenue and improve customers' lives is going to be skewing towards what's on the cloud, which is why Apple has this strategy, right? Apple's building a couple massive data centers, right? They're buying hundreds of thousands of their Mac chips and putting them in data centers.
这也是许多可穿戴厂商所倡导的观点。我认为设备端必然会有一些 AI，大家都会尝试；三星、苹果等公司不会袖手旁观。但真正能推动用户采用、带来收入并改善体验的，仍将偏向云端 AI。这也是苹果的策略：他们正在建设数座大型数据中心，采购数十万颗自家 Mac 芯片并部署到数据中心。
(39:19)
They hired Google's head of rack architecture for the TPU, Andy, to make an, they're making an accelerator, right? They see cloud as like where AI needs to go. They just like also have to push it on device, but like even Apple themselves, although they won't say it, wants to run a lot of this in the cloud.
他们还聘请了谷歌 TPU 机架架构负责人 Andy 来打造加速器。他们认为 AI 的未来在云端，同时也不得不推动设备端部署。但即便苹果自己不会明说，他们也希望把大量 AI 工作负载放到云端运行。
(39:35) Matthew Berman:
And they do have the, they have great chips to do that too. Okay, let's speaking of chips, let's let's talk about Nvidia verse AMD. I have read a couple articles out of artificial analysis. Sorry, semi analysis. Lately, that We have kind of said that these new AMD chips are actually really strong. Do you think AMD, with their new chips, is that enough to really tackle the Cuda moat? Or are they going to start taking market share from NVIDIA?
而且他们确实拥有能够实现这一点的出色芯片。好的，说到芯片，我们来谈谈 NVIDIA 和 AMD。我最近在 Artificial Analysis——哦，SemiAnalysis——上读了几篇文章，文章说这些新的 AMD 芯片其实非常强大。你觉得 AMD 凭借这些新芯片，足以真正跨越 CUDA 的护城河吗？还是他们将开始从 NVIDIA 手中抢占市场份额？
(40:05) Dylan Patel:
So I think it's a confluence of things, right? So AMD is trying really hard. Their hardware is behind in some factors, especially against Blackwell. But there are some ways their hardware is better, right? And I think the real challenge for them is, like you mentioned, software, right? Like the developer experience on AMD is not that great. It's getting better. You know, we've asked them to do a lot of things to change it, like specific fixes and changes on CI resources and all these other things. You know, there's a long list of recommendations. We provided them in December and again more recently. And they've implemented a number of them, a good number of them.
我认为这是多种因素共同作用的结果。AMD 的确在努力追赶，他们的硬件在某些方面落后，尤其是对比 Blackwell，但也有一些地方更好。真正的难题在于软件层面，正如你提到的，开发者在 AMD 平台上的体验并不理想，不过正在改善。我们要求他们做很多改动，例如在 CI 资源等方面的具体修复和调整。去年 12 月我们给出了一长串建议，最近又补充了一些，他们确实落实了不少。
(40:48)
But it's like, there's just so there's so far behind on software. It's incredible. Now, are they going to gain some share? I think they are going to get some share, right? They had some share last year, and they're going to get some share this year. The challenge is like, versus NVIDIA's Blackwell, it's just objectively worse, right? As a chip.
但他们在软件方面的落后仍然非常明显，这让人难以置信。至于市场份额，我认为他们确实会拿到一部分：去年他们已经占到一些，今年还会再增长。不过问题在于，与 NVIDIA 的 Blackwell 相比，单论芯片本身，AMD 还是客观上更差。
(41:06) Matthew Berman:
Oh, the chip alone, not the ecosystem.
哦，只看芯片本身，不考虑生态系统。
(41:09) Dylan Patel:
The chip alone.
没错，只谈芯片本身。
(41:10) Matthew Berman:
Okay.
好的。
(41:10) Dylan Patel:
Because of the system, right? Because NVIDIA is able to connect, network their chips together because of the networking hardware they've put on their chip. With NVLink, right? So the way that NVIDIA can build their servers is like 72 of them work together really tightly, whereas AMD currently they can only have eight of them work together really tightly. And so this is really important for inference and training. And then NVIDIA's got this software stack. It's not just CUDA, right? Like people talk about it's just CUDA, but a lot of people don't touch CUDA, right? Most researchers don't touch CUDA. What they do is they like Call PyTorch, and then PyTorch calls down to CUDA,
因为系统架构的缘故，对吧？NVIDIA 借助芯片上集成的网络硬件（NVLink）将 GPU 互联，使得一台服务器里 72 颗 GPU 可以高度协同，而 AMD 目前最多只能让 8 颗 GPU 高效协作。这对推理和训练至关重要。再加上 NVIDIA 拥有完整的软件栈，不只是 CUDA。大家常说只有 CUDA，但很多人并不直接写 CUDA 代码；大多数研究人员调用的是 PyTorch，而 PyTorch 再向下调用 CUDA。
(41:44)
and automatically it runs on the hardware, whether it's compile or eager mode, whatever you're doing. It generally just maps to NVIDIA hardware really well. In the case of AMD, it doesn't as well. And now, even less than that, so many people aren't even touching PyTorch. They're going to VLLM or SGLANG, which are inference libraries. They're downloading the model weights off of Hugging Face or wherever. And they're plugging it into this inference engine, which is an open source repository on GitHub, either SG Lang or VLM. And then they're just saying go. And then those things are calling, you know, Torch compiled. And those things are calling CUDA and like, you know, or Triton.
代码会自动在硬件上运行，无论是编译模式还是即时模式，总能很好地映射到 NVIDIA 硬件；而在 AMD 平台就没那么顺畅。更进一步，很多人甚至不再直接使用 PyTorch，而是转向 VLLM 或 SG Lang 等推理库：从 Hugging Face 下载模型权重，接入这些 GitHub 上的开源推理引擎，然后直接运行；它们再调用 Torch Compiled、CUDA 或 Triton 等底层组件。
(42:20)
And it's just like, there's like all these libraries down the stack. Really, the end user just wants to use a model, right? They want tokens. And NVIDIA is building libraries here called Dynamo that make this so much easier for the user. And now, obviously, there are people like opening eyes of the world and others who will go all the way down to the bottom. You know, deep seeks and open AIs and metas and stuff. But a lot of users just want to call the open source library, tell them, you know, hey, here's my model weights, run. Here's the hardware, run, right? And here, AMD is trying really hard, but it's still a worst user experience.
整条技术栈里有大量库，但最终用户只想用模型、拿到 token。NVIDIA 正在开发名为 Dynamo 的库，让这一过程更简单。当然，OpenAI 等顶尖团队会深入到底层（DeepSeek、OpenAI、Meta 等亦然），但大量用户只想调用开源库，告诉它们“这是我的权重，跑；这是硬件，跑”。在这方面，AMD 虽然努力，但用户体验仍然更差。
(42:52)
Not that it doesn't work, but it's that like, hey, if I want to use this library, it's like for NVIDIA, there's 10 flags. For AMD, there's 50 flags, right, that I can, you know, in each of these flags, there's different settings and it's like, well, what's the best performance? I don't fucking know, right? Like, you know, so AMD, I think, is getting there, right? They're getting there really fast and they're going to get some share. The other aspect is NVIDIA's not doing themselves favors. There's this ecosystem of cloud companies, right? You know, of course, everyone knows about the Googles, Amazons, you know, Microsoft, Azure, right?
并不是说 AMD 的库不能用，而是用起来很麻烦：在 NVIDIA 平台可能只需设置 10 个参数，而 AMD 平台却有 50 个，每个参数还有不同选项——最佳性能怎么调？我也不知道。所以虽然 AMD 进步很快，也会拿到一些份额，但另一个因素是 NVIDIA 自己也在“自找麻烦”。云计算生态里有很多公司——大家熟知的 Google、Amazon、Microsoft Azure 等。
(43:22)
Those guys have been building AI chips and NVIDIA's been trying to, and NVIDIA's got AI chips, obviously they've always been in contention for a while. And so NVIDIA, as a response, propped up all these other cloud companies. CoreWeave and Oracle, not propped up, like really prioritized them. Oracle, but there's actually over 50 cloud companies out there, Nebius and Together and Lambda. You just go down the list. There's all these different cloud companies. NVIDIA is really helping, right? They're taking what would have been allocations to Amazon and Google and others and saying, hey, you guys can buy them. Right.
这些巨头都在做自研 AI 芯片，而 NVIDIA 也一直在竞争。作为回应，NVIDIA 大力扶持其他云厂商：CoreWeave、Oracle 等被优先供货。实际上还有 50 多家云公司，如 Nebius、Together、Lambda 等等，名单非常长。NVIDIA 把原本要分配给 Amazon、Google 等的部分产能转给这些新玩家，告诉他们：“你们可以买。”
(43:54) Matthew Berman:
Is that to kind of hinder? To lower the play field and make it more of a commodity. Right, right.
这是为了削弱（巨头）吗？让竞争环境更平衡、更像商品化市场？对，对。
(43:59) Dylan Patel:
I mean like you go look at Amazon's margins on GPUs, they're charging like \$6 an hour if you were to just rent a GPU without talking to anyone. Right? Which is like the cost to buy an NVIDIA GPU and deploy it in a data center is like \$1.40 an hour, right? That's the cost. So then like what's a reasonable amount of profit for the cloud? Maybe \$2, maybe \$1.75, right? That's what NVIDIA wants. They don't want all the profit being sucked up \$6 on Amazon. Now obviously you can negotiate with Amazon and get much lower, right? But like you don't want to just like, yeah, it's just like really tough. So NVIDIA is propping up all these different cloud companies, which is driving down the price.
我的意思是，你看看亚马逊在 GPU 上的利润率——如果你不与任何人沟通，直接租一块 GPU，他们要价大约每小时 6 美元。可要在数据中心购买并部署一块 NVIDIA GPU 的成本只有大约每小时 1.40 美元。这就是成本。那么云厂商赚多少利润算合理？也许 2 美元，也许 1.75 美元——这才是 NVIDIA 想看到的。NVIDIA 不希望所有利润都被亚马逊每小时 6 美元的定价吸走。当然，你可以与亚马逊谈判拿到更低价格，但问题依然棘手。所以 NVIDIA 扶持了许多其他云公司，从而压低价格。
(44:34)
But now they've made a big major misstep in my opinion. They acquired this company called Lepton, who doesn't own data centers themselves, but they built all the cloud software for reliability, for making it run easily, you know, Storm Kubernetes, all this kind of like scheduling stuff. This is stuff the clouds do, right? Which the big clouds do, the Neo clouds, the new cloud companies that NVIDIA's propping up do. But now NVIDIA's bought this company that does this software layer. And they're doing this thing called DGX Lepton, which is if anyone has a cloud with spare resources, GPUs just give them to us and we'll rent them for you. And we'll just give it to us bare metal, no software on them,
但在我看来，NVIDIA 现在犯了一个大错。他们收购了一家公司叫 Lepton。Lepton 自己不拥有数据中心，却开发了一整套云端可靠性和易用性软件，比如针对 Kubernetes 的调度等——这些本是云厂商自己做的，无论是传统云还是 NVIDIA 扶持的新兴云。而 NVIDIA 现在买下了这层软件，并推出 DGX Lepton：如果任何云厂商有闲置资源，把 GPU 裸机交给他们，他们会在上面加装 Lepton 软件并出租给终端用户。
(45:09)
and we'll add all of this Lepton software on top and rent it out to users. Now the cloud companies are really mad at this because it's like... You're directly competing with me, right? And in fact, I think NVIDIA is also going to put some of their own GPUs on Lepton, potentially, that they're installing themselves. But it's like, you propped us all up, but now you're making a competing cloud. So a lot of clouds are mad. They won't say it to NVIDIA because NVIDIA is sort of God, right? You know, like, you don't mess with God, right? What Jensen giveth, Jensen taketh. But like, they'll tell us. The clouds are really bad, right?
他们装好 Lepton 软件后再出租给客户。这让云厂商非常恼火，因为 NVIDIA 等于直接与他们竞争。事实上，我认为 NVIDIA 还会把自家的 GPU 也放到 Lepton 上出租——由他们自己部署。问题在于：NVIDIA 一边扶持我们，一边又办起竞争性云服务。很多云厂商因此生气，但又不敢当面指责 NVIDIA——毕竟 NVIDIA 如同“神”一般，黄仁勋赐予，也可收回——于是他们只是私下向我们抱怨，情绪非常糟糕。
(45:42)
And so, so, you know, there's this like aspect and so there's some cloud companies that are turning to AMD, maybe partially out of AMD paying them partially out of AMD being in being them being mad at Nvidia, but like, Some of these cloud companies are now buying AMD GPUs. And then there's this third thing that AMD is doing, which is they're taking, they're doing sort of what everyone accused, I don't know if you've seen this Corrive NVIDIA fraud nonsense.
因此，一些云公司开始转向 AMD GPU——部分因为 AMD 给予补贴，部分因为他们对 NVIDIA 生气。现在确实有云厂商在采购 AMD GPU。AMD 还做了第三件事，有点像外界指责的“CoreWeave 与 NVIDIA 虚假交易”那种做法。
(46:06) Matthew Berman:
Yeah, sending revenue back and forth.
是啊，把收入来回倒腾。
(46:09) Dylan Patel:
Because NVIDIA pays them. It's like, yeah, NVIDIA rented one cluster from them.
因为 NVIDIA 会向他们付费。是的，NVIDIA 的确从他们那里租过一个集群。
(46:12) Matthew Berman:
Yeah.
对。
(46:13) Dylan Patel:
Cool.
好吧。
(46:14) Matthew Berman:
Seems like business as usual.
看起来就是常规生意。
(46:16) Dylan Patel:
Well, like NVIDIA needs GPUs internally, right? They have to develop their software.
毕竟 NVIDIA 内部也需要 GPU，要开发他们自己的软件嘛。
(46:19) Matthew Berman:
I mean, there's like a little something there, but it seems like, yeah, exactly.
我是说，这里面确实有点门道，但看起来，就是这样，没错。
(46:22) Dylan Patel:
They invested like a tiny amount of money, right? But whatever. It's like so irrelevant, right? But AMD is actually doing this and like taking it to overdrive. They're getting clusters at Oracle and Amazon and Crusoe and DigitalOcean and TensorWave and they're renting GPUs back from them. So they're selling them GPUs and renting them back. Like it's one thing if core we've like buys NVIDIA GPUs and a small portion of them go to NVIDIA But the vast majority are going to Microsoft for open AI.
他们只投了极少量的钱，对吧？但不管怎样，这其实无关紧要。可 AMD 真正在这么干，而且玩得更猛——他们把 GPU 集群卖给 Oracle、Amazon、Crusoe、DigitalOcean、TensorWave，然后再从这些公司回租 GPU。也就是说，他们先把 GPU 卖出去再租回来。比如 CoreWeave 买了 NVIDIA GPU，少量回租给 NVIDIA，但绝大多数都流向了微软给 OpenAI 用。
(46:52) Matthew Berman:
You're not calling this like accounting trickery though, right?
你不会把这称作“财务把戏”吧？
(46:55) Dylan Patel:
It's not accounting trickery It's like perfectly like the accounting is legal. Obviously you can sell someone something and then rent something from them Like NVIDIA's done this too.
这算不上财务把戏——这种会计处理完全合法。你当然可以把东西卖给别人，然后再向他们租同样的东西；NVIDIA 也做过类似操作。
(47:03) Matthew Berman:
They're almost funding the investment.
他们几乎是在为对方的投资提供资金。
(47:05) Dylan Patel:
Right, right, exactly. And so this is sort of like, in the case of like Oracle and Amazon, it's like, hey, buy our GPUs. We'll rent them back. You'll see that it's great. And you can actually have some, some of them you won't, we won't rent back. Some of them you'll try and rent to your customers. So that drums up interest. And if it works out, you can buy more, right? This is their, this is their reasoning, right? Or for the Neo clouds, it's like, well, you guys are only buying Nvidia stuff. Why don't you buy our stuff? Here's, here's a contract to get you comfortable. And yeah, here's a portion that you can rent out to other people, right? Like, it's like, this makes sense.
对，对，完全正确。以 Oracle 和 Amazon 为例：他们说“来买我们的 GPU，我们再租回来给你看效果。有些 GPU 我们不回租，你们可以自己租给客户，帮你们打开市场；如果运转顺利，你们以后可以买更多。” 这是他们的逻辑。对于那些新兴云厂商，他们则说：“你们只买 NVIDIA 的产品，不如试试我们的。给你们一份合同让你们放心，还有一部分 GPU 你们可以租给别人。” 这听起来就挺合理。
(47:35)
It's to some extent, but it's also to some extent like a lot of the sales are just AMD buying them back, but it's like this fosters really good, it's like really good relations, right? Now, TensorWave and Crusoe who are clouds, they're like, I love AMD, right? Because they're renting GPUs from me and they're selling them to me and they're renting them back and I make a profit off of this and now I can reinvest this in more AMD GPUs or I have a chunk of AMD GPUs I can rent to other people. These clouds are like, well, fuck, NVIDIA is trying to compete with me anyways. What else am I going to do? So it's like an interesting confluence. I think AMD will do well.
某种程度上，这意味着不少销量其实是 AMD 自己回购 GPU，但这反而促成了极佳的合作关系。像 TensorWave、Crusoe 这些云厂商就很喜欢 AMD，因为 AMD 又卖又租，他们中间赚差价，再用赚到的钱买更多 AMD GPU，或者把手头的 AMD GPU 租给别人。这些云厂商想想：反正 NVIDIA 也要跟我竞争，我还能怎么办？所以这形成了有趣的合流，我认为 AMD 表现会不错。
(48:08)
I don't think they'll surge in market share, but I think they'll do okay. I think they'll sell billions of dollars of chips.
我不认为他们的市场份额会暴涨，但表现应该不错，能卖出数十亿美元的芯片。
(48:15) Matthew Berman:
But if you're advising a company on which chipset to invest in for the foreseeable future, you're saying NVIDIA?
那如果你要给一家公司建议，在可预见的未来投资哪家芯片，你会说选 NVIDIA 吗？
(48:20) Dylan Patel:
Depends on the price you can get from AMD. I think there's a price where it makes sense to use AMD and I think AMD will sometimes offer that price to people. Meta uses AMD a good bit. They also use a lot of NVIDIA. They use a good bit of AMD. For certain workloads where AMD is actually better, when I have the software talent and AMD is giving me a ridiculous price. Yeah, you should do it and that's why Meta does it, right? But like in a lot of workloads, Meta still goes to NVIDIA because NVIDIA is the best.
这取决于 AMD 给你的报价。我认为当价格合适时，用 AMD 是有意义的，AMD 有时确实会给出这种价格。Meta 就用不少 AMD，同样也大量使用 NVIDIA。对于某些 AMD 表现更优、且我们有软件人才、AMD 又报出极低价格的工作负载，你当然该用 AMD——这也是 Meta 的做法。但在很多工作负载上，Meta 仍然选择 NVIDIA，因为 NVIDIA 最强。
XAI's Grok and the Future of Labor in an AI-Driven World
由 AI 驱动的世界里，XAI 的 Grok 以及劳动力的未来
(00:48:47)
The discussion shifts to XAI and their Grok model, with Patel acknowledging Elon Musk's marketing prowess. He finds Grok useful for deep research and accessing unfiltered data, particularly on current events. Patel then addresses concerns about job displacement due to AI, arguing that aging populations and a trend towards reduced working hours suggest AI could simply enable people to work less. He expresses excitement about robotics, which could automate tasks that are difficult for AI but undesirable for humans.
讨论转向 XAI 及其 Grok 模型，Patel 认可了埃隆·马斯克的营销能力。他认为 Grok 在深入研究和获取未经过滤的实时数据方面很有用。随后 Patel 回应了对 AI 导致就业流失的担忧，指出人口老龄化和工作时间减少的趋势表明 AI 可能只是让人们工作更少。他对机器人技术感到兴奋，因为它能自动化那些 AI 难以完成且人类不愿从事的任务。
(48:47) Matthew Berman:
I want to talk about XAI. I want to talk about Grok 3.5. Obviously, like at least publicly, there's not a ton of information about it. Elon Musk has said it's by far the smartest AI on the planet and it's going to operate on first principles. Is this all puffery? Have they actually discovered something new and unique? Specifically, he asked for divisive but true facts. There's a lot of things that he's doing where it just seems like either he discovered something new or it is pure puffery. What's your take on what's going on?
我想谈谈 XAI，也想谈谈 Grok 3.5。显然，至少在公开层面，我们对它的信息并不多。埃隆·马斯克声称这是地球上最聪明的 AI，将基于第一性原理运作。这是不是纯粹夸大？他们真的发现了全新的独特东西吗？他特别要求提供“有分歧但真实”的事实。他的很多做法看起来不是发现新东西，就是纯粹吹嘘。你怎么看？
(49:17) Dylan Patel:
I think Elon is a fantastic engineer, engineering manager, but I also think he's a fantastic marketer. I don't know what the new model will look like. I've heard it's good, but you've heard it's good. Everyone's heard it's good, right? So, you know, we'll see what it comes out. When Grok 3 came out, I was pleasantly surprised because I was expecting it to be a little bit worse, but it was actually better than I expected.
我认为埃隆既是出色的工程师和工程管理者，也是一流的营销高手。我不知道新模型会是什么样子。我听说它很好，你也听说它很好，大家都听说它很好，对吧？等它发布我们就知道了。Grok 3 发布时我本以为它会差一些，但结果超出了预期。
(49:41) Matthew Berman:
Do you use Grok 3 day to day?
你每天都用 Grok 3 吗？
(49:43) Dylan Patel:
Day to day, I don't, but there's certain queries I do send to it.
并不是每天用，但有些查询我会发给它。
(49:47) Matthew Berman:
What, if you don't mind me asking?
方便的话能举例吗？
(49:49) Dylan Patel:
Their deep research is much faster than OpenAI, so I use that sometimes. And then sometimes, like, models are just like pansies about, like, giving me data, right, that I want, right? Like, it's like, you know, it's like, sometimes I'm just curious, like, what is the, I like human geography, like the history of humanity, how geography, politics, history, you know, resources, like, interact with each other. And so I like to know about demographics and things like that as well, right? It's just interesting stuff. You know, the town I grew up in, right, is like, it's like on the Bible Belt, it's half black, half white, 10,000 population. But like, the one of the ways I describe it to people is like,
他们做深度研究的速度远快于 OpenAI，所以我偶尔会用。此外有时候其他模型在提供我需要的数据时显得很怂。有时我会好奇，比如……我喜欢人文地理，研究人类历史中地理、政治、历史、资源如何相互作用。我也想了解人口统计等信息，这很有趣。我长大的那个城镇位于“圣经地带”，人口一万，一半黑人一半白人。我向别人描述时会说，
(50:24) well, yeah, it's like where the, where the floodplains used to be in the ocean receded, it's extremely fertile land. And that's when in Georgia, when white settlers settled everywhere randomly happened to one of the more fertile areas, so they were able to have better harvests, and they were able to purchase slaves. And that's why it's a higher black percentage than most of the state. And it's like, that's an insane thing to say. But I like to like, reason about like sort of human geography like this. And like Grok is okay with doing that. Right. So sometimes like, you know, it lets me under and obviously, like, savory is bad.
这里原本是海洋退去后的泛滥平原，土地极其肥沃。因此在佐治亚州，白人定居时碰巧占据了更肥沃的区域，收成更好，从而有能力购买奴隶，这也导致黑人比例高于该州大部分地区。这听起来很疯狂，但我喜欢用这种方式来推理人文地理，而 Grok 对此也能接受。有时它能让我理解——当然奴隶制度很糟糕。
(50:51)
And like, that's, you know, it's like, but like, just like to understand, like, or like, hey, what, you know, oh, like, invasions from like, the step. To Europe were not because they just want to invade, it's because it was becoming more arid. They were forced off their land. These sorts of things are cool and interesting, or economic history. Why did Standard Oil win versus this other oil company before it got to monopoly levels? These sorts of things are just interesting to learn, but other models will start, if it's the Standard Oil thing, it'll be like, oh, it's a union buster, blah, blah, blah. It's like, no, just tell me what actually happened.
再比如，来自草原地带对欧洲的入侵，并非单纯想侵略，而是因为气候变得干旱，被迫离开家园。这些内容很有趣。又或者经济史——为什么标准石油在形成垄断前就击败了其他石油公司？这些都值得了解。但其他模型一谈到标准石油就会开始说“破坏工会”之类的套话，而我只是想知道事实。
(51:23)
I think Grok can sometimes, Get through the bullshit, but it's also not the best model.
我觉得 Grok 有时能拨开那些废话，但它也并非最优秀的模型。
(51:28) Matthew Berman:
So my daily, the model I go to the most is either O3 or Claude 4. You're using O3 day to day even though there’s, you know, it takes so much time to actually get your response back.
在我的日常使用中，我最常用的模型是 O3 或 Claude 4。你每天都用 O3，虽然你也知道它实际响应回来要花很长时间。
(51:41) Dylan Patel:
It depends on the topic, but yeah, I think a lot of times I’m okay with waiting. A lot of times I’m not. That’s why I use Claude. I use Gemini in work, right? So we feed a lot of permits and regulatory filings through Gemini. We feed a lot of like, it’s like really good at long context, right?
And document analysis and retrieval. So we feed a lot of stuff through Gemini in a workplace manner, but like I’m talking about pull out my phone. I want to know something mid-conversation or whatever. It’s a different model. So Grok, they have a lot of compute. It’s really concentrated. They have a lot of great researchers. They’ve got like 200,000 GPUs already up. And they’ve purchased a new factory in Memphis,
这取决于话题，但很多时候我愿意等，也有很多时候我不想等，所以我才用 Claude。在工作中我用 Gemini，把大量许可证和监管文件输进去。它在长上下文处理、文档分析和检索方面真的很强。所以在职场场景里我们把很多东西丢给 Gemini，但如果是我掏出手机、想在对话中随时查点什么，那就是另一套模型。至于 Grok，他们的算力非常集中，研究人员也很强，手里大概已经有 20 万块 GPU。他们还在孟菲斯买了一座新工厂，
(52:23)
and they’re building out a new data center, and there’s the craziness they did with mobile generators. Well, now they just bought a power plant from overseas and are shipping it to the US, because they couldn’t get a power plant, a new one in time. So they’re doing all this crazy shit to get the compute. They’ve got good researchers. Clearly, the models are good, and Elon’s hyping them up. Maybe it’ll be great. Maybe it’ll be good. Will it be open-AI level, or will it be just slightly behind?
正在建设一个新的数据中心，还搞过移动发电机的疯狂操作。现在他们干脆从海外买了一座发电厂运到美国，因为来不及批新电厂。为了算力他们真是无所不用其极。研究团队也很棒，模型显然不差，埃隆又在大力造势，也许会非常出色，也许只是不错。它会达到 OpenAI 级别，还是稍微落后一点呢？
(52:47) Matthew Berman:
Are they doing something fundamentally different? He specifically said rewrite the corpus of human knowledge because too much garbage is in the current foundation models. Obviously he has the X data, which is insane, but it’s also really low quality, so it’s hard to get through.
他们是不是在做根本性的不同尝试？他明确说要重写人类知识语料，因为现有基础模型里垃圾太多。显然他手里有 X 的数据量，这太夸张了，但质量也很低，很难清洗。
(53:07) Dylan Patel:
Oh, that’s another area where I use Grok sometimes, current events.
哦，这也是我偶尔用 Grok 的一个领域——时事热点。
(53:11) Matthew Berman:
Summarizing or giving you the context.
让它总结，或者给你背景信息？
(53:13) Dylan Patel:
When the missiles were happening in Israel and Iran and all this war stuff, you can actually ask Grok and it tells you exactly what’s happening way better than a Google search will or even a Gemini query or OpenAI query because it’s got access to all this info.
以色列和伊朗之间爆发导弹袭击、各种战争新闻时，你直接问 Grok ，它给出的实况比 Google 搜索，甚至比 Gemini 或 OpenAI 查询都准确得多，因为它能访问到所有这些信息。
(53:26) Matthew Berman:
So are they doing anything different?
那么他们真的在做什么不同的事情吗？
(53:29) Dylan Patel:
I think step function different wise, I don't think anyone is like, you know, like everyone likes to think they're doing different things, but generally people are doing the same thing. They're pre-training large transformers and they're doing RL on top, mostly in verifiable domains, although they're researching how to do unverifiable domains. It's like, oh yeah, they're making environments for the model to play in, but they're mostly code and math, but now they're getting into computer use and all these other things. It's like everyone's doing generally the same stuff, but there's such a, it's also such a challenging problem.
如果说有质的飞跃，我并不认为有什么团队真的截然不同。大家都觉得自己在做新东西，但总体上都是一样的：先对大型 Transformer 进行预训练，再在其上做强化学习，主要针对可验证领域，虽然也在研究如何处理不可验证领域。比如，他们为模型构建可交互的环境，但大多仍是代码和数学；现在开始涉足计算机使用等其他方向。本质上大家做的事情差不多，只是这个问题本身极具挑战性。
There's many directions to go with it, but I think generally everyone's going the same approach. Even SSI is not, I imagine SSI is doing some different stuff, but I don't even think they're doing that much differently than what I just said.
针对这个课题可以走很多路线，但总体而言大家的方法一致。即使是 SSI，也许他们做了一些与众不同的尝试，但我认为与我刚才描述的并没有太大差别。
(54:16) Matthew Berman:
I have kind of two different topics I want to let you maybe choose. Economics, labor, so I want to talk about the 50% of white-collar jobs could disappear. I know you probably read about that. Or non-verifiable rewards, which obviously that's maybe more recent, more on your mind. Do you have any preference?
我这儿有两个话题让你选：一个是经济与劳动力，比如有报道说 50% 的白领工作可能消失；另一个是不可验证奖励，这可能更近期、也更符合你的关注。你想聊哪个？
(54:33) Dylan Patel:
I think the prior is more, I mean, maybe the latter is more interesting for your audience. I'm not sure. But the prior is really interesting, right, in terms of like, everyone's worried about massive job loss, right? Or at least some people are in the AI world. But then the flip side is that like, you know, populations are aging really rapidly. And generally, people work less than ever before, right? Like we make fun of Europeans because they work really a lot less. But like the average amount of hours worked 50 years ago was way higher, right? And 100 years ago is even higher than that. And the amount of leisure time was way less. And the size of everyone's home is way larger.
我觉得前一个话题——也许后一个对你的听众更有趣，我不确定——但前一个确实很有意思。大家担心会出现大规模失业，至少 AI 圈里有人这么想。然而另一方面，全球人口正迅速老龄化，而且总体来看，人们的工作时间比以往任何时候都少。我们常拿欧洲人开玩笑，说他们工作时长更低，但 50 年前人均工作时间远高于现在，100 年前更是高得多，闲暇时间少得多，住房面积也小得多。
The Impact of AI on the Job Market, Open Source vs. Closed Source, and Predictions for Super Intelligence
人工智能对就业市场的影响，开源与闭源的超级智能的来源和预测
(00:55:15)
Patel predicts that AI will automate a significant portion of jobs, but the deployment will take time. He observes that the junior software engineering market is already struggling. While acknowledging that AI can increase productivity and enable companies to tackle more problems, he questions where junior engineers will fit in. Patel believes that open source AI will struggle to compete with closed source, particularly if China dominates the field. He expresses hope for a more distributed AI landscape. Finally, when asked to bet on a company to achieve super intelligence first, Patel chooses OpenAI, followed by Anthropic, and then a toss-up between Google, XAI, and Meta.
帕特尔预测，人工智能将自动化大量工作岗位，但真正部署需要时间。他指出初级软件工程师市场已显疲态。虽然他承认 AI 能提升生产率并让企业解决更多问题，但也质疑初级工程师未来的定位。帕特尔认为，在中国可能主导该领域的情况下，开源 AI 将难以与闭源竞争。他希望 AI 生态能更加分散。最后，当被问及哪家公司最可能率先实现超级智能时，帕特尔把赌注押在 OpenAI，其次是 Anthropic，谷歌、XAI 与 Meta 并列第三。
(55:09) Dylan Patel:
And like the food security is way better. It's like every metric were way better than 50 years ago or 100 years ago. And AI should just enable us to work even less, right? Now, is it gonna be like, there's gonna be psychos like myself and probably yourself as well that work way too much. And then there's gonna be like, Normal people who work way less, right? And obviously the distribution of resources is the challenge though, right? I think that's the big thing. That's why I'm super excited about robotics as well because robotics is like a A lot of jobs that are easier to automate are hardest to automate are robotics-influenced. The stuff people want to do is sit on a computer and be creative,
现在的粮食安全要好得多，各项指标都比 50 年前或 100 年前大幅提升。AI 理应让我们工作更少。未来可能会出现像我、也许还有你这样工作过度的疯子，也会有普通人工作得少得多。当然，资源分配才是真正的难题。我对机器人技术非常兴奋，原因就在于：许多容易自动化却最难真正落地的工作都与机器人相关。人们真正想做的是坐在电脑前发挥创造力，
(55:47)
but actually that's one of the markets that's been nuked the hardest is freelance graphics designers. What's the market that's not touched is picking fruit. That's the shit that people don't want to do.
但实际上受冲击最大的市场之一就是自由职业的平面设计师；几乎毫发未损的则是摘水果这种没人愿意干的工作。
(55:59) Matthew Berman:
It still seems like that's way in the future, even though robotics has been progressing at an insane rate, but it does seem like it's pretty far in the future. Okay, but do you foresee as human productivity increases like crazy, certainly a large swath of tasks will be automated. Do you think humans are going to be managing AI in the future or are we going to be reviewing the output of AI or some mixture of in between?
尽管机器人技术发展飞快，但那一切似乎仍遥不可及。随着人类生产力大幅提升，大量任务必定会被自动化。你认为未来人类会管理 AI，还是只负责审核 AI 的输出，或是介于两者之间？
(56:26) Dylan Patel:
Right now we're in the transition from using models on a chat basis To a longer horizon basis, you know, I mentioned I used O3 a lot because actually there's a lot of longer horizon tasks. Now these longer horizon tasks are 20-30 seconds. In deep research is a dozen minutes, right? Dozens of minutes. Over time, these like interactions with AI will become, obviously there will be an AI assistant that I'm just talking to all the time or will be telling me stuff that is noteworthy. But there will also be just long horizon tasks of like, AI is going to be doing stuff for hours. It's been days before coming back for me to review. And then eventually there just won't be humans in the loop, right?
我们正从“聊天级”模型过渡到“长程任务”模型。我常用 O3，因为很多任务需要更长时间：现在的长程任务大约 20–30 秒，深度研究可能十几分钟甚至几十分钟。随着时间推移，人机互动将演变为：要么我随时与 AI 助手对话，要么它主动把重要信息告诉我；同时还会出现 AI 连续运行数小时、甚至数天后再由我审核的超长任务。最终，人类会完全被移出流程。
(57:02) Matthew Berman:
And eventually like- I don't believe that. And what timeline are you thinking?
最终真的会这样吗？我不太相信。你预期的时间表是？
(57:06) Dylan Patel:
I think timeline questions are ridiculous. I'm generally more pessimistic on timelines. It's not, I don't think this decade for people to, for like 20% of jobs to be automated, I think it's like not, it's like maybe the end of this decade, maybe the beginning of next, for 20% of jobs to be automated, right? There's people saying AGI in 2027.
我觉得讨论时间表很荒谬，我对时间表总体更悲观。本十年内要实现 20% 的岗位自动化，我认为不太可能；也许要到这十年末、下一个十年初才能达到 20%。有人说 2027 年就有 AGI，这太乐观了。
(57:25) Matthew Berman:
But even reaching the tech doesn't mean the implementation is going to happen at that moment either, right? It's going to take years before we actually are able to deploy it in the field.
即便技术取得突破，也不意味着能立刻投入使用。真正部署到实际场景可能还需要数年时间。
(57:34) Dylan Patel:
I think deployment will be really fast. You already see the junior software engineering market is nuked. No one can get a job. You can already see the usage of AI and software development is skyrocketing. And we're not even at automated software development yet. We're just at like.
我认为部署速度会非常快。你已经看到初级软件工程师市场被“核平”，没人能找到工作。AI 在软件开发中的使用率正飙升，而我们甚至还未真正进入自动化开发阶段，我们只是……
(57:49) Matthew Berman:
Our company is going to choose to do more things. Are they going to choose to tackle more problems?
那公司是否会选择做更多事情、解决更多问题？
(57:53) Dylan Patel:
Yes.
会。
(57:54) Matthew Berman:
So then how do those junior engineers get into the market to begin with then? I spoke to Aaron Levy yesterday and he was like, no, as soon as a team tells me, look how productive we are, where do you think I'm going to invest? I'm going to invest back in that team. We're going to grow that team. Where is the place for junior engineers then?
那这些初级工程师最初如何进入市场？我昨天跟 Aaron Levy 聊，他说一旦某个团队展示出高生产力，他就会继续投资并扩大该团队。那么初级工程师的位置在哪？
(58:12) Dylan Patel:
I think that's nice and I agree. I myself, my company does a bunch of stuff, but due to the use of AI, we can do a whole lot more stuff and that makes us more productive. We're able to out-compete the old firms that don't do stuff in the consulting and data space. I still have basically doubled the size of the firm in the last year to 32 now, 33. How many junior software developers am I going to hire? It's like, no, it's like the junior software developer I have, we just cheered her on because she just did 50 commits last week. That's what used to take many more people. It's like how much stuff like there's obviously a lot of software for us to build.
这话挺好听，我也同意。我们公司做了很多项目，借助 AI 可以做得更多，生产力更高，能击败在咨询和数据领域不作为的老牌公司。过去一年我把团队规模翻倍到 32、33 人。我会招聘多少初级开发者？几乎不会。我现有那位初级开发者上周提交了 50 次代码，以前得好几个人才能做到。虽然我们仍有大量软件要写，但……
(58:47)
But it's like you know how many people can we like actually like you know add right? And it's like wouldn't I rather have like a senior person that's like commanding a bunch of AIs rather than like a junior person. So it's like sort of like it's challenging. At the same time like hiring young people because they can quickly adapt to the new AI tools. It's a balancing act. I think it's I don't know where the junior software developers would go because I get people pinging me on Twitter and LinkedIn all the time, like, you have a job for me? It's like, no, I don't really. Or sometimes I do, right? But it's tough.
问题是我们实际还能再招多少人？我宁愿要能指挥一堆 AI 的高级员工，也不想要初级员工。这很难权衡；同时年轻人适应新 AI 工具快，聘用他们也有好处。这是一场平衡游戏。我不知道初级开发者该去哪；Twitter 和 LinkedIn 上总有人私信问我有无职位，大多时候我只能说没有；偶尔有，但也很难。
(59:25)
And I don't see the major tech companies hiring junior software developers that much, right? It's just a fact, right? And that's why the market is really bad.
我也没看到大型科技公司再大规模招聘初级软件开发者，这就是事实，因此市场状况非常糟糕。
(59:33) Matthew Berman:
So they had to just self-skill up on their own, come with better skills.
因此，他们只能自我提升技能，带着更强的本领再来。
(59:38) Dylan Patel:
Or just try and build stuff on their own and show that they're not a junior software developer but they can actually use these tools.
或者自己动手做项目，证明自己并非初级开发者，而是真能驾驭这些工具的人。
(59:43) Matthew Berman:
That's not for everybody, though.
但这并不适合所有人。
(59:44) Dylan Patel:
Yeah, it's not. A lot of people just need a job. They don't need to self-start.
确实如此。很多人只是需要一份工作，并不想自我驱动。
(59:48) Matthew Berman:
They don't want to be founders, for sure. They don't want to be solo builders. Even if you're not a founder, they want to have that security.
他们当然不想当创始人，也不想单枪匹马地创业；即便不当创始人，他们也需要安全感。
(59:55) Dylan Patel:
I mean, that's been a problem for me. When I started hiring people, some people need a lot of direction, and I don't have direction to give. I'm like, I need self-starters. Now there's people who can do that in the firm, but it's tough to give people. Some people just need direction and need more hand-holding, at least initially.
这对我来说也是难题。最初招人时，一些人需要大量指导，而我却给不了那么多。我需要自我驱动型的人。公司里现在确实有这样的人，但要给予他人持续指导很难；有些人天生就需要更多方向，尤其在刚入职时。
(1:00:10) Matthew Berman:
Open source versus closed source.
开源与闭源。
(1:00:13) Dylan Patel:
The US is going to lose in open source unless Meta gets dramatically better, which they are. I think with a lot of the talent they're hiring, I think Sam is wrong that they're not getting any top researchers. Sam Altman, I think they are. There are some top researchers I know for sure are going there. Maybe not the first people they offered, like the ones that have the highest, highest profile, but there are still some top researchers going there. China is open sourcing stuff only because they're behind. The moment they're ahead, they will stop open sourcing stuff. And at the end of the day, closed source will win. Unfortunately, closed source will win.
在开源领域，美国会输，除非 Meta 显著提升——而他们的确在进步。我认为他们招聘了大量人才；Sam Altman 说 Meta 招不到顶尖研究员，这是不对的。我确定有一些顶尖研究员正加入 Meta。也许不是最初邀请的那些最具声望的人，但仍有顶尖研究员在前往。中国之所以开源，只是因为他们落后；一旦领先，就会停止开源。最终，闭源将取胜——遗憾的是，闭源会赢。
(1:00:46)
My only hope is that it's not just like two or three closed source AIs that dominate human GDP, right? Or types of models or companies, right? But rather it's like more distributed than that, but it might not be, right?
我唯一的希望是，最终主导全球 GDP 的不会只有两三个闭源 AI，或者少数几类模型、公司；而是一个更加分散的格局。但现实可能并非如此。
(1:01:00) Matthew Berman:
Meta, Google, OpenAI, Microsoft, Tesla, whoever else, you had to pick one company to bet on. Super Intelligence reaching it first. Who are you picking and why?
如果要从 Meta、Google、OpenAI、Microsoft、Tesla 等公司中选一家下注，赌它们最先实现超级智能，你会选哪家，为什么？
(1:01:07) Dylan Patel:
OpenAI. They're the first to every major breakthrough. Even reasoning, they were the first two. And I don't think reasoning alone will take us to the next generation. So there's gonna be something else. Anthropic second.
OpenAI。我觉得每一次重大突破都是他们率先取得。连“推理能力”也是他们最早做到的之一。我不认为仅靠推理就能迈向下一代，还会有其他关键进展。Anthropic 排第二。
(1:01:22) Matthew Berman:
They're so conservative though. They're so conservative at Anthropic in terms of what they release, what they publish, what they focus on, so much safety.
可 Anthropic 太保守了——无论发布什么、公开什么、专注什么，都把安全性放得很重。
(1:01:29) Dylan Patel:
I think they're conservative as a weekend a lot. I think they're a lot less conservative than they used to be. The process for launching Claude 4, as far as I understand, was much simpler and easier than the process for launching Claude 3. Whether it's that they're hiring a lot more normies, which they are, or like they recognize that others are just going to release stuff anyways and they should have theirs or whatever it is. I think Anthropic is like loosening up a bit. I think they just have really good people though. And then sort of like third is going to be like it's actually a toss-up between Google XAI and Meta now.
我觉得他们“周末模式”时才显得保守，现在比过去开放多了。就我所知，推出 Claude 4 的流程比 Claude 3 要简单得多。也许是因为他们招了更多“普通”员工，也可能是意识到反正别人都会发布新东西，自己也得跟上。总之 Anthropic 正在逐渐放松。我认为他们的人才确实很强。至于第三名，现在其实是 Google、XAI 和 Meta 打成平手。
(1:02:04)
I think Meta will get enough good people that they'll actually be competitive too.
我觉得 Meta 会吸引到足够多的优秀人才，届时也能形成竞争力。
(1:02:08) Matthew Berman:
Dylan, thank you so much for chatting with me.
Dylan，非常感谢与你交流。
(1:02:10) Dylan Patel:
Thanks for having me.
谢谢邀请。
(1:02:10) Matthew Berman:
I appreciate it. This is awesome, man.
非常感激，这次对话太棒了。
(1:02:12) Dylan Patel:
Yeah, very fun.
是啊，非常有趣。
(1:02:13) Matthew Berman:
Yeah, you can talk about anything, huh?
没错，你什么话题都能聊，是吧？
(1:02:15) Dylan Patel:
Maybe, maybe.
也许吧，也许吧。