Dylan Patel: AI in War, Jobs are Cooked, Chinese Hacking, Microsoft Cope, and Super Intelligence

[00:00] Now with like clock code, I think we move 10 times the speed of any of our competitors. All the executives, all these major companies watch podcasts like yours. They listen to these things because it's like, well, this is how I listen to the people that like matter. >> [music] >> Hey, if a nuclear missile's heading from China to America, we can use AI to stop it. And Dario was like, well, you can call us. Well, I'm sure we can figure something out. And it's like, like this is just the dumbest response you could ever come up with. I don't know how principled Sam is, but he's certainly one to take advantage of a good crisis.
[00:30] Wait, I actually think like UBI's like perfectly fine. Which I think is like crazy. Nowadays, people in the 90th percentile think they're middle class. And people in the 50th percentile don't think they're middle class. A buffet of dopamine is less happiness. >> I actually didn't know that. >> Open claw's pretty freaking insane. Even without AI going bad and rogue, we know people are bad. Really? Cooked. >> [laughter] >> Cooked. Dylan, round two. Really appreciate it. >> Yeah, I'm excited for this. >> Very nice office you got here. Yeah,
[01:00] thank you so much. Yeah. >> Yeah, we've got a fantastic office that we just got moved into. >> So, I want to think back to the last time we talked, which is about 18 months 18 8 months ago. It really does seem like a long time. >> Bro, 8 months ago is 3 years in AI land. >> Might as Yeah, yeah, yeah. Might as well be. All right, so you made a bunch of predictions. I want to go over those first. >> Yeah, yeah, yeah, [laughter] yeah. Do you got a scorecard for me? Uh okay, you said GPT 4.5 too slow, too expensive. We'll come back to these in a sec. I'll go through them one by one. Scale AI cooked. Junior dev market nuked.
[01:30] Open AI is your pick to reach super intelligence first. Now we're 8 months later, and I want to hear I want to hear the updates. Let's go through them one by one. So, first >> bad. GPT 4.5, you may have just absolutely nailed that one. What went wrong with GPT 4.5? >> Well, 4.5 failed cuz it didn't have enough data. Also, it was just very complicated and difficult on a scaling perspective, infrastructure-wise. Um and there were tons of problems and challenges there. Yeah. Yeah. >> Yeah. How you feeling about that now? I mean, I guess there's nothing to say. >> even use it. You can't even access it. No API. It was a good model, though.
[02:00] Next, Scale AI cooked. What does the Scale AI acquisition actually give Meta? First, let's start there. Yeah, so I think um you know, for one, Scale AI is like it's kind of cooked right now. As a company. As a company, because like >> canceling their Yeah, Google Google's backing out. How you feeling about that? Obviously, Alexander Wang running the Meta super intelligence department division. Still feeling like Scale AI is kind of the orphan child. Yeah, I mean, like they've had a good number of departures. They've had a good number of uh folks
[02:33] turning off of them or turning to other providers, right? Scale AI is not really picked up a lot on the environment space, right? When you look at the data labeling companies, there were a lot less than there are the environment companies. Um and in some sense, it's sort of the same end market, right? Although environments will be bigger and so on and so forth. There's like 30 environment RL environment companies. There's There's you know, so so Scale is has not really picked the ball on that. Um the leaders in that space are different folks. I think they've really missed missed that ball. But I mean,
[03:03] their business is doing fine. It's just like >> the point of the acquisition. >> Yeah, yeah. I mean, who cares, right? It's It's Alexander Wang. We got We got to see how Meta's avocado and all that stuff comes out, right? That's the more relevant thing. That's why they acquired. Yeah, I mean, it's uh lucky for them, they're sitting out of the frontier race at this very moment. But we'll get to that in a minute. Um Junior dev market nuked. That's uh It was a strong prediction 8 months ago. You already see the junior software engineering market is nuked. A lot has changed. What do you What do you think
[03:33] about that prediction at the time? >> Um I mean, it's it's it's true, right? Um you know, you you look around and this the the fresh grads are it's even harder for them to get jobs. You look at like quad code spin, right? It's freaking nuts, right? 19 billion of revenue for Anthropic now. All of that is you know, at some multiple is code, right? And at some multiple is like, okay, if you're spending $20 billion industry is on code, um some multiple of that is the productivity that that gains in value
[04:04] you're getting, right? So, you know, is it 4x more productivity versus how much you spend? Cuz there has to be some premium over, you know, value over the premium that you pay um for switching, right? >> Yeah. And I think I think it is, right? I think for us like right like our our spend late last year was something like, you know, 50k a month, right? Whatever. And it was like in the hundreds of thousands of dollars in run rate annually. Um then 4.6 came out, then 4.6 fast came out, 4.6 fast with 1 million contacts.
[04:36] That's like 12x the cost of 4.6 period, right? I have people who are not software developers. Literally we we we have we you know, like one day um my head of ops is like, guys, our run rate of spend is $6 million. This is not sustained And this was Super Bowl Sunday, by the way. This is not sustainable, right? And it was because one of one one one one of my engineers spent like $8,000. He's Canadian, so whatever. But on on on on Claude code, right? And it's like, oh right? Like is this And then And then And then I like asked him what he built, and he's
[05:06] like he he explained everything he built, and I'm like, oh, this is totally fine. Like I don't mind. Yeah. The spend is fine. And then And then now like you've got adoption from more and more people, right? I've got Jeremy who does our data center modeling, right? He he he works on the data center team. He leads the data center team. He tracks all the data centers around the world. He's not been a programmer, right? We've had other people doing a lot of the like back end and all these sorts of things like the the you know, vision model or like contractors and stuff or like going through and scripting all the permits and filings. He just started using Claude code, and he's building his own tools, and he's like, I'm not coding.
[05:36] I'm just telling you what to do, and it's doing these things, and his daily spend now on 4.6 fast 1 million contacts is $5,000 a day. >> [laughter] >> Jesus. And I look at the productivity, it's fine, right? Like so So I think I think it's beyond just the junior developers, it's like all, you know, Cloud Code is not a coding thing, it's like for all people, right? >> Yeah, I mean, ever since Open Claw became popular just like 7 weeks ago, I think I've spent close to 7 billion tokens on it. Uh
[06:06] in in luckily, friends at Cursor kind of supply me with the with the good tokens and What have you made? Uh a ton of workflows. I I Okay, so Open Claw now is a first-class citizen in my company. It has its own uh workspace, email address, drive. It is reading all of my emails, and yes, I have lots of prompt injection defense there. Uh and it's doing outbound sales, it's uh triaging inbound sales, it's it's really cool. But of course, it took so much iteration actually get there. So that's where those 7 billion tokens
[06:36] comes from. Mhm. Mhm. I I joke internally that like my company will not exist in 2 to 3 years because because AI will make it impossible. And and that's like the driving force. So like we have to adopt AI faster, faster, faster, faster than anyone else. And so we keep like taking market share from other people in like various areas. Um but I just got to like a shiver. I'm like, we're behind the eight ball. >> Can I tell you what just happened last night? So I was putting my son to sleep, he wasn't going to sleep. I was like, "Hey, you got to go to sleep. I'm interviewing somebody cool tomorrow. Got to get good night's sleep." He's like,
[07:06] "Oh, what Who are you interviewing?" I was like, "Dylan Patel, he runs this company, they know a lot about AI chips, and other people pay them to tell them about AI chips." And he goes, thinks for a second, "Why don't they just ask AI?" Cooked. Cooked. The child knows that my business is is is done [laughter] for. No, he I wasn't sure if I was going to bring that up or not, but I thought it was hilarious. >> No, I mean, if you don't have existential threat, then you're not moving fast enough, right? >> Totally. I mean, like you look at like Open AI and Anthropic, they're like the furthest ahead, but if you talk to
[07:36] researchers, they're like a lot of them are like freaking out. They're like, "Oh my god, you know, our model's behind in this one capability, that means and going to be compounding growth that means we're going to lose the race to AGI, right? It's like, okay. >> Yeah. Let's go Let's go straight to it. So, obviously a lot of discussion a lot happened this this last week. Department of War blacklists Anthropic. Want to get your thoughts on that. There's kind of a lot of nuance there. Um so, the US government just labeled Anthropic a supply chain risk. What What is your
[08:07] read and then literally later that day OpenAI closes the deal with the Department of War. What's your read on that situation and then I got a bunch of follow-ups. >> like really understand the two companies. And and and by the way, OpenAI reneged on that deal that they did on Friday and signed a new one on Monday, right? Uh with a bit more tighter restriction. But like, you know, the the two companies like there's a lot of interesting stuff, right? If you you could take one view from Anthropic which is like, "Oh, it's because Anthropic and Dario won't bend the knee to Trump and won't donate to him. That's why they're getting attacked." Because if you look
[08:38] at the OpenAI deal, it's like, "Oh, yeah, it doesn't allow mass surveillance." Um which is like the main thing, right? >> Yeah. And autonomous weapons. >> Yeah, and autonomous weapons. >> Yeah. Although Fully autonomous. >> Fully autonomous. I was going to say reportedly both allow autonomous weapons to some extent. You You could take one view that's like it's just like Anthropic is being done. The other view is like Anthropic's just being like dogmatic, right? Like there was a situation where I think um it was put out by some press um but it was like Dario was asked um or someone in Anthropic was asked, "Hey, if
[09:08] your if a nuclear missiles heading from China to America and we can use AI to stop it, um but it requires, you know, surveillance and and autonomous weapons, what do we do?" And and Dario was like, "Well, you can call us. We'll I'm sure we can figure something out." And it's like Like this is just the dumbest response you could ever come up with, right? Like I don't know if this is like even fully accurate, but this is like what was in the media. At least this is what everyone's perception is, right? So, there's like two It's like, "Okay, the government is and and Trump admin are out to get Anthropic." The other view is like
[09:38] you know, Anthropic is actually just super dogmatic. And I think the truth is it's kind of like in the middle, right? And then you look at OpenAI, it's like look, I don't know how principled Sam is, but he's certainly one to take advantage of a good crisis, right? That's maybe his greatest skill, right? Is to take advantage of any a good crisis. And so, you know, he he he goes out there and he signs deals with the US government. Now, the US government is like you know, even though Anthropic was the first to sign something with the US government in terms of a
[10:10] exclusive or not exclusive, but like >> one with access or or implementation in a classified environment as well. Yeah, yeah, exactly. But despite that, Anthropic has has lost the plot now and then and OpenAI just saw it and they swooped in and now they're going to be the one there, right? Yeah, I mean, it's funny you say bend the knee. I mean, just I think it was today it was reported Dario said the reason they're getting so much from the Department of War is because he did not give dictator-style praise to the administration. And you know, obviously David Sacks has tweeted a ton
[10:41] about Anthropic and Dario and their politics. So, how much of this do you think is is politics at play? And yeah, I'm just curious. >> you also have to recognize that Anthropic has a lot of their policy people are former Biden admin people, right? And so, it's like whereas whereas OpenAI has hired people from both parties, right? Which is the pragmatic thing, right? You go look at like Microsoft, right? Who is like known for navigating the government well, they have people from both sides in their policy teams and such, right? You go
[11:11] look at like any major company like Amazon who has government contracts or Lockheed, they have people from both sides, right? And it's like, you know, Anthropic like maybe maybe just like, you know, you you to get to your end mission, right? Your end goal unfortunately requires like being a slithering snake and like, you you playing both sides and like the political you can't just be a dogmatic person, right? And so or or company, right? And and but that is their DNA, that's their strength, that's why they're winning, right? At the same time, right? Is them being so dogmatic
[11:41] about the things they they believe in care about. Um that's why they're winning, right? Um so so I I I think it's it's a it's a tough one. The more interesting question to think about is like what happens from here, right? Um Especially leading up to the IPO. Yeah. Being designated a supply chain risk. Like what it Yeah, like what do you think? How does that Does that affect the business? So, you know, OpenAI OpenAI did the sign the deal Friday. All weekend people internally were like rioting and Monday they like pulled back and signed a different deal. Um
[12:11] that Anthropic is basically like what we we we exactly asked for this and you guys said no. But it's sort of like you you know, this shows, right? If you tell the government, you can have everything and slide a little bit back versus you can't have anything except for these things, it's a different argument. And so when you look at like, hey, what is the end state the US government wants um to be viewed as the as the winner, right? As the as the king, right? They they don't want to be the one that's like, oh, thank you, benevolent corporation for giving me this. >> Yeah. Um and at the same time, you know,
[12:43] the the AI labs are like, well, no, we made this technology, we have complete rights over it and like what what you would can and cannot do. And it's like, you know, if the US government wants to go full dictatorial, they can just use the Defense Prioritization Act and get whatever they want from you, right? They can just go get 4.6's weights if they really want it, right? Um They could do a number of things uh to to have access to everything Anthropic has. So, it's not like, you know, Anthropic is even stopping the US
[13:13] government in that case, right? Like from having access to things. And then and then the question is like, okay, how how do how do you go forward from here, right? The US government can press harder. Um Anthropic can stand up, but Anthropic can't just back down and kiss the ring immediately, right? They are They are so dem-coded. They are so not even dem-coded, right? Just like so so dogmatic in their beliefs. Yeah. If Dario and leadership were to say, "Okay, US government, you get this." People below that would leave, right? Or people
[13:43] below that would riot, right? Open AI doesn't doesn't get that, right? You know, sort of You know, you you've got you've got a lot more let's say zealots for the cause at Anthropic than any other company in the space. The only way to make these people understand and back down is to be like, "Look, the US government's forcing us." Right? I think there's a bit of like chess that you know, even even even if Dario wanted to accept the US government's proposals, he kind of can't because then a lot of his best researchers will like be pissed off and leave. Maybe maybe that's the risk you have to take, but also the other one is like you
[14:14] could just be like, "Okay, let's see the US government push us harder and harder and harder, and then when we back down, you can tell them, 'Please, we had an ultimatum, and it was like either they just take the weights and we do nothing, or we work with them, and yes, we make the killer robots, and we do mass surveillance, but we do it with Claude alignment rather than them having the open weights and doing whatever the hell they want." So, you know, this is the pragmatic thing to do to prevent AI from killing us all because, you know, our killer bots will be better than you know, Open AI's killer bots. And then everybody will be like, "Yeah, you're right." >> Yeah. Thank you, Dario. So, sort of like
[14:44] I feel like that's what has to happen. And at some point Dario does just have to be like, "I love Trump. I love everything Trump says, you know, I love I love the Department of War, you know." >> they're really like the only tech company that hasn't bent the knee, right? Every every other tech company has bent the knee, has donated. So, it's it's kind of an interesting position, and it does >> scary. Yeah. Yeah, a little bit. Okay, so I want to one more one more thread on this. So, Claude is currently the only model currently deployed in classified military networks. They obviously were
[15:16] just announced trying to rip Anthropic out. How do they do that within 6 months? Is that even possible? I don't see why not. I mean, I think I think the version of the model that they have is like it's like 3.5 Sonnet or something like that. It's like Really? >> Sonnet. Yeah, it's like it's like an older model that they have deployed cuz it's the weights, right? It's like they have access to those weights. I don't think it's like they have Opus 4.6 deployed in classified networks, right? >> Yeah. It is max fast. Yeah, no, no. I mean, I I I believe it's like an on-prem like thing. Okay. Okay. Yeah, I guess it
[15:46] would have to be, right? >> [clears throat] >> Yeah, so in that sense, it's like sure, get OpenAI to give you, you know, GPT-5. 5 GPT-5 weights, right? That's a better model, right? Or hey, Grok, give me 4.1 Grok 4.1 4.2 weights, right? Like, you know, there's a lot more out there. I mean, like Chinese models are better than Claude Sonnet 3.5, right? Um You see the government using this? >> No, no, absolutely not. But like, you know, the point being that there's like plenty of sources for that capability tier. Um
[16:16] And so like I think it's fine. Like, you know, like I don't think it's that hard to rip it. I mean, I imagine it won't be that hard to rip it out, right? Um but who knows? >> It's kind of wild to think that they're using such an old model for such important work. I actually didn't know that. And now I'm thinking about it and remembering the level of hallucination coming out of those early models. And and it's deployed >> remember 3.5 Sonnet coming out be like, "Holy this is this is so good. I see I see the sparks of AGI." >> Yeah, yeah, yeah. Um no, I mean, that's fair. That's fair. I just think it's kind of um
[16:47] there's a lag, right? Like, you know, government is just slow. And this is like I think the biggest risk uh that that also like will help Anthropic people understand and be convinced that fine is China's not using, you know, the oldest model. They're using, you know, the newest Qwen. Uh they're using the newest Kimi. They're using the newest Deep Seek, right? Like, they are not in their military applications. There is no, you know, waiting around. There's no slow timeline to deploy. It's it's shipping, right? Um and so like
[17:19] maybe the US has a 6-month advantage, but if the US government has a 6-month time lag between the new model coming out and deploying it, then there is no advantage. Yeah. Right? For the US government and military. And then if if China is able to like have all these drones that they can manufacture at high volumes and control these autonomous drone swarms with, you know, very advanced AI that is on par with the American military has, except the American military doesn't have drone swarms because they can't we can't manufacture drones in high volumes. It's like, wait, our military capabilities are actually just like worse cuz they have lower cost, more drones with
[17:50] similar intelligence. It's like, where where is the advantage for us here now? Where is the advantage? I mean, we you know, we have companies like Anduril, but they're, you know, still coming up. They're it's still nascent. So like what what is the advantage then? Exactly, right? I mean, there there there isn't one. And so that's the biggest risk and challenge. In in that type of like if you if you think like drone swarms are the end-all, be-all of warfare, right? There's other things, right? That potentially, you know, the US has I
[18:22] don't I don't know what the US military has to be honest, but like Yeah. could be. All right. Um yeah, you got me thinking on that one a little [laughter] bit, Dylan. What you thinking about? >> Yeah, no, just it's I think I think the craziest part is I realized that yeah, the US military has these old models and that makes it equivalent to what the Chinese military is able to have currently. >> doesn't have to be, right? Like No, of course not. >> out how to do faster, right? Like >> Yeah. I think I think like, you know, flip side is like the US government buys data from Google,
[18:54] from Twitter, from Facebook, Meta, sorry, on all the on on their users. And then they have their own classified sources of information and it's like right now building all the data pipelines to mass surveil everyone is actually just hard, right? We haven't done it as a as a country. I don't think because of the law, I think because it's hard. Right? I think the NSA would absolutely do it if they could, but they just like it's it's a monumental task. Whereas China has, right? Whereas like another countries have, right? Or some other countries have tried and gotten along the ways, but like China especially is the closest to that.
[19:25] But like if I look at how easy it is to use Opus 46 to like build data pipelines and, you know, uh transform data and scrapers and all these things, I feel like I feel like doing mass surveillance, building a mass surveillance system with Opus 46 would not be terribly difficult, right? Like, you know, um and with Opus 5 or whatever or or, you know, uh OpenAI's new model that they're releasing like, you know, like, you know, may maybe maybe it's even faster and faster, right? So it's like, you know, these autonomous like this
[19:55] like mass surveillance which is like one of the red lines in the sand is like it's it's a true moral quandary, right? Because I don't want the US government to have, you know, China-like mass surveillance or better, right? But at the same time if you give them full unfettered access to, you know, Opus 46, they can totally build this, right? Um So you think Dario's line in the sand is is uh coming from a a true place of belief and and morals? >> Look, if you think AI will kill is like X risk of AI killing us all is pretty high and it's because AI turns around
[20:27] and uses weapons to kill us all like autonomous weapon systems obviously makes sense to like limit, but also because like if there's autonomous weapon systems, um who gets to command these autonomous weapon systems, how are they aligned, like all these all these questions should come up, right? Um fine, that's one. The other one is like even without AI going bad and rogue, we know people are bad, right? Generally, like in positions of power. And if you create mass surveillance systems for the American public, you know, what does that now do to free
[20:58] speech and all these other things, right? You and I can come here and we can say whatever we want. Um you know, we can say Biden or Trump, but like, you know, like whatever, right? Probably nothing gets flagged. But now, if you have like these scrapers that are pulling every record of people on videos and like, you know, it's like, oh, somebody's account on YouTube, which is linked to this email, or like, based on these comments is based out of this area and based on this other information is based here and is this person. Now, all of a sudden, this person said Trump after I said Trump on some random podcast, right? Um
[21:29] or Biden after I said Biden on some random podcast. Now, all of a sudden, they're being surveilled and now, you know, can that be turned around? So, there's I think there's like a real risk to like, you know, mass surveillance, right? It's not that it's like trivial, right? Like, I do think the beauty of America is that we can do whatever we want, right? Generally. And AI systems make it very, very likely, even pre-AGI, that we can't do that anymore, right? Um that, you know, bad actors can now manipulate systems to control people,
[22:00] um especially as like social unrest grows and grows, right? Uh AI's causing social unrest to grow, uh the power of capital versus labor. >> of AI is causing social unrest? Is that the misinformation? Is it just the politics around it with energy and and I know you talk about water a lot. What what part is it? Um I I think it's like all of the above, right? Like, I think like, um generally over the last what, 50 years, we've had capital meet take more and more share of value versus labor. Um we have seen social media amplify
[22:30] um what other people have versus don't. And so, like, you know, even though 50 years ago, you know, rich people lived or middle class people lived worse lives than middle class people do today, I think, on an objective basis, at least in terms of like, what is the square footage of their house? And how much meat do they eat? And what is their access to medicine, right? Because the perception is not based on that, it's based on historical thought process of, oh, you know, we own our own home and we have two a car and we can send our kids to college and blah blah blah with one wage, even though that was actually like very rich 50 years ago, um and still
[23:02] possible today, but you still have to be very rich. But then or flip side, right? Like, you know, "Hey, I'm a middle-class person or I'm even like in the 25th percentile." There used to be a belief that people in the 20th percentile thought they were middle middle-class. And people in the 80th percentile thought they were middle-class. Or or didn't think they were middle-class, they knew they were well-to-do. Nowadays, people in the 90th percentile think they're middle-class, and people in the 50th percentile don't think they're middle-class. >> Yeah. Because it's the perception game of like, "Oh, well, like look at these people, they're going on vacations."
[23:32] It's like, "Yeah, because they're putting a fake life on social media, their peers, right? That's not what their actual life is, right?" And then on the flip side, it's like people, you know, see influencers, see like, "Oh, that is the norm, right?" And it's like, "I don't have that because I'm poor." Well, no, you're middle-class and like, what the amount of vacations and experiences you go on is actually more You know, I think I think like obviously it's like it's very privileged for me to say like lives are better, you know, at least, you know, especially for me. But like at the end of the day, like on many objective economic measures, lives are better. But people think their lives are worse, and now more and more over 50
[24:03] years because it's continued to be a slide of both capital taking more value, income inequality growing, this perception game of social media, and these things keep amplifying, especially in the last decade, social media's really amplified it. And now you have job loss, right? Potentially happening, and you look at like, you know, economically, usually people have been fine when GDP's like 2%, which is like, you know, fine, what it is now. Except now of that 2%, 1.7, 1.8%
[24:34] is is AI growth, right? It's like, "Okay, who's getting the benefits of AI growth?" It's like electricians, contract construction contractors, people in the semiconductor industry. Capital allocators. >> Yeah, capital allocators, that's it, right? Like it's like That's that's less than 10% of the economy, guys. Like, you know, so it's like most people are not experiencing economic growth, and they see this inequality, and they look for boogeymen, and the boogeyman is the richest companies in the world, the stock market soaring, and AI, right? AI taking and all the risk of AI taking jobs, which is about to happen in droves, right? You know, Waymo's about
[25:04] to start deploying in all these cities, robot taxis about to start deploying in all these cities. The boogeyman of, you know, few million people losing their jobs because of self-driving is going to happen, right? And in addition to that, white-collar work, right? Like, you know, the amount of work you know, my company can do with the how few people has, the amount of work that you can do with how few people you have would not have been possible, and so we're able to build an enterprise, but like we're we're enterprising individuals. What about all the people who aren't enterprising? They just work a job and then they get like, you know, they they see this tidal wave coming.
[25:34] Um, you know, it's it's it's very scary, I think, right? And so >> Yeah. Do you feel a lot of anxiety in the city right now, even with engineers? I It's interesting you say that. I I tend to be more optimistic. I I I think maybe there's going to be more smaller companies, but you're right, there there needs to be enterprising individuals to go out there and actually start these companies to build value because we're not going to have these massive 100, 200,000 person companies anymore as you, you know, Block just laid off half their workforce. So, are you are you generally pretty pessimistic with the, you know,
[26:04] what Dario called the white-collar bloodbath? Um I I I think generally there will you know, even though there's more surplus in the economy, the allocation of that surplus is harder to get to the people, and so there will be an an and markets take time to reorganize. Um, so there will be a a hard dislocation, right? Um, yes, there will be new jobs, more things for people to do, more service uh than ever, right? like,
[26:34] um you know, more experiences, right? Most people will then be able to go and do like random more painting classes or more whatever classes or more like park whatever, more beach, more surfing, like whatever it is than ever if the surplus was spread evenly and people were to able to just reallocate perfectly. But that doesn't That doesn't happen. the technology is moving so fast and the ability for society to keep up with it you're saying is certainly not fast enough. >> Right, exactly. Like the whole like, "Oh, but you know, we've had we had 90% of people working in agriculture and now it's like less than 2% and it's like,
[27:05] you know, it's like actually less than 1% it's just like people claim their farms for tax reasons." Uh and so it's like less than 1% in reality. Um anyways, like it's like Well, yeah, but that took 100 years, right? Uh it took more than 100 years, uh maybe even, right? And so like whereas like this AI dislocation is like immediate, right? And and so um probably I'm I'm generally an optimist. I don't have anxiety about this, but I know a lot of people do. Um both normies across
[27:35] the space, right? Like, you know, met some of my cousins, met some friends out that that aren't in the space and they're like they're more worried about it. Um you know, met met you know, recently, right? Uh my Uber driver recently. I still take Ubers because I think it's fun to be able to talk to an Uber driver versus a Waymo. Uh you tell them that their jobs are about to be completely >> No, I don't say this stuff. I just I just vibe, right? Um my my philosophy is actually insane, right? So Waymo's objectively the median Waymo is better than the median Uber, right? But like,
[28:06] you know, that's boring, right? Like, you know, Ubers I would say 90% of Ubers are worse than a Waymo. But there's that 10% that has some interesting music, some amazing conversation. Um you know, I practice my Spanish, right? Whatever it is and and there are good fun conversation. That spikes because of the human connection. It's like, "Okay, this is worth it." Um but I think in general, right? Like that that way I had a Waymo driver, right? Soviet grew up in the Soviet Union, moved here, was a taxi cab driver for 30 years. Um he started ranting to me about how Uber doesn't pay taxes um and has never paid
[28:36] taxes. Started ranting to me about how much he used to make uh taking taxi doing taxi driving and now how little he makes. And in my mind it was like, "Well, yeah, that that's like, you know, the commodification of driving for someone is like been a surplus for everyone in the economy. Obviously, that hasn't been distributed evenly, right? People take more taxis than ever because Ubers are so cheap, but you know, at the same time you started ranting about like how um you know, he's like, "Yeah, people don't need all this technology, right? You know, most people don't need all this information at their tips. They have it, but it doesn't actually benefit their lives. They'd be better without
[29:06] it." Um and like he just kept ranting about and I was like, "Wow." I asked I start I just asked him how his day was. What are your thoughts on that point? That we we have enough. We don't need more tech. We don't need more info. Like I mean, who am I talking to? What am I saying? Yeah. I love it. I love it. Yeah. Yeah. Give me more slop, right? Like, you know, like um I think I think I think it's fair like, you know, I think people people's, you know, uh dopamine receptors are short-circuited and there's probably some way to make it better and I mean, you guys put your your semi-analysis charts on the like
[29:37] the What is that game? You put You make it good for Gen Z so they don't get distracted. I I It's It's a bit of a shitpost, but yeah, yeah. So, what we do is we do uh Subway Surfers. It's Subway Surfers and then you like explain an ML paper over the top and and then you also add some background music. It's just great slop. >> So funny. It's slop. Yeah. I think I think dopamine is short-circuited for everyone. The reward circuit. Um probably a lot better would be done if the world had like, you know, not you know, if if the average person had a longer attention span,
[30:08] um could focus more, um I think generally a buffet of dopamine is less happiness. Yeah. Um I personally, you know, it's like type one versus type two versus type three fun. I personally think that like the pain and the grind bringing, you know, like the reward is the most fulfilling thing in this life, right? And I I'm I'm sure you feel the same way. Like when you're like, you know, you you you put your kid down instead of going to sleep like most normal people, you're probably like working on some something, right? And it's like, but like then you like, you're like, "Wow, that was fulfilling,"
[30:38] right? So, that's that's like but like a lot of people don't get that, right? Um because they go to work, they come back, and then they don't have a mean you know. And so so it's like sort of like, you know, there's something about like hey human psychology and what is the reward mechanism of a human brain? Like probably we could change. But like you shouldn't like you know, at the same time technology is like improving agricultural yields, right? And like people aren't dying of world hunger and like you know, technology is making steel production faster and like
[31:08] technology is making automobile production better and like batteries better. Like all these things that make normal people's lives much better, right? You know, we have more abundance than ever because of technology. So you can't stop that, right? Like otherwise like degrowth is like the silliest like ideal ever, right? Like especially popular in in Europe, but like degrowth means that like people people yeah, it's just like a terrible idea. >> Okay, so we we went on a super tangent there, which I I absolutely loved it, but I want to bring it back to something you said earlier. You were talking about Claude code being especially 46 being
[31:38] able to build out incredible data pipelines and and transformations and let let's talk a little bit about software Claude code needing software and and in February in your newsletter, you said Claude code was at an inflection point comparable to the chat GPT moment. You mentioned earlier that your token spend is absolutely skyrocketing. Although you're like yeah, it's fine. The returns are there. Uh walk me through what you think is happening right now with Claude code. A lot of people are liking codex like the
[32:08] whole software industry is changing. What are your thoughts? >> like like yeah, like cursor their revenue went from a billion to two billion in like a few months, right? It's not like it's not like Claude code's taking all the lion's share. Um you look at you look at codex. They've they've eclipsed a billion after having launched not so long ago. They're probably closer to two billion now. Like you know, Anthropic skyrocketing of course. Like I I don't think it's just a Claude code. I think there are many people who are a tools that are adopting and getting uh strong coverage. A lot of people perceive these tools as a coding thing and they aren't, right? I think
[32:40] if you take a long look at them and you take a long look and you force yourself as a non-programmer to use these tools, you can get huge productivity gains because this you know, when you talk about like, "Hey, what is Chat GPT like versus like what is like what are these like?" It is a completely different like Claude Code, Cursor Agent Mode, Codex, these are agent orchestration systems and you can get them to do anything, right? You talk to them in natural language and they go do stuff. And and you can you can Sure,
[33:11] you can make them make software. Um and so right now Claude is the best at this, right? But uh soon OpenAI will release their new model and that new model will not just be, you know, the the reason Codex 5 2 Codex X high or whatever the hell the name of the model that it currently exists is is so good it's better than Opus 4.6 at coding. It's worse at everything else because they took the Garlic model and then they just did RL on only the coding line, right? Whereas the model that they're going to release soon is is the pre-trained, you
[33:42] know, Garlic model but then they RL'd it for everything, right? And what's relevant here is you know, tool use outside of programming domains. Um ability to understand things outside of coding domains. And so when you think about what is an agent orchestration system and what am I, you know, like everyone's like, "When's the year of the agents?" It's it's now, right? Like because Claude Code Opus 4.6 is the agent orchestration system, right? And you tell it and it spawns these agents and it does these things and it comes back with work. Makes charts. You tell it to learn skills. And then these skills are like, you know,
[34:12] it's like just yesterday I heard the silliest one from uh hedge fund colleague, right? Um I think this one's super funny. He's never programmed in his life. He was told He started using Claude Code after our article on Claude Code and and Doug at our company who is is chief Claude He's he's the president of the company. He's he's like my partner, basically. But like, you know, he's he's so Claude code psychosis that he's just ranted to people about it. So, this hedge fund client goes, "Well, so what we what I did was like, you know, one of the things that I've been trained to do is listen to earnings, read earnings
[34:43] transcripts, and really digest the the wording that they use. And based on like how they speak and their tones and their reflection, know when they're bullshitting versus not or know when they're exaggerating versus not because this can lead to outsized returns." And so, there's entire funds that have been started on this premise and been successful, which is just you know, looking at and the tone of how people talk. And so, he built a skill within Claude code. He sent it these these books from this, you know, CIA negotiating tactics and like learning,
[35:14] you know, tone reading and language reading. So, he sent Claude code all these skills and made a new skill within Claude code, which was understanding this. And then he fed it any any fed it some earnings transcripts that are famous in the past of having tricked most people, but then actually have like had extreme alpha just from the wording or the tone rather than the actual words they said. And then the skill was then able to go through all earnings and he's like using this to like it's like, "Wait, he didn't code anything though, right? What what did he code?" He told Claude code to learn and read these books and then
[35:45] understand how to like tonally and like you know, all these so so it's like you don't have to be a programmer, right? This is all knowledge work. You know, if you know how to do it, if you can be descriptive, the model can learn, right? It can get a skill and can do it, right? So, today Claude code is the best at this, but as you as you step across the world, it's going to be so many other places, right? As I mentioned, right? You know, we're we're doing you know, Jeremy's doing it for data center stuff. Doug's never going to program either. He's a hedge fund person before, right? So, he's
[36:15] doing it for you know, so it's like you keep stepping across the world. It's like there's so many places where you know, you mentioned you're not a programmer, you're but you're using you're using Claude bot for all these things, right? So, there's there's there's no reason to think that Claude code or Codex is for code. These things are agent orchestration systems for And And And this is how you interact with AGI. Now, maybe maybe, you know, you actually need, you know, like normies will need a GUI to work with this stuff rather than like CLI, Well, that's Cursor. Yeah, but I
[36:46] mean, it's even that that's that's an IDE, right? I mean, Codex UI is is very kind of simplistic and perfect for the average user. I mean, I don't know. I think I think like I've forced people in my company to use Claude code and they're like, "Oh, terminal scary." Right? They're like, "I'll wait for co-work to be good." And it's like, "No, no, no, you're going to use Claude code." And then like they force they try and then like, "Wow, this is amazing. This is so good." Um My point earlier, by the way, so I I I have been a software engineer for for
[37:16] a while, but I am not actually writing code nowadays. It's all natural language. It's all vibe coded. I did I don't have time to actually like read all the lines. Plus, it's all software for myself and my company, so it's not a big deal if there's, you know, some issues here and there. But, I I think you're right. It's It's interesting that you can you can basically build anything at this point and anybody can do it. Um there are still some rough edges. And so, like if you vibe code long enough, you start to realize where those rough edges, where those limits are. I think a lot of people aren't using the million context Claude code, though. They're using the 200K or whatever. Um cuz the the plan that they subscribe
[37:46] you to, right, is is capped context. And but if you use the like million context, which is only available via API that you have to pay out the ass for, Yeah, you do. it's like way better. Yeah. But, I agree. I agree. There's rough edges. I think I think what it is is as well as like in the past in major companies, you know, leadership or like, you know, team leads, you know, people who are doing the actual work of the company would like create all these specification documents, blah blah blah blah blah, and then they'd get another team to implement it. >> Yeah. And it's like all of that is gone. The person who actually just understands the domain can just
[38:17] describe what they want, look at it, and iterate iterate iterate. Um And it's not programming. there it's business logic being encoded, right? So so let's continue on this. Let's talk about SaaS in general. I think we talked, you know, obviously there's a lot of talk about SaaS is dead. And I think where the world is going is probably agents on one layer and then the layer below them, some kind of file system, some kind of CRUD database. What does that leave everybody else? Like if if those are the only two things right? Satya famously said the entire application layer is collapsing down into agents. So agents,
[38:48] file system, where is everybody else going to be playing? Where's the value going to be captured? That is a tough one. That is a tough one to answer, describe. Um I think it's a lot easier to say who's a who's a loser than it is a winner today in software because of the pandemonium. Um But as far as um who who actually like wins, I I actually like I don't know if this is like non-consensus or not, but I actually think like Databricks and like Snowflake are like reasonable angles, right?
[39:19] Because these are like scalable data and compute engines, right? Versus um you know, a lot of things are like these are easily replaceable. At least for the next like super short term, right? Like let's like let's say by the time Opus 5 or Opus 5.5 comes out or whatever, you know, GPT-6 comes out, maybe even that is like easily one-shottable, right? Purpose-built solutions for uh or or or vibe-coded solutions for these things. And so I I you know, the scalable aspect, this is the the problem that
[39:49] um >> [snorts] >> vibe-coded things have is they're not scalable, right? So how do you have the agents build something that has hooks into stuff that is scalable, right? Um and so one of one of like like one of the guys in my company is like he's basically he he was our head of like, you know, data and all this stuff. And then and then the like sort of cloud code apocalypse happened. And now all he does is run around and like help people's vibe-coded things be scalable, right? Because everyone else is like they're like, "I don't know what crud
[40:20] is. Like I don't know what I don't know what that is." Like you know, they're like, "You know, like" He's like, "It's okay. I'll just change this out and now all of a sudden the solution that they vibe coded with like 10% more work from someone who actually like knows how to build scalable systems is scalable." And it's like plugging into thing like various layers of software that exist already, but it's like it's not application software. It's like sort of like these scalable infra software. Are you are you seeing a lot of the roles inside your company change? Cuz even with my small company, I hired a researcher 6 months ago. Now Claude code open Claude does it for
[40:51] me. All of the research puts together entire outlines of the >> them go? No, they're busier than ever. >> Yeah. Absolutely. You know, cuz enterprising, super sharp, willing to learn, willing to adapt. And he is busier than ever. He is building out different tools, different websites. I mean, he you know, so no, didn't let him go. We have plans to hire, which is why I I flip-flop pretty often. Although overall optimistic about the future of white-collar work. So I just wanted to drop that >> I I I'm I'm I'm the same way.
[41:24] As AI has helped us like we already we always ran like my my company's average age is like 30. I'm 29. We have a bunch of people are super young, right? We we move faster than everyone else. I think in this like space of like consulting and research and data services. And so like I think that's why we win, but it's been a compounding thing, right? With AI. And it shows it's just moving faster and so we're winning, right? And then like you know, hired and we were all moving faster together. But then like AI started coming out, you know, started like being more integratable. We started
[41:54] moving faster and faster. Now with like Claude code, I think we move like 10 times the speed of any of our competitors, right? Like it's actually insane. And so like my plans of hiring and the business we win and the business we're able to do is actually just like growing, right? Because there's this like dislocation of like what can AI help us do today versus what can everyone else do? And so there's an arbitrage in the value of like what we're able to take because AI exists, right? And because we're just skating ahead of the puck. >> You you're describing Jevons paradox, right? There's more use cases because you're able to leverage AI so much more effectively. It's becoming cheaper.
[42:24] You're you're implementing it in more places, but at the same time correct me if I'm wrong, you are actually still hiring. We're hiring more than ever. >> More than ever. >> We have we have like 20 positions open right now. Right. Like you know, we we we've we've hired like what, 10 people this year already? And we've got like 20 positions open. Yeah, so it's like you know, we're we're hiring more than ever, but I think the flip side is when I look at my competitors, I'm taking their revenue, right? And do they now need to do layoffs, right? And in fact, we're doing we're taking revenue in a way that
[42:54] is way cuz I I pay my people more than competitors do. Um and we do it with less people, right? And so sort of like there is like this like arbitrage of like, okay, well like the competing company in XYZ space has way more people than us. Right? And so like now as we take revenue, uh now obviously my space is just one that's growing overall. So more so they're like flattish and we're taking all the growth rather than, you know, we're taking the growth and they're shrinking. Um Are you are you
[43:24] taking on use cases and new business lines that your industry historically has not touched? Is that maybe part of what's happening and why you're growing so quickly? >> Oh, yeah, for sure, for sure. I mean like there's there's growth in the areas that we already cover, right? All these aspects of semiconductors, data centers, etc. etc. etc. Right? Um but we're also like inventing new industries like tokenomics, right? Which we've we we sort of I I I I we we started the beginnings of it like 2 years ago, 3 years ago, the economics of tokens, production of infra inference volumes, like sort of like cost of tokens, all these sorts of things, usage of AI. Uh
[43:56] there's that, but then there's also like existing industries like energy modeling, right? Turns out energy modeling is like a like a billion dollar a year data services business, right? >> And now you can just go expand into that because you have all the tooling necessary. Vibe could you know, lack of a better word. Well, some some some level of vibe coding, some level of we already have a lot of customers in the space that use us for data center stuff or AI stuff. And so we have some in into the sales side, right? We have a name brand and recognition, right? So sort of like and then we have the like capital to spend on hiring some great people and
[44:28] vibe coding things that, you know, people wish they could have built because we're building the solution that like is needed for now instead of the thing that worked two years ago, right? And took so long to build. So I think like, you know, that that's also an example of this is right was like energy modeling. I think and I hope that we're going to just dominate it because all the other companies in the space are just moving too slow, right? And they have like all these archaic old practices and built up crud, right? Crap, not crud like the Anyways, yeah. >> Yeah. Um so I I I think there's a bit of like both, but it's like by and large I'm hiring and I'm hiring
[44:58] super fast if anyone's like an enterprising individual who wants to work really hard and and talk to the great biggest companies on the planet and like work on their stuff, great. But like and and like help consult for them, but like at the same time, I fully recognize we're taking a lot of share from competitors, right? The the airwaves like of like you [snorts] know, especially like, you know, content creation I think is the same way, right? We're in an age where the cost of creating content has fallen and fallen and fallen and fallen. And so objectively things splinter, right? You
[45:28] know, we're you know, what is it? It's not Pax Americana. What's the American monoculture? American monoculture meant that everyone had seen the same movies and so movies like could be Blockbuster always, right? And and this this this spreading of like what people watch, yes, people watch more stuff and yes, there are the peaks are higher than ever, right? With with regards to the biggest movies, arguably. But like Titanic and like Gone with the Wind were anomalies would be complete anomalies
[45:58] today, right? And so like >> Those can't happen anymore is what you're saying. >> I mean I mean and creation is going down, the number of content amount of content out there is growing, the fact that your channel is growing and so rapidly means you're eating more and more share from other people who can now no longer make a credible living. Now, does that mean that's Hollywood or does that mean that's some like, you know, CNBC podcast or I don't know what market people are shifting away from, but what I do know is that all the executives of all these
[46:28] major companies like watch podcast like Dorcas's and yours. And like they listen to these things because it's like, well, this is how I listen to the people that like matter, which is like crazy. It's traditional media. I'll just say it. It is traditional media that's getting eaten alive right now. Right. And so how many more people work in traditional media that, you know, your hiring is irrelevant. >> Crazy. Yeah. My six-person team puts out so much content compared to what a traditional media company has. >> Engaging content. That's like good, right? Not like slop like watching CNBC all day, right? Yeah.
[46:59] >> their viewership is terrible. I was looking into it because I think that's the same across a lot of traditional media. It's like it's like all just a bunch of polish. We all thought right, it was the same when the late night host got canned and and everyone kind of up in arms and then they realized, oh no, wait, they actually have terrible viewership. Yeah, obviously there's a political angle to it, but you know, if you're bringing the eyeballs and you're selling ads, they're not getting rid of you. Yeah, exactly. And and I think like the the viewership stats on like, you know, again, like people had
[47:30] have this allure in their mind of CNBC, you know, Jim Cramer, Money Talks, all these other shows. Like Jim Cramer pulls eyeballs, but like the rest of the shows don't. >> Yeah. And and if you look at any of the other shows, it's like their viewership average is like 100,000. It's like, wait a second. You put out videos regularly with 100,000 views with pretty high engagement. Right? Um whereas there's like >> at a fraction of the cost. >> fraction of the cost. Whereas their 100,000 is that is that just the TV's on, right? You know, it's like it's like
[48:00] actually insane. >> Yeah. Yeah. Um And so, you know, we'll we'll we'll end up seeing but like media media is is is the same way. I think I think every industry is like, "Great, you and I and any enterprising founder who is taking share, making revenue, creating whatever it is wins, and they'll be hiring, but is that hiring going to supplement and make up for more than the people that are being left behind?" What do you think? No. No. Yeah. What happens? You know, I've always been
[48:30] such a capitalist in my life. I grew up in a small business. We grew up in the motel that my parents owned. My parents and and my my my parents and then my mom's brother and his wife owned that motel. We lived there. We worked there, etc. Um worked in a gas in gas in my dad's gas stations as well when I was a kid, right? Like, you know, sort of like as gas station. Um and so sort of like always been a capitalist, right? You know, because like that's what I grew up in. It was a small business. Uh over the last like couple years, I've realized, "Wait, I actually think like UBI is like perfectly fine." Um which I think is like crazy because like again,
[49:01] like I'm a I'm like, you know, if you if you I'm like very capitalist. I No, I I understand that, but yeah, I mean, a lot of people >> else do you do? Cuz society's going to rip itself apart. In fact, I believe politically the next election is going to be so so AI focused. Um and the Democrats will just win because they're going to be the anti-AI party. Um because right now >> the sentiment is that negative right now? >> The sentiment is already more than half Americans have a negative view of AI. Um
[49:33] and and as we fast forward to the end of this year, as Anthropic's revenue goes from 19 billion to maybe 60. And OpenAI's is is similar if not higher, right? You know, the amount of jobs that get supplemented, the amount of change that happens in society. Um you know, the stock market's going to get really impacted as as Google and other companies like Google next year, they've produced cash flows for the last like 20 years and and of ridiculous amounts, hundred billion dollars of cash flow a year, right? Google will have no cash flow next year because they're they
[50:03] see AI so clearly and they know that they need to spend every dollar they make on compute, data centers, energy, etc. Waymo, etc. etc. etc. because they know that the returns for that long term are going to be insane. Um But like that's going to nuke the market, right? Like you know, like people are like, wait, you know, these companies don't make cash anymore. And people are going to see, you know, job you know, there's a crowding out of everything of resources, right? Electricians and plumbers are demanded in massive volumes because setting up
[50:33] data centers and the liquid cooling for them is so crazy that now you crowd out other areas and industries. Uh could you say with construction, same with like, you know, all these you know, energy costs are going to probably not go up a ton, but they are going up like low single digits, mid single digits. Uh and then there will be other market factors that cause prices to go up as well, which then will be blamed on AI. Like anything and everything will be blamed on AI. People will hate AI. And the Democrats haven't turned to be anti-AI yet. Uh but if I was a Democrat strategist, I would become anti-AI as hell just to win the election, right? Regardless of what
[51:04] the opinion is because I have a jaded view of politicians. They say anything to get elected. Um And and and sort of the incumbent Republicans can't really say they're anti-AI because they're the ones that are pro-business and signing all these deals and like, you know, they they can't just turn around be like, we're anti-AI. Even though you would think like a could do so from like the censorship perspective, the you know, the companies making the models lean very left, right? I mean, that that has been said in the past. Doesn't matter. No? Doesn't matter. Less so than the fear of AI taking jobs and
[51:34] that's kind of more the left side. Yeah, I mean like and and it could be like an establishment part of the Democrat Party maybe doesn't want to, but some anti-establishment person wins. And And you know, in fact, I don't even think that like, you know, would you call someone like an AOC an establishment Democrat or not? Probably not. But then, you know, I'm not saying they run for she runs for president, but like someone like that is already anti-AI, right? Um and and and then like you think about you know, there is a wing of the Democrat and Republican parties that will both be anti-AI {slash} are already
[52:05] anti-AI, right? Um and and you know, much like Trump pulled a chunk of the Democrat party that like, you know, especially in the Rust Belt into becoming his voters, I think the same will happen with anti-AI and Democrats. Um I think that's a very big like sort of risk for uh and and and and then like what happens is like, you know, we do we have regulations? Do we have like a big slowdown? Now does America lose the global race in AI because of this? Like all these things are in the balance. >> Yeah. All right, so we've been talking a
[52:35] lot about politics. I want to move to basically uh a very close cousin to that, which is open source, and specifically DeepSeek. Uh we're probably going to come back to politics cuz that's obviously >> right? It's like, come on, do something. >> Yeah. Yeah. Yeah. Uh okay, so, you know, DeepSeek V4 reportedly dropping very soon, optimized for Chinese chips. I want to get your thoughts. Open weight, trillion parameters, and last time Last time we spoke, you said the export controls were actually making it harder
[53:06] for China to catch up. Do you still believe it? What do you think about the new DeepSeek? I know it's not out yet, but what are your thoughts uh of this upcoming open source whale? DeepSeek is not optimized for Chinese chips. Um they trained it on Nvidia chips. Um specifically Blackwell chips. Thought you were going to say that. >> This has been This has been reported by multiple like very legitimate medias um through their networks of sources, as well as from what we understand.
[53:37] Um there is a debate whether or not the chips were smuggled. I in fact believe that they were trained in Southeast Asia on rented clusters, not smuggled chips, but regardless And then did they walk the weights out on like a USB stick? What? Just send it over the internet. It's like a trillion parameters at eight at FP8 is a terabyte. That's nothing. Yeah, but it wouldn't it be caught on the web? It's FTP, right? Encrypted FTP, like whatever. You probably send a terabyte
[54:08] over the internet all the time. Yeah, absolutely. >> you're a video guy, huh? Right? Like you know, like it's like you know, it's like and your stuff is encrypted, right? When you send video to Google, it's not like the it's not like anyone can intercept it in between. It's like they have to intercept it at Google or at you, right? I I don't know. I I I I think I think like there have been many times where there there has been model There was I don't remember what company it was. It was a major Chinese company. They did send like people with suitcases full of hard drives, but that was because the data was like so
[54:38] sensitive. But they had all the GPUs in like some Southeast Asian country. And so they they like flew the data in with people just because the data was so sensitive. They didn't want to send it over the internet. But like weights are trivial, right? Like you know, Anthropic isn't like you know, every time they deploy a new data center with whether it's Amazon or Google or whoever it is, right? Fluid stack, etc. Like they're not sending the weights on a USB stick. That's so insecure. They're sending it You know, they they have this whole talk at AWS re:Invent. It was like very cool
[55:09] about how they keep the weights secure, but they send it over the internet, right? Same with OpenAI, right? Like there's no there's no like Yeah, it's not it's not a big deal. All right, let's actually stick on weights for a second. How like when you're looking at the competitive landscape, you look at Anthropic, you look at OpenAI, how important are the model weights versus all of the harness, all of the infrastructure around it, the knowledge to be able to train the models really well? Like what what where's the value really accruing inside of these companies? You know, I I I used to think the model weights for everything, Uh
[55:40] but more and more starting to like, you know, see other things that are very valuable, right? And so, as an example, there are still a couple folks in my company who use Cursor Agent mode with Claude 4-6. Um but for the most part everyone uses uh 4-6 Opus, even though it's this uh through Claude Code, even though the model's the exact same, right? You can it's a million context fast mode, right? Um it's the exact same model. The performance is very different, right? And it's something to do with like the harness that Claude has uh Claude Code
[56:12] has. Um that's one. And then two, um you know, in our internal GitHub we have all of our skills that are shared between all of us, right? Like it's like, you know, we have like these skills that, you know, someone builds. It's like, "Okay, now we have the expert data center permit analyzer." And it's like, "Okay, but that's only on Claude Code." Like if I want to remake this One, you can't have a skill for uh for Cursor yet. I at least I don't think so. Um and I don't think you can for Codex either. Uh and Codex is again like currently, although not soon, only IRL'd
[56:42] on code, not on the entire world like understanding. So, we'll see. Um But yeah, like these skills are super powerful. And it's like, "Oh, yeah, yeah." Like like Jeremy, because he's doing data center modeling, has built skills for like things that he's a super expert in. But now someone else who isn't even on the data center team can take that skill and like use it and be like, "Okay." Right? >> Yeah. Um and and like, you know, you you you You know, I I described earlier in the episode this like financial, you know, earnings transcript CIA level analysis of like tone
[57:14] uh to determine like deception within like a CEO or CFO's like statements. Like these sorts of skills are like I mean, it's trivial, right? Read these books, become a skill. But over time these skills are going to be way more powerful, right? Like I can only imagine like, you know, this is this is effectively like people like, "Oh, recursive self-improvement." Like it's like you kind of have that by like having skills, right? Um on Claude code, right? If you develop a skill, over time that skill can get enhanced. Um obviously it's like very manual today, but you can see that. And and and
[57:44] none of that has to do with the weights being updated. It's just like a block of KB cache that has, you know, the model read it and understood it, tried to like analyze it, made some output that then like put it into that mode, right? By prompting itself in certain ways. So like you kind of have this like you know, blocks of skills. I think that's like a competitive advantage. Um but we'll see. The you know, the reason I asked about the model weights is because as soon as DeepSeek V4 comes out, open weights, uh open source, they put it like how much of an impact is that really going to have? And then I also wanted to ask you because you did say these are trained on
[58:15] Blackwell chips, is DeepSeek going to lie on their white paper? Good question. So so um another another point back to that what you mentioned is uh Anthropic I think had like a agent swarm mode and it sucks because they didn't like train it properly. >> Really? Whereas like >> it. I don't think it's that good. Okay. Uh maybe okay, compare fair fair fair fair. >> And it's funny cuz I use it in Cursor and I really like it in Cursor. Oh, okay. Uh testing we've done is in Claude code. Maybe maybe it's like something to do with like the harness again, right? Um
[58:45] Kimmy's Kimmy's like model itself is much worse than Opus 46, but when you do agent swarm, it's it's boost from agent swarm is like much larger because they they did something there, right? I don't know if it's the harness or I don't know what it is, but Kimmy K2.5 uh with the agent swarm is actually like quite good. So like I think that's interesting. Um I'm not sure if it's just model weights, right? Like there's a lot of things built on top, right? Um there are clearly preferences between Claude code and Cursor and differences on their agent modes. Um you know, like there's
[59:16] people who use Claude code and never use plan, right? They just go go go go. And there's other people who think like who swear and live by planning mode, right? >> Yeah, I'm one of those. >> Um >> I have to plan. And then like think Cursor likewise has like like a bug agent versus like a you know, like a not bug I I don't remember the exact you know, I think they have three different agent types. And it's like very interesting whereas like, you know, Claude code has only one, right? But then you have these skills which kind of like it's like it's like a very like how do you build these agents and orchestrate them? Like that framework
[59:46] and such is is probably a competitive dynamic as well. And onto your other question is like, what is Deep Seek going to do then, right? Um, they've they've had some cool models since, right? They had their OCR model. Um, but have they built anything up in this sense, right? I don't I haven't seen anything yet, right? And so in that case is it like do you just use Cursor with Deep Seek and that's the best thing or do you like, um, does Deep Seek release the weights, but now, you know, people's harness I I I bet harnesses have to be like optimized to each model, right? Uh, cuz cuz I mean when you give
[60:16] instructions to a human, it's optimized for the human, right? Like once you work with a person long enough, you know what to tell them to get them to do the thing you want. >> even in Open Claude, I have multiple versions of each prompt because I'm using a daisy chain of different models. And that's the only way to do it because in you know, Opus 46, don't use bold, don't tell it no, don't use all caps. Exactly the opposite for GPT 52. It's like, use use all caps, do the Wait, you you you yell at your your your your
[60:46] Yeah, yelling in text, right? And so like the point is, yeah. Wait, what does that mean? >> [laughter] >> Like Opus doesn't like being yelled at. That's what that means. Coda uh I'm trying to like I'm trying to like I'm trying to like link this back to humans though, right? Because there are humans in which like you know, you tell them something calm and like that's how they work and then like other people you have to like inspire a sense of like >> Totally. deadlines and like, you know, you did this wrong, here's how you do it, like do it right, right? Like and and like I I I you know, I think as a
[61:16] manager like that that like does you do see that. And so like I love that. I I haven't used, you know, 4252 Codex enough to like know, but like >> Yeah. Yeah. Uh so, I mean, to your point, the the skills, how you build them out, you do need different prompts, different skills per model, in in my opinion. And I get the most out of it when I do that. Right. >> obvious when I'm using an Opus prompt or an Opus skill in uh Codex. Very obvious. Mhm. Okay, so but [clears throat] so so in that sense, like,
[61:46] back to the point is like, is DeepSeek's release going to like be Is it going to be as ground-shaking as R1? I don't think so. Which, by the way, a year ago. Is that wild? Yeah. So, you don't think so? I don't think so. I don't think so. We'll see. We'll see. I I just the gap that was closed with R1 was so large, and since then the gap is probably extended a little bit. Um there's the argument that like the labs are iterating, you know, like,
[62:17] back then, OpenAI hadn't released a new model in like 6-9 months, right? You look at OpenAI and Anthropic now, they're releasing new models like every 2 months, right? They're they're on their too, right? Um and part of this is because they're using the AI models to develop software that helps them build the next model faster and faster and faster, right? But like, their release cadence is insane. And so, when you when you look at this, it's like, you know, they're on their more, their advantage in compute is larger now than ever, right? Um you know, OpenAI has, you know, you know, north of 2 gigawatts, Anthropic has
[62:48] like, you know, a gigawatt and a half of compute. Some of a lot of it is spent on inference, to be fair, but like, more than more than half of it is spent on R&D, right? So, research, development of models, right? Um so, finding new ideas. Whereas, you know, back a year ago, right? OpenAI had 600 megawatts, right? And of that, again, like you say, okay, maybe they had only a couple hundred megawatts of training. And then you talk about they had some failed runs of Orion, things like that. So, then that like knocked it down even further and like, you know, or
[63:18] they allocated so much compute, so it's like um you end up with like the gap in compute of like what DeepSeek had versus what, you know, Anthropic and OpenAI had has extended a lot, right? DeepSeek's gotten more resources, to be clear, but OpenAI and Anthropic have gotten so, so many more resources, and Google, right? And in so far as much as compute is capability, um and it is to some extent, right? It is I mean, it is to a large extent, right? For any given company, more compute on a model is better. It's just what is their research What is their model
[63:48] architecture? What is their RL pipelines like, you know, that that may differ, you know, hey, Anthropic can build a model better than OpenAI for less compute, uh which is like an objective fact, at least today. Uh we'll see if it is in the future. Um but they've built better models with less compute, but internally with Anthropic, the more compute they spend on a model, the better it is, right? Um and so I think I think like when you when you when you square these, right, it's like there's a strong argument to be made that the gap between Chinese models and American models was the
[64:19] smallest it was in, let's call it, Q3 of last year, Q4 of last year, and it's going to widen again because of this compute gap, right? Um yes, [snorts] America always had more compute, and yes, Google always had more compute than everyone, but like kind of irrelevant. Anthropic and OpenAI didn't, and like, you know, the the the the the the gap was smaller. So I think I think there's like a legitimate argument there. I don't think DeepSeek V4 will be terrible, but I don't think, at least from what I've heard from like you know, people who work at like other labs in China, they're not as scared.
[64:50] Let's keep talking about the Chinese models, MiniMax, DeepSeek, Kimmy. Uh Anthropic put out a pretty scathing blog post about a week ago accusing them of distillation. Uh so there's a lot to unpack there. First, what did you think about that blog post? Do you think that's like Do you think they were right? It didn't seem like at least two of the three had enough data extracted distilled from the Anthropic models to really be meaningful. So, let's start there. Do
[65:20] you do you What did you think about that? I mean, it's it's very obvious that both OpenAI and Anthropic have outsized traffic in Japan and Korea that is in Chinese. Right? Like it's it's like there is like some things that we've seen that like show this. Right? Now, that could be anything. That doesn't necessarily mean that it's like, you know, distillation. Could be anything, but like, you know, there there's obviously a lot of like usage, right? And if you go like look at like various
[65:50] coding companies and their conversations that they have, a large percentage of conversations are Chinese. But like again, like like it doesn't mean anything. Like there's a lot of Chinese developers. Um And and so, we've seen models like do this, right? Like there have been Chinese models that people have been like, "Oh, this is clearly distilled from OpenAI. This is clearly distilled from Google. This clearly distilled from Anthropic." Right? It's more more like, you know, there's a class of people online who like just are like peak model vibes people. Right? Um you know, on on you know, there's a class of them on Twitter, right? Like
[66:21] like like Jan and all these other people, but like, you know, they they they have like peak model vibes. I don't know what they do to like understand the models, uh but they they talk to the models a lot and they just kind of understand what's going on. And and I think it's like at the same time, there have been models that have like very different capabilities from China as well. It's not like every model looks like a Anthropic, OpenAI, or Google model. They are I do think they are distilling to some extent. >> Yeah. Um Do you Do you think that Anthropic >> distilled from Chinese models, and which which distilled from, you know, US
[66:51] models, right? And like, you know, sort of it's like, you know, Right. Well, they're all distilling from the open web. And so, is it is is it hypocritical for Anthropic and OpenAI to accuse the Chinese companies of distilling from their models when, you know, they're they're all using stolen data at the end of the day. Less and less is that capabilities that's stolen data though, right? Like pre-training scaling is good and important, but more and more of the capabilities come from RL, which is not web data. But that was the original sin. Right. I agree. I agree.
[67:21] Um The original sin wasn't even that bad though, right? You know? Um this is like a religion You know, we we talked politics, let's get into religion. How bad was the original sin, you know? Yeah. Was the apple of Eden actually an apple or was it a pomegranate? You know, let's get into this. Have you Have you Have you heard this? >> No, no. Oh, so so pomegranates are a fruit that um across a lot of history um have been like used to describe love. Um
[67:51] and actually there's a lot of like religious scholars who believe the apple of Eden was not an apple, it was a pomegranate. Um and it was the original sin. Um I I think that's like just very interesting. Complete sidebar that Yeah, yeah. >> [laughter] >> I hope you keep that in. You don't think Anthropic is being hypocritical in in accusing each other Do you Do you think it is enough the data points extracted is enough to actually make a meaningful impact? Yeah, I I think so. I think so, right? Because you can train a small model on a big
[68:21] model on the same data. Um but then you still want to distill the big model into the small model. Yeah. Because the big model's general understood more from that data than the small model, right? And I don't mean to say small and big model, but this is like something that like people do already within the labs, right? Like um and you know, within their own models, right? Within their own company they distill from the big models to small, right? Like obviously. And so in the same sense, you take the best model out there because it's got a better better generalized understanding of the world and the open web It used to be just the open web, but like the open web and the
[68:51] data out there than any other model, and therefore you you may train yourself and then distill a little bit, right? Um I I disagree with the the comment that you mentioned earlier, which is that like, "Oh, these companies didn't get enough data to distill." It's like, "No, they they totally could have because they could have gotten it. If they didn't get it directly from Anthropic API, they maybe got it from one of the coding companies that uses Anthropic. And then, you know, cuz these companies use, you know, like between Cursor and Lovable and um And they can't you can't tell at that point. Anthropic's not going to point to a certain traffic and say,
[69:21] "Well, it went through Cursor." It's almost like a proxy. I mean, kind of, right? Like, I'm not saying it's Cursor per chance, by the way, right? Cuz that's a pretty big acquisition. But, like, there's there's a number of like Replit and Lovable and Cursor and like, you know, you go on and on. There's a lot of code companies out there. Or, you know, in in areas of And so, like, and you can use the model through there. And um yeah, there's an API key, but sure, like, you know, the traffic is obfuscated in some way. Like, you know, there's there's many ways um you can you can do this. Um And even just a little bit of data is enough to help still, even, right? Um
[69:52] you know, fine-tuning is a very small amount of data, right? Yeah. Um Right. >> Okay, so let's let's continue on open source. Last time we talked, you said closed source is going to win. Uh I think open source is having its moment, but not in the sense of the models, but I think Open Cloth made a big impact. I know it's primarily a lot of tinkerers and a lot of insiders using it, but I think when you look at people buying >> get hired by OpenAI, so closed source wins? >> [laughter] >> No, I'm just kidding. I'm Yeah, yeah. So, look, uh you know, they're putting
[70:22] Open Cloth on Mac Minis. Everybody's hosting it themselves. A lot of people are trying to actually run the models locally. Do you think there's almost this renaissance of of open source or or is it just like a a blip in in the timeline? I mean, Okay, in some argument, open source is winning because all this vibe coded stuff more than more of it than ever is open sourced, right? Um so, sure, open source is winning in that sense. Um And like as you mentioned, right? Like, you can build a lot more stuff on top with open source uh in an open source way by using closed
[70:53] source models. Now, as far as like usage of models, right? Which is like I think the like the strict definition, right? >> Yeah. Closed-source models are taking more and more share than ever, right? Open-source models are not getting adopted nearly as much as open-source models, right? Even It's like when you look at building out production systems, you have obviously the the fully hosted best of the best models powering a lot of it, but I think slowly and especially for my use cases, I'm picking out different use cases that can be run locally. And so, I have almost this >> Really? Yeah, definitely. Why? Cuz first of all, I'm a tinker, so I just
[71:24] like that. So, maybe not everybody's like that. Uh I I just like the fact that it's low latency, it's local, it's private, and and I remember you said last time people don't give a a about privacy. Um but it it just it's like nice. I >> I think with Open Claw, it's a little different. Fair. Open Claw is pretty freaking insane, like, you know, you give everyone I remember someone someone um a contact in the industry downloaded Open Claw, and Open Claw sent me a message like, "Dude, this is Open Claw." He's like, "Yeah." I'm like, "Dude, don't let Open Claw read my text
[71:54] messages with you." Like, we talk about like pretty sensitive like stuff, like >> Yeah. Um and [clears throat] and there's like vulnerabilities, known vulnerabilities in this. >> even worry about those vulnerabi- vulnerabilities, although I should. I think the prompt injection services worry me the most. Like, I am I I turned it off, but I was ingesting all my emails, and if somebody and it's a public-facing email, people know me, and so they would email a prompt injection, and and you're done, right? And um >> One of one of my one of my employees put in his email in all white text a prompt
[72:24] injection. >> Yeah. I know, it's hilarious. >> Do you know you know Pliny the Liberator, Pliny the Prompter? Oh, yes, yes, yes. >> Yeah. So, Another one of those people that I think is like an omega vibes, like, you know, like >> Yeah. Twitter person, right? >> So, I'm bringing He He's going to work with me on Friday. Maybe we'll cut this, but uh I'm I'm going to basically give him a single entry point into my Open Claw system, and we're going to work on it to together, record a video, and he's going to try to prompt inject and break into infiltrate into And I said I was like, when I was talking to him, "What do you
[72:54] think the chances are you'll be able to do this?" He's like, "Above 90." Above 90% and do you know I've spent billions of tokens hardening the system looking for prompt injection angles and and um even with all that he was still like, "Yeah, I I I'm going to get I'm going to get I'm going to get you." Wow, that's obviously that makes sense. >> Yeah, cuz they're you know non-deterministic. They're they are meant to be broken. One more thing, so we have like the Apple M5 Ultra, DJX Spark, RTX 5090. These chips, these local chips are
[73:25] getting better and better. The models are getting better and better that you can run locally. I want to ask you just one more time, like where is open source? Where is local inference? Where does it fit in in the overall architecture of somebody's workload? I I still think like these devices are for hobbyists and tinkerers, right? Um at the end of the day it is cheaper to you know, first of all, you can't you know, if you want to daisy chain like a bunch of these, whether it's Macs or DGX Sparks, you can't do
[73:55] with 5090s, but Macs or DGX Sparks, daisy chain them so you can run Kimmy. One of my buddy has like I think 10 DGX Sparks daisy chain so they can run Kimmy. But it's like, "Okay, but now the tokens per second is like tiny super slow, right? This system cost you like tens of thousands of dollars." With tens of thousands of dollars you could have rented or paid for you could have paid for an ungodly number of tokens at A at some margin from one of the providers, but B you could have also just rented an entire H100 or H200 server and run the model
[74:27] there, right? Um and so I think it's like again, like anyone who is a tinkerer, sure. But like the moment your volumes are big enough, the moment your requirements from your user are fast enough, you should have been using cloud, right? Um and also you know, we're in an interesting moment. I think there's an even stronger argument against these tinkering type systems now, right? Which is before, great, you know, these things are cheap, you know, you can do it locally. Um We're now in the age of shortages
[74:59] of 3 nanometer wafers, 5 nanometer wafers, and of memory, right? You've seen memory prices, you know, go up a ridiculous amount. You've seen the same wafer wafer volumes and shortages being really really capped and mobile companies having to cut orders. And there's a report that Xiaomi's going to produce 35% to 40% less phones year on year because they can't get the SOCs that they need because AI's taking all the wafers, right? Xiaomi's one of the biggest phone manufacturers in the world. You go down the list, it's like all these companies are going to, you know, probably do something similar. Um So, phones, PCs, and these sorts of
[75:30] devices are getting you know, stabbed. AI's taking it all. And the question is, okay, if if the world can only produce this many bits of memory, right? Um do I want it to go to a bunch of DGX sparks that are, hey, for, you know, a terabyte of memory, I'm only producing this many tokens, right? Or do I want it to go to, hey, for a terabyte of data center products, their memory, they're producing this many tokens. You're saying that's an Nvidia strategic decision. >> Not Nvidia. I think it's just like a a
[76:01] >> Or or whoever's purchasing. Yeah, whoever's purchasing, whoever's like, I've got I've got X dollars. If the memory's so expensive, I'm going to buy the thing I'm going to buy the thing that gives me the most tokens per dollar, which is data center class hardware because I can batch it out. And maybe it's not the person buying the tokens, maybe it's the person who's delivering tokens. Maybe it's the the semiconductor company like an Nvidia who's like, well, I can only get so many bits, let me allocate it all to, you know, Blackwell and Rubin rather than to DGX sparks. Or maybe it's like, you know, you know, it's sort of like it
[76:32] doesn't necessarily have to be one individual. I think capitalism does a good job of like in not a short term, but like in a medium term, allocating resources effectively. Um and so, you know, if that if that's done, you know, if if the world is in such a compute shortage, which it is, um, and I think the shortages are worse than ever, uh, because revenue is skyrocketing now. It's not just like people are wanting to build because AI is coming. It's like because people are building because AI is here. Um, you need to maximize the tokens you can produce for the available resources in the economy. And if memory memory prices
[77:04] will keep going up, right? Because the demand is there and the capacity isn't. And at some point this dislocates people, right? If, for example, memory prices double again, which I fully believe they will, um, then the cost of the memory in my phone would go up by about 80 bucks, right? Okay, if it goes up by 80 bucks, then my phone needs to cost $100. Does that dissuade customers? Oh, okay, what about DGX sparks, right? Um, the cost goes up by X dollars. Does that now increase the price? Now, that prices people out, right? Whereas, if you do that in the
[77:35] data center class, you know, the price goes up by X dollars. Um, the cost per token for, you know, what can be produced on the DGX sparks versus what can be produced on the Blackwells is is is off and and and sort of like, okay, the cost goes up for both, but the cost for the DGX spark goes up a lot more than it does the Blackwells. And now, tinkerers are more host, right? Um, and so, you see this already, um, Nvidia claims that they're not doing this, but there's been many reports that they're cutting supply to their gaming GPUs. And if you look quarter on quarter revenue, their gaming market went down.
[78:07] And yet, if you looked over the last quarter, you use Wayback Machine, you analyze Reddit, you scrape it, you look at it, you're like, wait, everyone wants more GPUs. People are complaining that, you know, they go to Micro Center or they go to Amazon and they can't buy a 5090. So, it's like obviously there's an allocation of resources problem here. One is that Nvidia just makes more margin on the data center stuff, obviously allocate your memory there. Um, and they work and it's not the exact same type of memory, but they can work with a memory manufacturer to direct their supply, um, their wafer fabric capacity, um, and or their or TSMC on their wafer capacity, right? Like sort
[78:37] of it's like as the world gets more and more constrained on compute power, etc. It means the allocation of resources goes to the most efficient thing and the most efficient thing is cloud resources, which can be shared, which means how many bits of memory you produce or how many square millimeters of silicon you produce versus the tokens that are outputted, you go to the thing that's the most efficient, which is these large data center class chips. And so this is also an argument against why these tinker tinkering type things fun, people who do them are an irrelevant amount of
[79:07] volume, but they're really cool people, they have the money because they they generally are just super smart people, so they have the money and they and they can, you know, they do this for fun, not for like an actual use case. But you think everything is going towards the data centers at the end of the day. This is maybe again, you said tinkers. I it's unfortunate. I'm I would love to know that, you know, eventually one day maybe my wife will kind of appreciate the fact that she can run in France locally. This I don't know, there's something special I don't know what kind of wife you got, bro.
[79:37] [laughter] I'm I'm happy your wife cares about these things. >> She doesn't. >> Yeah, okay. Okay. Yeah. I'm saying I would hope in the future she would. >> [laughter] [gasps] >> Yeah, fair. Fair. >> Yeah. All right, last thing on on this point, you know, Satya Nadella, who you interviewed, that was awesome by the way. He recently said the capex spend for from a lot of these hyperscalers, it seems foolish and specifically because if there's one algorithmic update or innovation, it completely breaks the
[80:07] math. What what do you say to that? I think it's cope. I think it's like major major copium. Basically like, you know, the whole the whole path was like they pulled the plug on the They were They were on track to be bigger than Amazon, they pulled the plug cuz like this is kind of a >> my 80 billion. >> Yeah, I'm good for my 80 billion, but he was going to do more. And then he pulled the plug and then open I had to scram scrambled and went to like and found Oracle capacity, found Uh SoftBank capacity found. Uh CoreWeave capacity, etc. etc. etc. Um you know, they used to have to be exclusively Microsoft. That
[80:38] exclusivity, but you know, Microsoft's like, "We're going to pull the plug on a lot of this. You guys go elsewhere. You You know, we have the right of refusal to sign these contracts, but you know, in general, you can go elsewhere." Then OpenAI signed all this compute with other people, Amazon, Google now, too, right? Um so, OpenAI's gotten compute from a lot of other people. Um And and you can argue, yes, it is a reasonable business decision on the risk and reward to say, "I don't want to sell compute to OpenAI at this margin, right?" Um you know, 35, 40% gross margin. I want to sell software at 70%
[81:09] margin. But they're not doing that, right? They're They're They're losing share. There's actually a report today that OpenAI's trying to build a GitHub competitor. Um because GitHub keeps breaking and we have a lot of problems ourselves with GitHub actions, with inference acts, cuz we run uh the largest fleet of GPUs that are constantly benchmarking with through GitHub actions, and it's like breaks all the time. They're They're helping us. They're building. It's getting better. The people that engineers working on it are great, but like it's like they have a lot of like weird infra, you know, and like decision-making processes because Microsoft is a behemoth. Anyways, um
[81:41] Microsoft's Copilot has been a complete flop, right? You know, there has been very little adoption. You know, why why is the intelligence workers, you know, why why is like Jeremy, again, like someone who's never programmed in his life, um you know, doing data center uh modeling using cloud code for this, right? It's like because that's the best tool, right? It's like that should have been Copilot, right? It should have been stuck to Copilot a year ago and like 6 months ago and a year Anyways, um so, I think like, you know, he missed the boat on OpenAI compute. He has
[82:12] missed the boat on a lot of AI software revenue. Um He's generally still growing really strong, right? I'm not saying Microsoft's doing poorly, but they're clearly getting completely like mugged on the compute volume side by like Amazon and Google, right? Uh if we look across the last year, about 100 uh gigawatts of data center pipeline capacity was added, which means data centers that are planned, right? Not doesn't necessarily mean built. Over the last year, a different number like a much smaller number was built. But 100 gigawatts was added to the pipeline. Half of that was Google and Amazon,
[82:44] right? And and Microsoft is not even in the top five, right? Of new data center sites under planning and construction and so on and so forth, right? So, that is an insane like sort of like you know, choice, right? Or it could be an insane choice, right? I think it's an insane choice because I think AI lifeblood is compute. And obviously obviously like I I I had the funniest the funniest tweet happened the other day where someone was like, "Holy crap, who could have expected Amazon's going to spend 200 billion dollars and Google 180?" And
[83:15] then I replied, "We did. We said it here." And then someone replied, "The whale watcher says that there's going to be whales. Wow." >> [laughter] >> Because they were implying like of course like I think computer is important because I work on it. But anyways, like I I think like it is a like a bit of a like an insane like view because Anthropic is so capacity limited, right? That's why you have all these cloud code instability issues. OpenAI's capacity limited. And you go down the list like everyone's capacity limited. Um you know, Microsoft could have been serving that. They could have figured out creative ways to get higher than 35
[83:45] to 40% gross margins on that. They could have been building software services on top of that. They're trying to do some of these things, but they're failing a lot. And like so it's like sure, these hyperscalers are spending a ton of money. Um and there is a risk, right? There is a risk that AI flops, right? I don't think that's happening. I think if Anthropic were to discover a method that makes AI 10x more efficient, which by the way they tend to do every year, right? For a given capability level, the cost is at least last year the cost fell
[84:16] about 1,000x. Um and for the year before that it was like 800x, right? Basically from GB Yeah, so so like you look at all these different models like across the year for the same benchmark level on many things, it's like a 1000x decrease in cost. So, it's going to happen again. Right? And it's it's actually happening, you know, a couple orders of magnitude every every year, right? Uh not just one, right? Because software, hardware, co-design, um data, RL, all these things make the models much better for a smaller size or, you know, cheaper cost, etc. etc. etc. New hardware, blah blah blah. Okay, something changes in
[84:47] paradigm. That's happening every year. That's happening every 6 months. It's happening every 3 months. New models are coming out with new capabilities levels. New models like Google just released Flashlight 3.1 or something like that. I don't remember the exact name like this week, I think. Yep. And and that model's like GPT-4 level for like it's like better than GPT-4. It's better Crazy faster. And it's orders of magnitude cheaper, right? It's like it's it's it's probably better than like it's probably like comparable to like maybe
[85:18] even like it's not quite GPT-5 level, but it's like these cost decreases are happening. And paradigm shifts like just mean that like okay, now great. The model that capability is cheaper. That means okay, I'm still going to spend this much to make this model or more, and the capabilities are going to be better. Right? It's so it's like yes, there's a curve of cost declines. But that also means scaling laws are still The only paradigm that could change is scaling laws break, right? But we haven't seen pre-training scaling laws break, right? Dario said pre-training
[85:48] scaling is still happening. Open AI has finally fixed their pre-training and it's happening. Google says it's happening. Why wouldn't I listen to the people with the best AI teams? You know, I don't want to listen to Satya. He doesn't have a good AI team, right? Um RL scaling laws are happening, right? Um and and you look at all these companies that are like there's no end in sight. It's like okay. If there's no end in sight, you think there's no end in sight. The models are improving at this ridiculous clip. Um 1000x reduction in cost for the same capability levels. And or for the same price, the capabilities are going up by
[86:19] it's a difficult way to measure how much capabilities are going up. I think by an infinitesimal amount because like things that were not possible are possible now. Um but like clearly and then the revenue shows this too, right? Anthropic added, you know, 4 billion in January and and something like 5 billion in in February, right? Which by the way, February's a shorter month, too. So it's like, you know, like the Anyways, it's like, you know, the the the the rate the rate is like adoption is insane. Um so why why on earth would, you know, you like Yes, there is probably some tail risk,
[86:50] but I don't understand it. What is the tail risk, right? Um So I agree with you. It it does seem a little bit foolish. I want to I want to wrap it up on one last question. The same question I asked you the first time we chatted. Uh who's going to reach artificial super intelligence first and why? Ooh. Ooh. >> [snorts] >> You remember who you said last time, right? >> I said OpenAI last time. Um and and OpenAI has really struggled in that time period, right? Um Google >> of late, maybe not. They've they've really like accelerated. So I
[87:21] would I would say like that episode I'd said I think OpenAI would struggle for a while, but then they were going to come back. Um and they did struggle for a while, right? Especially with Gemini and nano banana releases. They lost a lot of users. Um you know, they they or or rather their user metrics flattened and Google skyrocketed. Um now they've returned to user metric growth in January, February. Um and then Anthropic really towards the latter part of the year started skyrocketing in revenue growth and share. And if you model out the revenue, Anthropic will be bigger than OpenAI by
[87:52] April. Um you know, even though OpenAI's growth has upticked. So you know, right now And and probably this this gets released in in a few days or whatever, but like right now OpenAI's metrics are not that great and they look bad. But new model coming out comes out came out already, presumably. Well, we're we're specifically asking about ASI, which is almost like a a a cultural factor inside these companies. Not the revenue, not anything. I mean, I guess it's a function of it, but like who who is in the lead when it comes to
[88:24] recursive self-improvement, artificial superintelligence? Yeah, I I I still I you know, it's it's it's it's harder for me to say OpenAI this time. I'm on you know, people people accuse us of being on Dario's dick um all the time because of how much we talk about Claude code. Like we've been banging on the drums about Claude code for months and the death of software for months. Um I mean, even on your last show we talked about it, right? Like sort of like, you know, that was 8 months ago. So, we've been we've been banging on the drums.
[88:54] People are like, "Dylan, you're on Dario's dick." And it's like, "Yes, but I I I you know, I I I think the consensus answer probably is Anthropic for everyone that's like paying attention. I mean, the consensus is Anthropic. And so, for that reason, it's OpenAI. All right, OpenAI. Dylan, thank you. Awesome as always. >> Uh I'm going to get cooked for that one. This video right here is a personal recommendation from the YouTube algorithm. It uses some crazy AI to look at everything you've clicked on in YouTube, all of my videos that you may have liked, and try to determine the
[89:25] best possible video that you may like, and this is it. This is their recommendation. So, click on it and let me know what you think.