These 5 Infrastructure Giants Secretly Rule AI

[00:00] There are companies that get to decide whether your agent actually ships or not, and they're probably not the ones you're thinking of. I bet you're thinking of OpenAI or Anthropic. No, no, no. What about Cloudflare? Cloudflare ran Agents Week last month. What about Stripe? Stripe rolled out its Agentic Commerce Suite. Or Octa? Octa launched Octa for AI agents at the end of April and has continued to expand it since. Auth0 has been building out its AI agents documentation. Datadog has been quietly building out its LLM observability platform. None of these companies build models. None of these
[00:32] companies are on most teams' AI stack roadmaps, but all of them are going to decide whether your AI agent gets deployed in production. So, the model is one piece of the agent economy. The infrastructure companies, the ones that decide where the agent runs, who it's acting for, what it can know, what it can spend, and who can stop it. Those companies own effectively the infrastructure that controls whether an agent reaches production. And the control layer, the infrastructure layer that drives agent success, is where a lot of AI power is moving. So, let's
[01:03] dive in. First, I want to talk about compute and how compute is not the whole story. So, the easiest AI infrastructure story is the physical one, and I've told it in another video. So, if you're interested, dive in, right? I've talked about GPUs and data centers and power and memory and networking and capex. And that story is real, but the physical infrastructure only determines whether AI can be served at scale. If the power isn't available, the model doesn't serve, right? Once agents start doing work, the next bottleneck is whether the intelligence you can generate is
[01:34] governable. Where does the agent run? What does it remember? Who is it acting for? When does it need approval? What can it spend? And of course, who can stop it? Those kinds of questions must get answered by infrastructure, not by a model. So, compute is really important to scale agents as we start to talk about a full agentic economy, but it's not sufficient. Now, I've talked about protocols before. I have a whole article on Substack. I've got a recent video on it. I've talked about six key protocols for the agent layer recently. This video
[02:04] is about the layer underneath those protocols. Who is building the infrastructure that makes those protocols come to life? And the first layer I want to talk about underneath the agentic protocols is the runtime control point layer. That has Cloudflare, AWS, and Vercel. Okay, let's let's start at runtime and talk about that. Where does an agent actually live? So, a model is stateless. You send a prompt, you get a response, the conversation is over unless you send the history back next time. And that is
[02:34] perfectly fine for chat, but it doesn't work for an agent that has to remember what happened and wake up later and continue after a disconnect and run on a scheduled task and recover from a tool failure or stay connected to a user in real time. Real agents need a runtime with memory and execution built in. And that's why Cloudflare is building agents SDK. Every agent runs on what Cloudflare calls a durable object, a stateful microserver with its own SQL database, its own websocket connections, its own scheduling. The agent can call tools, it
[03:05] can serve tools through MCP, it can schedule tasks, it can coordinate with subagents, it can browse the web, it can react to events. And Cloudflare is not alone here, right? AWS is making the same control layer claim inside its own cloud with Amazon Bedrock Agent Core, packaging runtime and memory and identity and gateway and browser and code interpreter and observability into a stack. Vercel is coming at a little different angle with AI Gateway where the control point is model routing and budgets and monitoring and load balancing. So, there's some different bets here, but it's the same thesis. So,
[03:36] runtime is becoming a control surface in its own right. If your agent has durable work or deadlines or callbacks or streaming UI or tools or approvals or payments or state, and most production agents tackle those kinds of tasks, then runtime is something that you have to decide intentionally. It belongs at the top of your control map because it shapes the rest of the environment your agent operates in. Now we move to the next layer, the identity control point. Auth0, Okta, WorkOS, Entra, let's sort
[04:06] of understand what this means. It's a huge topic. In in ordinary software, identity means authenticating a user and authorizing that user against application resources. The user logs in, the app checks the permissions, the work proceeds. That model breaks when an agent acts on behalf of a person. The agent might be acting for one user or a single team or a company or another agent. The APIs it calls might span Google or Slack or GitHub or Salesforce. You get the idea. Approval often comes asynchronously while the user's away.
[04:37] And when the agent retrieves documents from a rag pipeline, only some of those documents are the ones the user's allowed to see. Auth0 is tackling this complexity by building this layer in public. Their AI agents docs cover user authentication, OAuth based API access, token vault, asynchronous authorization, and fine-grained authorization for rag. The mechanic is delegated authority with constraints. An agent does not get a broad permanent credential just because a user decided to sign in once. Instead,
[05:07] it calls APIs on behalf of a user. Similarly, token storage doesn't expose secrets to the agent. The agent has to ask for consent for sensitive or long-running operations. And rag queries only retrieve documents the user is actually authorized to see. Now, Auth0 is not the only player here. Okta, WorkOS, Microsoft Entra Agent ID, and AWS Agent Core Identity are all sort of converging in the same problem space. Because the dangerous agent in a company is not necessarily the most capable one. It's the one with very fuzzy authority.
[05:38] Where nobody can clearly say whether it's acting as the user, as the company, as the application, or as itself. And nobody knows whether the permissions persist across sessions or cover a class of actions beyond the original request. That's all manageable when agents draft text. It is not manageable when agents transact and deploy and refund and schedule and provision or make serious commitments on their own. A serious agent product needs a serious authority model. Who is the principal? What can be delegated? What can be revoked? What
[06:09] does the audit log show? If those questions aren't answered, your agent is going to hit a ceiling in any serious company. The full operator landscape, I put that over on Substack. You can dive into Auth0 and Okta and WorkOS and Entra Agent ID and AWS Agent Core Identity, and you'll get the whole list on Substack of what each one is trying to control and a clear protocol for how you pick an identity provider, which is an extremely impactful decision. So, if you want to dive in there deeper, that's
[06:39] where you find it. We're going to keep moving though to the data control point, and that's where we talk about companies like Snowflake or Databricks. Agents are essentially only as useful as the data they can safely interpret. A generic agent fails at data in predictable ways. It joins the wrong tables, it trusts the wrong column, it misunderstands a metric, it receives stale documents, it answers confidently from ungoverned context, it presents an assumption as a fact. Every one of those is a data control failure. The model is doing what
[07:10] it can with the data it sees and the data it sees hasn't been governed for agent use. Snowflake's bet on the data control point is very specific. Their Cortex Agents docs describe agents that work across both structured and unstructured data. Cortex Analyst handles structured queries, Cortex Search handles unstructured retrieval, and the agent routes between them. All of that happens inside Snowflake's governance perimeter. And so, the key thing that they want you to take away is that they are governing
[07:41] the distribution of meaning. Because a data warehouse is really where companies try to build a reliable meaning, right? A version of business truth, whether that's around revenue or customers or inventory or churn or margin or forecast. And agents make that semantic layer, that meaning-making more important, not less. What is ARR? Which customer hierarchy is authoritative? Which data is restricted? Which agent do I trust? An agent that can't tell current revenue from forecast revenue? That's not the one that should be drafting the board right? You shouldn't have that in production. What about an
[08:12] agent that can't tell public docs from confidential customer commitments? They shouldn't be answering support questions directly, should they? So, that's the problem space that Snowflake is going after. Databricks is going after a similar problem and making a parallel argument with its Mosaic AI agent framework around building, deploying, evaluating, and monitoring agents inside the same governed environment where enterprise data already lives. BigQuery and Gemini are the hyperscaler native version of this same move. These companies are doing more than
[08:42] adding chat to databases. They're trying to make the governed data platform the place where agents are allowed to reason and act. And if your business has a semantic layer, and almost all of them do, even if it's informal, your agent needs to be operating inside that layer's governance, not around it. This all matters because if your business has a semantic layer, if it has a data layer that helps you make meaning of business metrics and customer relationships, and most businesses do, then your agent
[09:13] needs to be operating inside some kind of governed permission structure. And that's sort of what Databricks and Snowflake are going after. Okay. Let's move on to the payment control point, Stripe and the networks. The moment an agent touches money, the control problem becomes really, really critical. Now, the protocols themselves, if you want to understand them, I I dive deep on AP2 and X402 and others in a different video. I want to focus now a layer above the payments protocol. I want to focus on the operator side. Stripe sits at the center of all this,
[09:44] not because of any single protocol. Stripe supports several of them, but because Stripe already lives in the middle of agentic commerce, payment credentials and fraud and disputes and risk and billing and subscriptions and issuing and treasury and merchant onboarding and all of the developer APIs underneath all of that. Agents make every one of the intersections I just named very valuable and Stripe is the company best positioned to play across that bundle and it is moving extremely aggressively to outline the pathways for
[10:16] agents to handle all of what I just described, right? Issuing, payments, authorization, fraud mitigation. Stripe is coming for all of that with agents and it's doing so because Stripe believes that the future is a larger internet economy with a lot of agentic commerce going on. And so it makes sense for them since they're on a mission to grow the entire internet economy to go after the agentic part of that economy and make sure that they enable that and set that up so it's easy to transact. That's the larger vision for Stripe and
[10:46] that's why you see them making these moves across all of these agentic pieces as an operator at the center of this network across multiple protocols. The card networks are operators in the protocol space but they have a very different set of incentives than Stripe. Stripe is looking to grow the economy as a whole but the card networks need to make sure agentic payments run on their rails and that's true for MasterCard and Visa and American Express. And so they're competing on something different from a protocol perspective. They want to prove and show that an agent
[11:16] transaction can run on their rails and clear the same institutional trust chain a card transaction can clear. And that's how they think about fraud and dispute and merchant onboarding and everything else. That's a different bet and the networks have a lot of infrastructure to back them up there. That's how card payments work, especially in the US. So, the payment ecosystem is dominated by operators who move above particular protocols and have differing motivations, but have the same larger pattern of behavior in that they're all trying to put out agent payment
[11:47] solutions. And the payments ecosystem for agents is going to have to evolve quickly. That's part of why operators are moving quickly because they know from experience with other payment products that if you don't quickly move to enable legitimate patterns of transaction, you get fraudulent patterns of transaction, and that becomes a big issue. So, the reason payments matter is that payments essentially are a form of institutional trust. And the company that is able to facilitate that institutional trust owns one of the most important control points in the agent economy. And if you are building in that space, if your agent is going to touch
[12:18] money, you have to be thinking about which of these partners you are working with in order to enable that experience. And right now, you know, unless you are in a special jurisdiction and it's extra complicated, the default is probably Stripe. In most cases, for most startups, that is the default. And if you're an enterprise, you may have your own payment stack. You know, Amazon has its own payment stack. There are others that do as well. Regardless, you have to be thinking about whether you are investing in your own payment stack and extending it to provide the agentic
[12:49] capabilities that you see from Stripe and others or whether you're going to partner with them and build customer experiences that are that rich because these big operators are playing in the space and providing the rails on which agentic commerce can run. Next big theme for shipping production agents, observation. This one's easy to underrate because it sounds like you're logging. That's not what this is. You're not just logging. Agents fail differently from ordinary software. They call the wrong tool, but they might call it with valid
[13:19] syntax. They ask the right agent, but they might ask it the wrong question. They can retrieve authorized data, and they still draw the wrong conclusion. They might complete a task technically while violating the user's intent. They can stay inside permission boundaries and still create a very expensive loop in terms of tokens. They keep retrying. Maybe they escalate too late. Logs by themselves don't catch those sophisticated failure patterns. What you need is a way to observe agent runs as work, not as API traffic. What was the
[13:50] goal of this work? Which tools were called for this work? Who authorized the action? Which data sources were used? Which policy blocked that action? Which cost was incurred? Did a human accept the results? Infrastructure companies can gain real power at this layer. Data Dog has been building LLM observability. It traces what the agent did end to end, prompts going in, responses coming back, tool calls, retrievals, and it connects all of that to the back-end services and user sessions sitting around the agent. LangSmith sits closer to the developer
[14:20] side. If your team is building on LangChain or LangGraph, LangSmith is where you trace the agent's workflow and run evals against it. Braintrust and LangFuse are taking very different bets here, right? Braintrust leads with evals, running quality checks against agent output. LangFuse is more of an open-source tracing option. And AWS, they're just interested in wiring all this together, right? Agent core observability supports open telemetry, which means you can send agent telemetry into CloudWatch and Data Dog and LangSmith or LangFuse from one single place. So, in this situation, the
[14:50] market is converging toward a single control plane where traces and cost and tool calls and evals and security events and business outcomes all get stitched into one operational view of what your agents are doing. One more thing, the kill switch. The kill switch is absolutely a product feature, and it has to be implemented at more than one layer to work well. And most teams aren't thinking about that. Because your agent runtime can cancel or pause the run, the identity system might revoke the credential, the gateway might block the tool call, the payment system might
[15:20] freeze the payment instrument or or hit a spending limit, a framework like LangGraph can interrupt a workflow before a sensitive notice hit, right? So, if the only way to tell your agent to stop is to just tell the model to stop, you don't have a kill switch. Look at all the nuance in what I just described. So, I'm going to go into kill switch architecture in the Substack piece as well. How to implement it at runtime, how to think about identity and gateway and payment and framework layers, so that tell the model to stop is not your only option. You can dive
[15:50] into that. But for now at this level, I want to leave you with what you can think about for this week. And I want to suggest, as I often do, that you start somewhere specific. Take a support refund agent, or a customer email support agent, or or or a claims agent, or an agent that reads conversations and checks usage. Pick a specific agent workflow. Then, understand these seven things. Where does the agent run? What is the runtime? Is it Cloudflare? Is it Agent Core? I don't know. Who is that agent acting for? It needs an
[16:21] identity layer. Is it acting for the company? Does it have delegated access? That's Auth0 or Okta right there. What can it know? Uh does it understand the data layer? Does it understand usage data and can it pull that in? Is that a Snowflake thing? What can it change? The tool layer. Where does it have read access? Where does it have write access? Where does it have approval? What can it spend? That's the payment layer, right? Does it have refund limits, approval thresholds? Are you working with Stripe? What gets observed? Look at the observability layer. Did the agent issue
[16:52] a refund that violated policy? Did it get tricked? Now you're talking about LangSmith, maybe you're talking about AWS or Datadog. And then who can stop it? Where's the kill switch? It can runtime cancel, can payment freeze, can identity revoke? What How do you do that? Fill all of that out for this one workflow. If you have a TBD on one of those rows, you need to tackle that before it gets to production. And you need to have owners for these because otherwise, someone is just going
[17:22] to say it's someone else's problem how to solve agent identity, and then you're going to find you never solved agent identity, and now you're launching, and that's a problem. Agents do not respect org charts. Your governance model has to compensate for that. I was talking recently with someone who leads a data team, and what she was observing to me is that there are cases where your agents can literally hack around permission structure inside your internal system because it was designed for humans, and it can come back, and
[17:52] the agent may have done a successful run, and it did so outside of the permission structure. And this is a problem for her from the data perspective because she needs to figure out whether the agent executing that run was authorized to work its way around the human permission structure, or whether that agent was acting inappropriately and showed data to the human that it shouldn't have. And there's multiple layers to that because like one, you could say, "Well, the agent shouldn't have worked around the problem." Two, you could say the human shouldn't have seen the data. Three, you
[18:23] could say, "You know what? Maybe the agent was goal-oriented in a positive way, but we need to give it better tools to solve the problem." Four, you could say the agent was allowed to do that, and we're okay with that degree of chaos. And there are a surprising number of companies that do that, but I think that that's going to become a riskier and riskier stance to take as agents get more capable. So, these are real challenges. You run into these control layer issues more as agents get more capable inside your systems, and it's often platform teams that face it first. If we step back for a minute, look, the
[18:53] AI economy is going to keep consuming much more compute over time. So, the hyperscalers are still going to matter. Nvidia is still going to matter. Models, they're going to continue to get better, but none of that decides how your agent actually shifts. None of the things I just named. Instead, the companies that decide whether your agent's successful are the ones that are building the layer that determines whether agents can act. And I've named a bunch of them here. I've talked about Cloudflare and Auth0 and Snowflake and Stripe and Datadog. That this control layer helps teams
[19:25] figure out which control surfaces they need to enact, engage, build on in order to actually ship agents this year. Now, if you want to see an example of how you work through a workflow with all of those control points named all seven questions answered, I wrote that out on Substack in full. Where the agent runs, who it's acting for, what it can spend, what what gets observed, who can stop it. If you want a copy it for your team, you can go grab it. Otherwise, thank you for tuning in. We are digging into agents because if we don't understand
[19:56] agents well, we are going to end up in a situation like my friend on the data team who's trying to figure out whether or not the agent that hacked around the authorization to get the data pull done is doing the right thing, the wrong thing, or something in between. These are stories I'm hearing all over the industry. They matter a ton. I'll see you soon. There'll be more AI news tomorrow. Cheers.