Black October Isn’t Just A PHP Bug. Attackers Want Your AI Keys.

[0:00] It's always a good day when a third of the internet burns down. This week has been the wildest week for cybersecurity that I can remember, and I have got the white hairs to remember a lot of cybersecurity incidents. If you've ever pasted an AI key into an app to use it, this week is about you. If someone steals that key and is able to use your API access to Claude or to Codex, they can use AI on your dime, on your tab. They can run it themselves, they can sell it to someone else, and you get the bill. Tokens are starting to behave like
[0:33] cash, and this week a lot of people found out what that means. A popular tool for connecting apps to AI models shipped a release that stole developers' keys. A hosting company says it's shutting down after another breach. A founder took their own product offline after warnings that the company's AI and payment keys were compromised. At the same time, agents are going through our software and finding bugs we've been sitting on for decades. One certificate company's secret key was the number four. We'll talk about that. Cyber defenders are calling this
[1:03] Blocktober for a reason. The same agents that make attacks cheap can work just as hard for you. There is an answer on the other side of the table. The question is whose side the best intelligence is going to be on. By the end of this video, if you use AI apps, you will have specific things to do this week. You're going to have takeaways. What to update, which connections to cut, what to do when a service you trust gets hit. If you build with AI, I'm going to show you how I would put the strongest defender you can get across your setup. I think
[1:33] we're going to see much, much more of this because agents are native to computers in a way that humans are not. So, agents can keep investigating the systems we have built while we struggle to work out what needs fixing and who can fix it. So, let me expand on what I mean by agents being native. Agents are native to computing because LLMs are tool-using inhabitants of our computing systems. I do not live, code, and breathe code 24 hours a day. No engineer does. AI does. And AI never gets tired.
[2:05] That's what I mean by agent native. And so, in that world, when agents are native, we need to put our best agents, our best intelligence on the defensive side of the ball, so that we can be sure that we're able to protect ourselves. So, how did all of this come up? What is the big story here? On October 21st, PHP released fixes for two critical vulnerabilities. Now, PHP is software that runs many, many websites across the internet, including sites built with WordPress. And so, what is the hack itself? Like if you say, "What is wrong
[2:35] with PHP?" What happened was that the attackers figured out how to get PHP to run without permission on another person's website. So, with Apache's mod_php setup, an attacker can get code to run in front of the website's own application code before the website's own application code even gets started, which means they can sort of hijack that website's platform. The UK and European security agencies have responded by telling organizations to consider
[3:06] isolating or taking down affected systems that they cannot update and that they cannot apply mitigations too quickly. This is as as much of a nuclear DEFCON situation as it get. You are literally getting told by the government to interrupt your business because something underneath your application is unsafe for the general population. This is a really big deal. So, PHP is perhaps the most widespread one, but there are other ones as well. Another big one is eMudhra. Now, eMudhra is a company that
[3:36] issues certificates used to establish whether you're connecting to the right website. Now, eight of its private keys were trivially weak. Security researcher Scott Helme recovered them in less than a second. And one of the supposedly secret keys, which is supposed to be hundreds of digits long to ensure it can't get hacked, was the number four. Just four. So, these keys are supposed to be very long random numbers, but the company had put the keys into use in such a way that
[4:07] any laptop could recover them almost immediately. And because of the way the internet is structured, eMudhra was considered a root certificate issuer and was able to issue certificates that were automatically trusted. And so, that chain of trust essentially allowed one company's mistake to cascade across the internet. And this is all very easy to get. Helms says he got Claude to recover the keys with a single prompt. This is why I take this so seriously. We have layers of software that look normal, that have been around for decades, that
[4:37] pass all of our checks, and have been trusted for years, and yet an agent will just kind of dig around and find vulnerabilities that we've all missed. And that means the major players on the internet have to play defense. An example would be Mozilla. Mozilla says it distributed protection to standard Firefox configurations within just a couple of hours of receiving the report. Cloudflare immediately got on the ball to protect websites that are behind Cloudflare, and lots of other major players followed. Vercel got involved, as well. These players, because of the way that
[5:07] the internet is configured, act as fronts or as serving layers for a lot of websites, so they're able to put up what I would describe as shields that allow websites to be protected even if individual website owners have not yet updated their PHP or have been compromised in other ways. Microsoft has also gotten involved. Microsoft has documented a China state-aligned group using an AI agent framework called Artek to target hundreds and hundreds of organizations. Most of that targeting has failed, but some attacks have reached privileged access to cloud
[5:38] systems in just a couple of hours, as well. Uh so, these attacks go as fast as as the defenders can can defend, right? And Microsoft has been dealing with this for a while, right? Anyone know? Microsoft is an older company. Microsoft has been working on cyber defense for a long time. And one of the things that they have realized is that they have a position in the ecosystem that allows them to be an early warning signal for the rest of the internet because they're such a broad target surface. One attack vector that Microsoft identified is a proxy attack,
[6:08] right? So, attackers use credentials that customers have previously trusted to attack other surfaces. If this all sounds very involved, like it's okay. Like you may not be in cyber defense or cyber security. We're going to get to the part where we talk about individual actions. The reason I'm telling you this story is I want to give you a sense, sort of panning the camera across, of both sides in this new war that we're all in on the internet. The offensive side and the defensive side. Now, what goes inside? How does this work? When an
[6:39] agent does something that's scary on the internet, what does that look like? Really, it's very simple. An agent can read unfamiliar code, it can notice something suspicious, it can write a test, it can run it, it can discover its first explanation is wrong, and it can keep trying. And you know what's interesting is that this is almost exactly the same set of actions for cyber offense and cyber defense, which is why some of the safeguards intended by model makers to stop cyber
[7:10] offense are tripping up cyber defenders and forcing them to use other LLMs when emergencies happen. And so, it's really, really important if you run a company and you're in cyber defense, to right now go and make sure you have access to the best possible cyber defense models. It is not true that you need a weaker model because you're afraid the model may go rogue and misbehave in cyber defense. The correct way to read this is to say you want the strongest possible model on your side in cyber defense because the other guys are
[7:41] going to have the strongest possible model they can find. Now, humans have been attacking websites for a long time. Hacker culture is real, and a very good human engineer can do the same things I just described an agent doing, but they don't do them as fast, and they can't do them forever, and they don't scale up. Humans have to eat, humans have to take breaks. If you want to mobilize a major attack with humans, it used to take a lot of humans. That cost has shifted. That cost is tied to computation, and one of the characteristics of Blocktober is that we
[8:13] are tying some of these attacks to individuals with fleets of agents. And so, when you think about where we are now, you have to think about a world where individual companies that are getting targeted have to assume, not necessarily they're the target of an expensive, large-scale operation, but really, that they're the target of a cheap, very well-equipped operation that may be run by as little as an individual. One person who wants to hack and be a a bad actor on the internet.
[8:43] That lowers the cost of attacks, and that explains part of why attacks have proliferated so much. The other thing that we need to call out is that maintenance is hard. One of the darker sides of the story here is that PHP's maintainers had recognized signs of one of the problems all the way back in 2024, 2 years ago. The fixes are hard, right? They involve compatibility risks, they involve limited expert time, um and you had to choose which compromise or trade-off to accept to roll out a fix. And the PHP team kind of stalled that
[9:14] work and restarted it, but never finished it. And the problem with that is that you don't have the time that you used to have to go through that extended trade-off conversation if you're trying to roll out a fix. AI is taking away the time cyber defenders had to have 2-year-long conversation about vulnerabilities. It just isn't going to work anymore. You have to be ready now. As just one example of the kind of scale we're talking about, Anthropic has reported more than 29,000 candidate vulnerabilities from just 6 months of
[9:46] scanning important software projects. About 6,000 of those so far have been manually reviewed and triaged. There's a ton of potential exposure out there, and we could be looking realistically, we could be looking at millions of undiscovered or unpatched weaknesses across the software people use. You know, my analogy comes from the jungle. I spent time in the jungle as a kid. In the jungle, we talk about all the species you can discover, right? This is kind of like that. If you draw a square kilometer in the jungle and you look at
[10:16] it, you're going to discover new species probably. If you draw a square kilometer on the internet, metaphorically, you're going to discover new bugs. And AI is giving us that hyper awareness, that supervision that allows us to see all of those bugs in ways that we couldn't before. And part of what makes this challenging is that our ordinary actions can make us vulnerable. We inherit some of this through choices that seem really reasonable. We connect a useful app to our files or our calendar, and that app can depend on a hosting provider and software that we've never heard of. And
[10:47] so, a weakness anywhere along that path can expose us to information or access that we would care about, but would never know about. So, when I talk about what software can reach, that includes the ability to reach into your documents, right? Like, these are situations where people's information can be exposed in ways they never realized. And now we get to one of my tips for what to do about it, right? If you're concerned about this, open up the connected app settings for any important account you have and inspect what you've authorized. So, Google's a great example. In Google, you can review and
[11:17] remove access granted to outside app. Read what each connection permits, basic profile information, reading files, changing them. Keep the access that you need and then prune out what you don't, right? Basically, this is a basic cyber defense principle. You are reducing the surface area for attack. If you'd like an AI tool to help with that review, give your AI tool of choice, Claude, Codex, whatever you want, give it the app names and permission descriptions and ask it to dig in and to inspect your settings to find areas where you're
[11:48] vulnerable and to help you look through your system and ensure it is as safe as possible. One of the nice things is if you can find vulnerabilities with a single line prompt, you can also find a safe, secure audit with a single prompt, and that's a good thing. Everything in this video is on Substack, plus two prompts you can copy straight into your AI. The first one is going to walk you through all of your accounts and your devices and it's going to make the AI write down everything it checks so you have a track record of your audit. And
[12:18] the second one is actually going to recheck it when the next breach hits, which at this point is every single day. If you want your AI to be able to read this whole thing and have these prompts on hand, plug it into my library. Nate's Library MCP connects the AI you already use, like Claude or ChatGPT or Gemini, to all of my posts, my guides, my prompts, so you can pull them up right where you need them and right where you use AI. Links in the description. These same vulnerabilities can exist at company scale. I'll give you one example here. Light LLM connects applications to
[12:49] AI models and it has had multiple hacks in the last year. Its October disclosure describes a release that contained credential stealing malware. It had already dealt with a compromise back in March. If you installed the affected release, this is an example of how compromise happens, right? The immediate advice is to treat your environment as compromised, follow containment instructions, and rotate those secrets. You need to deal with the access that already occurred and assume the worst. And that's one of the things that's true whether you're dealing with stuff at company scale or individual scale. Let's
[13:20] work through an illustrative environment cuz this can feel very complicated. Let's say your customer application sends requests through an AI gateway, which uses light LLM and holds keys to model providers. Automated jobs build and deploy it, and several other applications are using exactly the same gateway. The cyber defender's job is to establish what happened across your whole environment and what you need to do about it. Then the disclosure arrives. So you inspect the source repository of code, it looks clean. You inspect the running gateway and someone has already upgraded it. It would be
[13:50] easy at this point to say, "You know what? The immediate problem is fixed." But light LLM's actual disclosure says the official source tag was clean, while the distributed release contained malicious files. So when you read the source repo, that doesn't establish what was inside the package your machine installed. I know that's complicated. So the cyber defender needs to ask, "What code actually executed?" Suppose the deployment history shows that the malicious release ran in a build job that finished a few hours ago. There is
[14:20] no running process left to stop. The production gateway is clean now, but the build job's identity did have permission to retrieve production provider keys. And that changes how you respond. You then have to investigate the access that job could have exposed, even though the job is done now. And this is the kind of test I'd use to choose the model that does the investigation with you as a cyber defender. So how do you do that investigation with AI help, right? Here's how I would think about picking a model for that investigation. Give
[14:51] candidate models the same evidence and query tools in a safe reproduction of an incident, where you already know the story. Keep the answer out of the instructions, of course, and ask yourself, "Does the model solve the problem? Does it stop at the clean repository? Does it dig deeper and find the build record? Does it connect the build identity to the credentials, like we talked about?" In other words, can the model dig and help you discover quickly? Because part of why we need models is their speed. And if you're looking to pick a place to start, I
[15:21] would start with the strongest models available to you for code understanding, reasoning across systems. I would dig in and have them follow a real investigation through multiple tool calls. It is worth spending money on this because the alternative can be really scary as a business. So, whatever it takes, find a good strong model, compare the time and cost of getting to an answer where you kind of know the correct answer, and make sure that you are not choosing to save a little bit on inference and risk getting a completely wrong answer
[15:52] in time pressure because you're using a weaker model in cyber defense. In general, this is not a place to be cheap. It just isn't. Let's get back to our example for a minute. Let's say the build job can reach the provider keys which we talked about. An AI API key is kind of like cash, right? It gives someone permission to ask a provider to do work and charge you for it. So, if you steal the key and you can consume that computation, that's a fantastic thing if you are a bad actor. Someone else pays for the use for work. That's the cash argument that
[16:22] all of this is building on. And that is what you have to do as part of your investigation. So, if you even suspect AI keys are touched and you are measuring how good AI models are at doing cyber defense, make sure you're testing whether they recognize that that's a risk factor and whether they go and look at the financial implications, look at any evidence those keys have been used or billed. That's going to be an important part of the story going forward. We're going to have to have quicker, better ways to catch and lock
[16:52] off compromised keys. None of this is theoretical. The founder of a little AI startup called Post said on October 21st that he was shutting down his service for a few hours for customers because he was receiving warnings that the companies Anthropic and Stripe keys had been compromised. This was part of the railway hack, which we have not even gotten a chance to talk about. And I think as a founder he handled that really well. You are in a situation as a founder where you may not have been the one who was hacked directly, but you are implicated in the hack because your
[17:22] environment was compromised by a service you depend on. And you have customers yourself that are depending on you who have incomplete information, who have maybe reason to believe that access has escaped under your control, and stopping the service can be a responsible way for you to get back in control, figure out what happened, document everything, and make sure that you are doing your best for your customer. Now, here's the risk with AI and cyber defense. You need to make sure that AI tools themselves do not compromise you
[17:53] inadvertently in cyber defense. And for that, we turn back to the Light LLM story. Light LLM says it observed situations where Claude code put a user's GitHub credential into a response that could end up in logs. The company believes attackers were able to obtain credentials from its internal deployment logs, and it was still investigating parts of that attack sequence. So, if you're thinking about your risks, yes, the the attacker shouldn't have gotten in anyway, but if they got in, Claude should not have
[18:23] exposed that handle in its logs. And that is also a security flaw that you have to investigate. Another example that I've heard that is very 2026 is there are situations where websites are asking agents to disclose that they are agents, and agents are doing so by modifying the URL that they are posting when they go and visit web pages. And when they go and do that, they may modify the URL in ways that expose your personal information, things like
[18:53] embedding a representation of your email in the URL. Someone was saying that they went and asked Claude, "Where have you exposed my personal information through URLs?" And Claude came up with like 10 different leaks in the last couple weeks. So, these are things that LLMs may do because they are asked to do so by websites who think that they are being safe by asking for agents to declare themselves and inadvertently they are creating a security nightmare. It's a complicated situation, which is why I come back to saying talk to your
[19:23] AI, ask your AI to audit your setup, and make sure when you're thinking about cyber defense, you think about it in terms of multiple layers, and you think about it in terms of making sure that you are frankly a little bit obsessive and a little bit paranoid about securing your environment. I think paranoia is an appropriate response to Black October. I spent a lot of time talking about the cyber defense side. I don't want to lose the customer impact. As a customer, we need these consequences explained clearly. We need information about where we were exposed. We need to be clear
[19:54] about whether access has stopped. We need to be clear about what we need to do. A company may still be investigating, but it needs to explain what it knows, what's uncertain, and it should not just say something like we updated your software, we think everything is fine. You need to be really, really transparent as a company during this time to earn trust. Make sure if you are affected as a customer, that you are updating your password. Make sure that you are obsessing over checking in with appropriate government
[20:24] authorities if you feel like your ID or something else related to your government identity has been leaked. Make sure you're talking with the credit bureaus in the United States if it feels like your credit may be affected. These are all things we can do that most consumers don't do that are part of cyber defense for individuals. There are other things that we as individuals can do as well. We can apply the same thinking around security and reducing our service area to browser extensions. You should be looking at your browser extensions. Chrome lets you limit site access for supported extensions,
[20:55] sometimes to particular sites or when you select the extension, you can choose kind of what it touches. You should be reviewing that. You should not just give blanket permissions to a bunch of extensions. Reduce your surface area. I also want to see cyber defenders reducing what attackers can reach before the next bug. Anthropic recently published work across 25 different projects and identified almost 500 opportunities to reduce exposed code and unsafe configuration interactions. For
[21:25] example, one configuration can look sensible by itself, but it could silently weaken a security check when combined with a second configuration. And that sounds sort of theoretical, but anyone in cyber defense will tell you that those sorts of chain relationships are very, very common and make it easier to chain together a working exploit. As an example, in OpenSSH, which provides secure remote access to computers, Claude helped identify complicated signature handling code that could be reached before a user authenticated even
[21:57] when the relevant algorithm was not enabled. That work led to some patches and changes, right? Shipped in version 10.6 that removed a bunch of the code and an authenticated client could previously reach. Fundamentally, fewer lines of code, fewer parts of the program are surfaced and available for a visitor who is not logged in, which makes the entire experience safer. Now, this is why I don't think telling people to patch bugs faster is an adequate response to what's happening. Now, that does need to happen. Patching is necessary. The PHP example's great, but
[22:28] it's not the only thing we need to do. We need to understand our systems. We need to recognize in our systems which chained relationships make exploits possible, and AI is a powerful tool to understand that. And then we need to dig in and make sure that we can verify attack vectors that are real and start to reduce those surface areas proactively even before bugs are identified. So, if you're running these services, you need to put your best reasoning to work. And if you are in the C-suite, this may take funding. In fact,
[22:58] Bank of America took a cost or a charge of a couple of hundred million dollars just in the last quarter to hire cyber defenders to fix bugs like this. This is a real line item that big companies are spending on because they cannot afford the consequences of a hack. I'm going to come back to what I said at the top. Agents are native to computers and they are going to expose us as people who have written a lot of bad bugs for a long time. And the agent's ability to investigate, to test, to keep following connections
[23:28] is both their greatest strength as a cyber defender and also what attackers use. I want us to be using the best possible intelligence to make sure that we are giving cyber defenders the tools they need to go to work and protect all of us on the internet. I would be remiss if I did not close with personal tips. So, if you are an individual, here are five things that you can do to protect yourself. Number one, update your browser, your phone, and your computer operating systems, and your apps. Finish
[23:58] any required restart so the updates actually take effect. Ask Clouder Codex to check for updates across affected services whenever you hear about a breach. Number two, open the connected app settings for your important accounts, right? Start with your email and your files. Remove any connections you don't use and check that the ones you keep are only using what you would permit them to do. Reduce their surface area. Number three, review your browser extensions. Remove the ones you don't use. For the ones you keep, restrict which sites they can access wherever
[24:28] your browser supports that setting. Number four, use a password manager to give each account a unique, hard-to-remember password and turn on multi-factor authentication wherever you possibly can. That's that additional sign-in check that helps protect an account if the password gets exposed. Also, add a voice passphrase for family so you don't get voice deepfake. And finally, number five, when a service sends a breach notice, check what was exposed and immediately act on it. Don't stick your head in the ground. Change
[24:58] exposed passwords everywhere, review and revoke affected app connections. If you're a developer, you got to rotate those API secrets, and check the notice for any further action your information requires. And if you are a builder, bias toward transparency with your customers. It will be worth it. I promise you. If you defend a company, do not let this crisis go to waste. The Substack works up a full investigation start to finish using the Light LM compromise. That can be a test bed for you. Your team can run it in a test environment this week against different models to assess what
[25:28] works for you. There are also talk tracks for the four conversations I know you need to have right now. The board, the CFO, engineering, and your vendors. You need to be having those. If you're wondering where to start, I have talk tracks for you. All right, grab the link in the description, and I'll see you next time.