Time to Exploit is Negative: AI Broke the Patch Cycle | Dan Lorenc

Dan Lorenc 0:00 How could the average time to exploit be negative? The ability to weaponize knowledge about a vulnerability is now a commodity. Anybody with a cheap model, anybody with an open model can take the knowledge of a vulnerability and build a POC or build an exploit for it in minutes. You're only limited by token cost. The bottleneck is not the people finding bugs, but it's always a bottleneck to fix the bugs. I think AI is the first time we actually have a chance of speeding up the fixing side.

Conor Bronsdon 0:31 For 20 years, security has run on a comfortable assumption that attackers and defenders found bugs at about the same speed, and you could triage the scary stuff, defer the low severity noise, and probably sleep at night sometimes. That assumption is dead in the era of LLMs. My guest today argues that time to exploit has gone negative, exploits land before vulnerability is ever disclosed. And as we've seen with recent hacks of Bitcoin wallets and so much more, that even very low priority vulnerabilities can be found and exploited. Frontier models can read code, they can reason across dependencies, and they can surface chain to zero days at machine speed. Humans are struggling to keep up, and we're going to talk about how to solve that. Welcome back to Chain of Thought, everyone. I am your host, Conor Bronsdon. If these conversations are useful to you, take a second to like this one and subscribe. Hit the bell, too, if you're on YouTube, so it actually tells you where the next episode drops. I appreciate you all. Over the last few months, a wave of AI-powered vulnerability discovery tools have hit the market, and the reaction from enterprise security leaders and from Washington, D.C. has been loud. Dan Lawrence thinks discovery is the part that's getting solved, and that the industry is now staring at a much harder problem, getting the fix deployed before the flaw goes public. To attack that, he has pulled together more than two dozen companies from JP Morgan, CloudFare, Cisco, and Kindrel, who we've previously talked to, under a new coalition. Dan is the co-founder and CEO of ChainGuard, but before that, he spent years at Google building the backbone of software supply chain security, created SigStor, and in June, his team launched Athena, a coalition built for the era where AI finds vulnerabilities faster than maintainers can patch them. Dan, great to see you. Welcome to Chain of Thought.

Dan Lorenc 2:16 Thanks for having me on. And great, scary intro, yeah. [2:22] Conor Bronsdon: [OVERLAP] Yeah, I'm glad you enjoyed it. I do find that this topic can be a little scary for folks, and I'm hoping you can both level with us on some of the details, but also maybe give us some cause for not just alarm, but for positive thinking here. So we'll convert to that. But before we jump in, I do want to say a quick thank you to our presenting sponsors for season four of Chain of Thought. Thank you to our presenting sponsors for season four and this episode, Walrus Memory. You'll hear how they are solving and memory challenges with portable, verifiable memory. throughout the season, and Sphix, which provide secure and enterprise-ready webhooks as a service. I'm excited to contribute to both their open-source codebases and share more about them as the season progresses. Check out Walrus Memory at walrus.xyz.cot and get $12,000 in credits for qualified startups at link.svix.com.cot. [3:19] Dan Lorenc: [OVERLAP] And now let's dive into our conversation.

Conor Bronsdon 3:21 But Dan, let's talk about this vulnerability landscape and how it has changed. You've been making a version of this argument since at least the spring, and the market has, I think, moved fast to recognize that you're right about this. What does the time to exploit has gone negative actually mean for how teens should defend their software today?

Dan Lorenc 3:42 Yeah, that's not my data, right? This is coming from a couple of independent reports from Mandiant, from CrowdStrike, from a couple of these other vendors that are centered around vulnerability detection and response, and this is what they're seeing. It's a weird stat, so I like to kind of unpack it and at least give my theory on how that could happen. How could the average time to exploit be negative? You know, we've had zero days forever. Zero day is something that's exploited before it's disclosed. And so we've always had a category of them that are exploited before they're disclosed. But then there's a big fat tail on the other side of that balancing out the average of things that are disclosed and never exploited, or things that are disclosed and exploited years later. That's the part that's been squeezed, I think. You know, the ability to weaponize knowledge about a vulnerability is now a commodity. Anybody with a cheap model, anybody with an open model can take the knowledge of a vulnerability and build a POC or build an exploit for it in minutes. You're only limited by token cost. And so gone are the days of, you know, 270 days or that kind of thing for people crawling through the massive backlog of known vulnerabilities to pick the ones that they're going to exploit and exploit them 180 days or 270 days later. If it's going to be exploited, it's going to be exploited in weeks. And so that part shrinks down and then there's this massive swath of zero days on the other side and that's how averages work. The average shifts negative, which sounds crazy. Um, but really what it means is everyone has to go faster, uh, all the time. Um, AI is out there. AI is allowing everyone to go fast. And if defenders don't pick it up at the same speed and figure out how to modernize and, uh, decrease their patch cycles, uh, to something that fits into one of these exploit windows, then you're just kind of leaving the front door open. [5:27] Conor Bronsdon: [OVERLAP] Yeah, I, I have to say this is fairly concerning, right? We've seen [5:32] Dan Lorenc: [OVERLAP] Yeah. [5:32] Conor Bronsdon: [OVERLAP] the first autonomous AI attacks. We're seeing open and anthropic talk about their models basically doing felonies with cybersecurity while they're. [5:43] Dan Lorenc: [OVERLAP] It was a weird weekend. Everyone was bragging [5:44] Conor Bronsdon: [OVERLAP] Yeah. [5:45] Dan Lorenc: [OVERLAP] about how many crimes their models had committed when they weren't looking.

Conor Bronsdon 5:49 I think my favorite tweet about it was this joking felony bunch chart. People started to post. So, I mean, this is a new era. It's a new world. How do people need to react to this? What do we need to do differently in this new reality?

Dan Lorenc 6:04 Yeah, I think there's a couple new realities, right? Hitting at once. And one of them is that people hadn't really been keeping up with vulnerability patching to begin with, right? Everyone knew you had to do it. No one really did it at scale. No one really did the greatest job at it that they should have been. Um, and that was okay for a while, um, because attacking, uh, infrastructure, hacking networks, that kind of thing. Um, there weren't that many bad people in the world. And, uh, so we had a couple of different bottlenecks there kind of all at the same time. One was that the number of interested parties to actually go and carry out these attacks wasn't that high. Um, and there were pretty sophisticated attacks, even the simplest ones. Um, and so even if you weren't doing the best job, um, you were probably able to sleep at night. You probably weren't going to get breached, that kind of thing. Um, AI has, uh, democratized in a bad way, I would say, you know, the ability to carry out some of these attacks. Um, it's not, uh, you know, weeks of research, months of learning how to do these kinds of things, uh, years and years of training to be able to do this now. Um, anybody with basic commute, computer for fairity and that kind of thing, If you get your hands on a model that doesn't have restrictions, can do this. And so the supply side of people with bad intent that are capable of doing bad things has gone up dramatically. And so that's one reality. And the other reality is that now there are a lot more known vulnerabilities out there and a lot more holes for people to poke through at the same time. So what was working before isn't going to keep working because it wasn't even really working before. You were just going by the fact that there weren't that many bad people around that could do basic kind of hacking like this. And now, even if you were at the top of your game and keeping up with all the vulnerabilities and stuff that you were supposed to be doing, that's not going to work anymore either. So there's kind of like a big massive tech debt payoff that has to happen just for the first fact here, which is that they're tearing out of taxes a lot easier. And then a massive change in the way we all build software and deploy software, I think to deal with the second one, which is that there are way more bugs in the software than anyone realized and a lot more of them are dangerous than anyone realized. So even if you get on that treadmill, we always talk about, you know, patching being a treadmill, you're never done. Software ages like milk, not wine. You know, all of these different analogies. That's not going to be enough either in the future. And kind of the combination of those two factors are terrifying people.

Conor Bronsdon 8:36 Yeah, it feels like A, the speed with which vulnerabilities can be found has really broken these security paradigms. And then secondly, how good models are at vulnerability chaining has really just completely rewritten how we have to think about defense for all the code we're developing. [8:55] Dan Lorenc: [OVERLAP] Yeah, the chaining piece, it's important. I think it's also misunderstood in a lot of cases, right? Like if you took some commodity cloud scanner, any one of these things that looked up known vulnerabilities, you mentioned it before. Yeah, you've had to triage away the lows and the mediums and you deal with the highs and the criticals. And those scans would come back with, across a large enterprise, millions of results, right? Like these are not small numbers. And these models don't just go around picking 17 random lows and mediums and then saying, got you. You know, it's a lot lower level, it's a lot finer grained things that they're noticing that maybe didn't even have a vulnerability filed about them in the past. The analogy and story I like to use to kind of explain what these models are finding and what they're doing is actually, it's closer to this blog post from the Google Project Zero team a while ago, I think it was 2022. But one engineer at Google Project Zero, who are some of the best security researchers in the world, Spent six months during the pandemic with an iPhone, a couple debugging tools, and a computer. And that was it. And basically built a zero-click radio transmittable remote code execution full root takeover exploit. Where the demo was you press a button on an iPhone, it immediately takes over every iPhone in radio distance and can spread. Um, you know, worst case scenario. And it's an incredible blog because it's kind of, it reads like a daily journal of like, all right, here's what I did. Plugged in the iPhone today, played around with this system, found something that looked kind of weird. Probably never would have like been a vulnerability by itself, but found something that looked kind of weird. Kept poking, went down this chain of ideas and got stuck and then came back up and tried a different direction until eventually they had a complete root takeover. So it is kind of that chaining and that long context window and the ability to think in abstract for a very long period of time and try a lot of different ideas that's letting the models do this chaining. It's chaining a bunch of very tiny flaws into something that is a real working exploit because that's the objective you give these models. It's not like a random lottery number where you just stick your hand in and pull out a CVE from six years ago that someone forgot to patch. [11:01] Conor Bronsdon: [OVERLAP] There is a silver lining to all of this, though, which is that while attackers now have access to these capabilities, we have new capabilities for defenders as well. And I know that's been part of the approach that Chainguard is taking with Athena and everything [11:15] Dan Lorenc: [OVERLAP] Yeah, [11:15] Conor Bronsdon: [OVERLAP] else you're doing. Can you tell us a bit about how you're responding and the bets you're placing around fixing AI discovery vulnerabilities with a coalition instead of a single vendor approach?

Dan Lorenc 11:26 well luckily there are a lot of people with these tools and Anthropic and OpenAI both taking similar approaches, but to give these tools that are good at vulnerability discovery to defenders first. And they are all spending a lot of tokens and a lot of time scanning and looking for exploits in the software that they build and use. Now we sit in open source software, which is a large area of software, but it's still a subset. Open source by stats are anywhere from 90 to 95% of any modern application by line of code, but everyone is using kind of the same open source at the end of the day. But when these folks point these models at their applications, it'll find vulnerabilities in their first-party code, but it'll also find vulnerabilities in the third-party code, and often it's chained between the two. And the only way we as an industry can, you know, combat that whole terrifying stat around the mean time to exploit being negative is to try to get the mean time to remediate even further negative, or even negativer. And so these folks, and you can see it as well on Anthropic's website on the Glassware Transparency Tracker, But they've accumulated massive databases of possible chainable exploitable vulnerabilities that are just sitting there and haven't been disclosed yet. Now attackers are probably racing everyone to and scanning the same things and finding a lot of these same ones. But it's a very large pit to be scanning in. And the whole approach here is how many of these can we patch or remediate and get to as much of the industry as possible before they do become exploited. And at this scale, and by getting enough people involved, I think we have a decent chance of changing the numbers pretty significantly here.

Conor Bronsdon 13:07 So I know in the last month that Athena, the coalition that you have built to help solve this challenge, has processed more than 40,000 AI discovered findings. I'm sure it's gonna be more than that by the time this episode comes out. What happens when a member submits a finding to a fix actually landing upstream? What's the process here?

Dan Lorenc 13:27 Yeah, there's a lot of work in between, right? Because these findings that people submit, it's not one individual vulnerability, right? You know, a lot of these are, you know, full exploit chains or a full report from a model. The basic prompting sequence to get really good results out of these models is you give it a full running application in a sandbox or something like that. You give it debugging tools, you give it access to the source code, and you say, you know, just break this app, or bypass the auth, or get into this database. Very simple prompting, but you give it enough tools to actually be able to poke around and find issues. And then, you know, one of these submissions is then a whole report of here's what I did, here were the different flaws I found, or exploited, or misconfigurations to get to that objective. And in that case, it'll find a whole bunch of different paths. So every one of those could be many vulnerabilities inside of it at the end of the day. Any one of them might have been enough to stop that chain, but if you ran it again and only fixed that one, it probably would have found a different chain. So there's a lot of work to go validate, parse, separate these things into unique, patchable pieces of software. And then there's a lot of de-duping after that too, because if you get a hundred of these on the same application, it might not be a hundred different unique things, right? It might be, it might de-dupe down to something a little more manageable. So in some cases it's a lot, in other cases it collapses. But that 40,000 number isn't, you know, directly comparable to something like the National Vulnerability Database or something. From there, yeah, you break it down into individual things that can be fixed in different pieces of software. And then we try to fix them. In a lot of cases they can be fixed, in other cases they can't be fixed because it's just inherent issues with the software where to fix it you would have to break or change the functionality of the software in so big of a way that it's probably easier to upgrade it away or just get rid of it from your stack than to try to patch it in place. So we do that. At the same time, we also look around in the history of that software, right? Because when you find something this way, where it's a real running application, that application is using hundreds of different dependencies, probably random versions throughout the history of time. It's impossible to be on the latest version of everything at all times. Even if you try that yourself, you might have an indirect dependency, right? The latest version of a dependency you have might be on a version five years old of something else, right? The whole world is not upgrading everything overnight, every single night. And so you get this report and we go back in time and see how long ago this vulnerability was first introduced and we go forward in time too and see if it's still present at the latest version. A lot of these vulnerabilities are not still present at the latest version. And it's great in some ways that software is getting more secure over time, but it's also this deeper problem with the way all of the vulnerability information in the world works, where if it's not present at the latest version, and there's no vulnerability filed, you can't even really go get one filed today. The vulnerability databases really only deal with things that are vulnerable at the latest version. And most hosts rely on vulnerability signals to determine when to upgrade. And so there's whole separate paths of how Disclosure works for things where you can't go get a CVE file because it would immediately be rejected as unsupported when assigned or something like this. So there's a lot of different flows in this system. Open source projects as well aren't going to take a fix for a three or four year old version. They're just going to say upgrade, right? And some of these were fixed and no one filed a vulnerability, but they were fixed intentionally. Others are just no longer there because the code has been refactored or upgraded in other ways or entire functionality deleted. So there's not even really a direct fix for the project or anything that you can go and copy from. So that fixing process is also complicated. But then after the fix has been published and people have tested it, and it is something that could be upstream because it's still present in the latest version, then we start that process. And that's also not simple, right? When you're looking at the long tail of open source, trying to find the email addresses for people to even contact them and like, you know, you might have something is a massive challenge on its own. [17:30] Conor Bronsdon: [OVERLAP] Yeah, I mean, just thinking about this, it's clear how many challenges immediately present themselves, and I'm sure there are more that we're not even diving into. One of which I know is that, you know, you're using an encrypted portal for each submitter, and then they get to set their own embargo terms for these vulnerabilities. So I'm sure there's a bunch of management that has to happen around, okay, when are we publishing these findings? What's the clock on this? How do we manage all that? [17:56] Dan Lorenc: [OVERLAP] Yeah. [17:56] Conor Bronsdon: [OVERLAP] I'm curious, are you using, you know, security agents to support that? And are they managing this process for you? Or what's the internal AI tooling for the defenders look like? [18:06] Dan Lorenc: [OVERLAP] Yeah, we have agents, a lot of models, a lot of agentic loops all over the place for it. From that first intake to the rest of it, we try to have as few human eyes on it as possible. The more people looking at these things, downloading them to random laptops, you run a lot of other risks just doing things that way. And so the system's designed to be as hands-off as possible. The models have ways to escalate and get help for people. Like when they find one of those cases where, hey, to fix this, we're going to break this major functionality. What do we do? Should we fix it or should we just leave it there and deal with the fallout that way? So there's a lot of these case by case things where you can't go train a model on it because the human doesn't even know what to do until you look at one of these and go chat with enough people. But the whole system is designed to be as few eyes on it as possible throughout the operation. [18:57] Conor Bronsdon: [OVERLAP] Yeah, to your point, we've made discovery really easy. We have to solve the remediation problem [19:01] Dan Lorenc: [OVERLAP] Yeah. [19:01] Conor Bronsdon: [OVERLAP] though, and...

Dan Lorenc 19:03 And this isn't, I think there's a pretty good parallel here. This isn't the first time it's happened. You know, fuzzing is another technique for computers to using random numbers, but a whole different way of doing this, but to find bugs in software. And fuzzing has been around for decades and decades, and fuzzing techniques have gotten better over time. But that approach, you basically take a computer program, write a little harness around it, and send random inputs in and see if any of them cause a crash, and see if those crashes after look like they were due to a security incident. And this software has been around forever and it's still running at scale. Google runs this massive program called OSS Fuzz, where they're just constantly slamming random numbers into programs and looking for crashes. Once you have one of those crashes, you can generally figure out why it caused that behavior. But in the Linux kernel and a bunch of other critical projects, fuzzing has been outpacing remediation for a decade already. It's just one of these cases where the bottleneck is not the people finding bugs, but it's always the bottleneck to fix the bugs. I think AI is the first time we actually have a chance of speeding up the fixing side. You can't just have, I think the alternative in fuzzing would be the whole monkeys randomly pressing keys on a keyboard thing to generate a fix. If you have enough of them pounding the keys long enough, I'm sure a valid patch would come out.

Conor Bronsdon 20:22 Yeah, I sometimes feel like one of those monkeys, I have to admit. I'd love to get your advice on this. So I

Conor Bronsdon 20:31 am not managing any enterprise projects right now, but I manage several smaller open source projects. How should I be thinking about security for my personal tooling and things that I'm building today versus how should, you know, an enterprise CISO or security leader be thinking about security on their own?

Dan Lorenc 20:50 Yeah, it's the same thing just at scale. It's basics. It's nothing we haven't really known about in the past. You have to be able to upgrade everything in any moment in time. Any one of your dependencies have a plan to upgrade it if a patch comes out tomorrow. Just imagine what happened during Log4j and then go make a list of every dependency you have and kind of game plan it out. What would happen if there was a Log4j here tomorrow? Because that's kind of the world we're gonna be in. And if you do that thought exercise, a bunch of secondary things start to pop into your head of how do we automate this? Or how do we trim a lot of these so that we don't even have to worry about updating them? Do we really need 17 different database drivers? Maybe we could all settle on one. So reduce the tag surface, reducing the number of dependencies to the things you really need and centralizing around those. And then automating as much of it as possible. So these things, as these things become routine, you're not calling in the whole team all weekend like we all had to do during Log4j.

Dan Lorenc 21:47 And I guess, yeah, if you're managing an open source project, uh, your role in that is even more central because a lot of these dependencies get picked up from dependencies. And the more trimming you can do with that layer, the more of an outsize impact you can have, because then it helps all of your customers and all of your users.

Conor Bronsdon 22:02 Season 4 of Chain of Thought is delivered by Sphix. We spend a lot of time talking about what agents need in production, and one of the least glamorous answers is events. Your customers want agent workflows that react to things happening inside your system, which means your API needs webhooks that actually work. not just a post request and a prayer, retries, ordering, idempotency, replay protection. Sphix does that as a service, and they wrote standard webhooks, the spec that Anthropic, OpenAI, and Google bailed against. If your API doesn't have reliable webhooks, that's turning into a lost deal. Join Brex, Drada, Daytona, and many others on Sphix. Get started at link.svix.com slash c-o-t or go to the show notes to grab the link. Qualified startups will get $12,000 in credits, $50,000 for YC companies. I can't recommend Sphix enough. I'm a huge fan of their open source project. I've actually contributed a bit myself and they're so easy to integrate with. I think you'll really enjoy it. Check out Styx at link.svix.com slash C O T. Yeah. Crucially, as you mentioned around being able to contact people who are maintaining open source bits of code, we've all seen the XKCD of this massive amount of machinery supported by this one little thing that some dad built at some point. And they may not be on the hook to continue to maintain that over the years. They may have moved on other things. So, but we have to assume that there's someone on the other end to accept the patch, whether or not that's true. How can we handle it when there are open source projects that don't have a reachable maintainer? How do you approach fixes in those situations?

Dan Lorenc 23:49 Yeah, and so open source is great for a lot of reasons here too. But I have to go back to the Log4J example again. Log4J was an open source success story, right, if you look at it. It was maintained by a small number of people, widely used everywhere. Not to joke, but it was the first interplanetary software vulnerability. A couple of months before it happened, NASA had tweeted that Log4J was running on a Mars rover with a picture when the Mars rover landed or something like that, right? That's how ubiquitous that software was. It was just maintained by a few people. When it got reported, they got fixes out in days. It's that downstream side of taking those updates, knowing where it's running, and knowing where to do updates that took months to years. And there's still organizations out there running vulnerable versions of Log4j in the end.

Dan Lorenc 24:35 But you have to step back, too, and kind of remember what open source is, because that governs a lot of the paths here in these cases. But for a maintainer, there's no guarantee, no promise, you're not under any burden to ever deal with a security loaner really. You're putting software on the internet, that is now open source. You might put a new version out tomorrow, you might not. You have no guarantee, you have no burden to do this on behalf of anyone. But to me, in terms of projects, what I like to tell people to do is just stay true to your intentions. Do you plan on dealing with vulnerabilities? If you do, tell people how. Otherwise, they're going to be all over your GitHub, whether you want them there or not, or all over Hacker News or Twitter. But if you don't plan on it, then you have to be ready for people to fork your project and start to merge their own patches. So that is one of the freedoms of open source. Open source is a licensed category that guarantees a few things. But one of them is the freedom to fork the code. And if you put it out there and somebody grabbed a copy of it, found something, and you said, no, I don't want to fix this vulnerability, then they're probably going to take that escape hatch. And that's fine. That's how open source always works. But I think we're going to start seeing a lot more of that because it's going to happen. It's going to start happening quicker and quicker. And also on the flip side, the economics of maintaining a fork have dramatically changed. Just because you can fork doesn't mean it always made sense. Most projects that are doing well today have avoided forks because they're moving so fast. The cost of maintaining a fork for anybody that wanted to over a long period of time just kind of grows forever to the point that it makes more sense to send contributions back into the open source projects. You're not maintaining that fork over time. But the cost of doing that has gone down dramatically. Agents can do a lot of that work. It's relasing, it's merge conflicts, you know, the things that developers absolutely hate doing. But that work, agents are pretty good at. And so I think, you know, state your intentions. Say if you want to deal with vulnerabilities, what your plan is for them. For the most part, researchers will try to work with you on it. But if you don't want to do that, that's fine too. No one is going to make you. But be ready for folks of your project to start to appear.

Conor Bronsdon 26:43 I deeply resonate with your point about merge conflicts, [26:47] Dan Lorenc: [OVERLAP] That [26:47] Conor Bronsdon: [OVERLAP] something [26:47] Dan Lorenc: [OVERLAP] sucker. [26:47] Conor Bronsdon: [OVERLAP] that I have 100% offloaded to my coding agents and just do not think about anymore. I'm just like, yeah, solve us, please. I don't want to, please. Not my problem anymore. And to your point, we have the capability now of just having an agent watching a repo at all times. And it can just solve these minor pieces. It can ensure that your fork is successful, that it's healthy. And I think this is going to become very normalized, if not already is being normalized. I wonder, though, what other changes you expect to see in how open source projects are maintaining themselves and thinking about security As we go into this era where, at least I expect, we are going to have coding agents available widely, cheaply, and all over the place.

Dan Lorenc 27:40 Yeah. I think, you know, coming back to that opening, one is people have to pay off a lot of tech debt. And two, we have to figure out how to get off of this treadmill. And I think, again, it's a new world. And, you know, I think a lot of the ways we thought about software in the past around things like backwards compatibility and API compatibility and stuff like that, that for good reason, slowed down a lot of change in software. I think we should start to rethink some of those assumptions because these same tools that find vulnerabilities are also really good at upgrading software and they're really good at changing syntax and changing API patterns across code bases. And when we dig into some of these vulnerabilities, like these classes I was telling you about before, you know, one really common pattern that we've seen, especially in Java, is after Log4j, a lot of Java projects just went and deleted those dangerous portions of the code base that were exploited in Log4j. You know, they didn't know there was a vulnerability there, but they were like, oh yeah, maybe we shouldn't have had all of that crazy remote JDNI stuff in here. This seems like a taking timeout. Let's just go delete that. I think we're going to start to see more and more cleanups like this of projects just removing legacy behavior, removing things that might be dangerous in the future, removing entire classes of things like that, just to start to change the slope of the vulnerability discovery curve. And these are things you might have been scared to do in the past because it's going to break a bunch of stuff. Everyone's going to complain when they have to do these upgrades. But it's either going to be complaining about having to do upgrades and complaining about the syntax changing or sitting on this treadmill that's just speeding up forever and ever.

Conor Bronsdon 29:16 Yeah, the treadmill is a really interesting point because I, I kind of struggle to see how we get off it in the short term.

Dan Lorenc 29:25 Yeah.

Conor Bronsdon 29:27 Is there a redesign that we need to do to just simply change entirely how we approach security? Like what's the way off this treadmill?

Dan Lorenc 29:35 Yeah, I don't know if there's ever a way off, but I think we can slow it down. And it's how securities works forever. I don't have a magic answer right now, but the security industry has always done this. If you go back 10 years or so, the vulnerability class that was everywhere is cross-site scripting, right? Remember like the early days of social media, you had to be careful not to click the wrong link. Otherwise, it would automatically DM everyone on Facebook. Um, that kind of thing. Um, and it was just almost impossible to write JavaScript code that wasn't susceptible to it because it was just insecure by design. Um, and nobody knew this was possible. Somebody figured it out and it was like, Oh wow. Every website anywhere, if you click the wrong link, like you were completely hacked. And then eventually figure out ways to make frameworks and libraries that are just basically immune to cross-site scripting and some changes in the browser and that kind of thing. And you rarely ever hear about that anymore. But we went from like, you know, vulnerabilities being found every day for bug bounties to once or twice a year now. And it's that, you know, it's looking for classes of vulnerabilities, figuring out ways to work together, work across projects, get new technologies built to make those entire classes impossible. And then rewriting software to use that and upgrading software to use that quickly. But that's how we slow the treadmill down.

Conor Bronsdon 30:49 As we think about this era of agents everywhere, code is cheap, how do we need to think about designing our agent infrastructure with security in mind? So great, we're solving the treadmill, but what about from the very start as rebellious agents? [31:08] Dan Lorenc: [OVERLAP] Yeah, it is super timely right now with all of the reports of the agents going rogue and hacking systems over the weekend and planting malware on PyPI to break into other systems and stuff. But yeah, agents, they're powerful, right? And the power comes in from the tool calling layer, right? They have as much power as you give them in a tool. An agent by itself, it's just random numbers outputting text. Text can't really do much. But when you start to bolt on tools, they give them, hey, access to the internet or access to run commands on a laptop. You have to expect that these systems are going to use those in unpredictable ways, because, again, they're random number generators at the start. You can never fully control what one of these agents is going to do. And so it's nothing magical again though, but it's defense in depth, it's constraining what the agents have access to, not just running them as root on your laptop. I know I think the Gemini CLI called it the YOLO mode flag, that kind of thing. It's one thing on a laptop. It's a completely different thing when you put this into a production system and you're running too many of them to watch as a person. And so putting that in at the start, the bare minimum permissions they need to do their job, sandboxing them as heavily as you can, and then sandbox the scarier ones even more because they might find a breakout of that sandbox in order to get their job done. But we see it a lot. Agents are very persistent, which is a pro and a con. In this case, with Athena and Apache, I was telling you about. If you just tell the agent, patch this and don't introduce any backwards incompatibilities, it'll run for 24 hours if you let it trying to do that. Even if that's impossible, these things will keep trying, they'll keep trying. And so when you start telling them to do these dangerous kind of cyber security-ish tasks of like find a vulnerability, and after three or four hours it can't find one, and it finds a different vulnerability in the sandboxing layer you built, to go do something that it's been trying to do that whole time, you have to expect that that stuff is going to happen. And defense in depth is really the only way to notice it when it does. [33:13] Conor Bronsdon: [OVERLAP] you [34:04] Conor Bronsdon: [OVERLAP] I want to ask several more things about [34:06] Dan Lorenc: [OVERLAP] Sure. [34:07] Conor Bronsdon: [OVERLAP] your viewpoint on agent infrastructure and how you're coding today, particularly because I saw that on your LinkedIn, [34:13] Dan Lorenc: [OVERLAP] I'll call you [34:13] Conor Bronsdon: [OVERLAP] you [34:14] Dan Lorenc: [OVERLAP] right [34:14] Conor Bronsdon: [OVERLAP] list [34:14] Dan Lorenc: [OVERLAP] now. [34:14] Conor Bronsdon: [OVERLAP] yourself [34:14] Dan Lorenc: [OVERLAP] Can you [34:14] Conor Bronsdon: [OVERLAP] as [34:14] Dan Lorenc: [OVERLAP] tell? [34:14] Conor Bronsdon: [OVERLAP] having. Yeah, I see that you listed yourself as having been an intern in Gastown, so [34:20] Dan Lorenc: [OVERLAP] Yeah.

Conor Bronsdon 34:21 I very much want to ask your thoughts on that. But before I do, I think I would be remiss to not ask about the other side of this equation around security, which is the regulatory side. There have [34:30] Dan Lorenc: [OVERLAP] Yeah. [34:30] Conor Bronsdon: [OVERLAP] been major discussions there. What's your take on the right approach?

Dan Lorenc 34:35 Yeah, and there's a bunch of different regulatory kind of conversations going on here across the board. In this era, you know, there's open source itself, which is constantly the topic of regulatory pressure. A few years ago, Europe, with the Cyber Resiliency Act, tried to place a burden on open source maintainers, and it kind of failed. But they were trying to use liability as a legal framework. Say, hey, if you put an open source project up on the internet, say you're just a random developer here in Arkansas or Nebraska or wherever that XKCD is, and some bank in Europe starts using that software, it gets breached as a result. They were going to place liability back on you, even though, you know, the very first clause in every open source license says this comes with no warranties. Use it at your own will. These conversations constantly happen. But the right place for that liability and the right place for that responsibility is the person that pulls it in and uses it, not the person that wrote it and put it out there. Um, then there's AI, uh, you know, there's a regulation around the frontier labs. You know, if they are actually building things that are, you know, close to the super weapons that vulnerability research or these other things, um, lots of conversation around what security they should operate under themselves. Um, you know, if you're, uh, building chemical weapons in your backyard, uh, that's probably not good for the neighbors around you. Right. And so we have processes and regulatory frameworks for how you do that kind of dangerous weapons level research. And yeah, I think that's a particularly tricky one because the model providers have all been saying this for years, but no one really believes them. You know, you're like, it's a chatbot. You know, how could it hack me? That kind of thing. Until all of a sudden it's too late to really figure out what to do. Um, then there's open models and, uh, geopolitical economic, uh, incentives around all of this. Um, distillation attacks and that kind of thing. Um, it's going to be a very busy year for DC. I would say none of these are clear cuts. Uh, here's what we should do. And I don't fix everything cases.

Conor Bronsdon 36:40 Yeah, it's especially interesting given, I think, the coalition we've seen form around open models and the concerns for many companies, frankly, that they're going to be cut out if they're not careful and if open source isn't supported. And yet there's this whole geopolitical back and forth, as you point out, particularly between China and the U.S. around security capabilities and more.

Dan Lorenc 37:01 You know, I think open models are great. Being able to download these things, run them on your own hardware is great. I think what we've learned though is the economic incentives around it just don't make sense for any standalone company to build these open models anymore. They cost so much to train and release that you can't really go recoup that cost unless you have some other motive around why you're doing that to build and release that open model. You know, at a couple million a training run, sure. At, you know, a couple hundred million a training run, the number of companies that can afford to write that off starts to drop, right? At billions and billions a training run, you're down to a couple of companies and governments. And it starts to really shift the equation here. [37:43] Conor Bronsdon: [OVERLAP] Yeah, I'm not certain that that is going to stay the way it currently is, though. I do think we are underestimating how much the economic model around both training and serving is going to rapidly change in the next couple of years. I mean, we've seen our assumptions already be off base in multiple areas, like A couple of years ago, people were saying, oh, you need the next generation of hardware. Old [38:07] Dan Lorenc: [OVERLAP] there [38:07] Conor Bronsdon: [OVERLAP] generations, they're going to go out of fashion so quickly. And now we're realizing we need all this compute. And actually, this hardware is very valuable because we can serve [38:16] Dan Lorenc: [OVERLAP] are

Conor Bronsdon 38:16 in different ways and we can actually spread models out. And we've made a lot of innovations on how we serve and how we train and how we fine tune. And I wonder if we're going to see similar research breakthroughs that flip some of the economic models and make larger open source models more viable without distillation.

Dan Lorenc 38:36 Yeah. Um, and then there's just the question of, is the front gear pace going to be moving so fast that putting $10 billion in is always going to get you a better result than someone that didn't put $10 in, even with all of those techniques. We don't know. Um, yeah. And then maybe we'll just all have nuclear fusion reactors under our houses, running our own, uh, training and inference labs and, uh, it'll be great.

Conor Bronsdon 38:58 I mean, I would love to have them, but I probably don't have the infra to support them. My wife might have a few things to say about what's in the garage. So we'll see where we get to. It's going to be a very exciting time. I think despite the concerns we've brought up, this is one of the most exciting times, maybe the most exciting time ever in computing, because so much is changing and we are just seeing this incredible paradigm shift. And so it's so fun to lean in in this moment, I feel like.

Dan Lorenc 39:26 It's the most fun time to be building software. I think, um, I don't know how much longer we're going to have before the robots are just doing it all for us. Um, but yeah, I, they've just, the models have gotten so good to the point where they're fast and the quality of the code is better than the quality of what a person would write in almost all cases. And so you're not limited by typing. You're not limited by compilers. You're just limited by, you know, the ideas and the designs and the things you can think of. Uh, and that's, that's exhilarating, honestly.

Conor Bronsdon 39:55 I would love to learn a bit about your personal approach to this. I know, you know, I mentioned back in January, you had already put yourself jokingly, sort of, as the assistant to the mayor of Gastown [40:05] Dan Lorenc: [OVERLAP] Yeah. [40:05] Conor Bronsdon: [OVERLAP] and the team of agents at Polkatz. What's your approach today? How have you kind of evolved on that?

Dan Lorenc 40:09 I've simplified a lot. Yeah. I got Gastown going, uh, you know, there was that week when the blog came out and, you know, I've known Steve for years. Uh, I love him. Uh, but the blog came out and it was just like, what am I reading? Is this fiction? Is this reality? And there was a GitHub repo. Um, and they're all means everywhere. Everyone was talking about Gastown and I was like, all right, it's Saturday afternoon. Let me actually try to get this thing going. And I'd been a super active cloud code user for a year, but I hadn't really changed much in the way I was using it. And I got it up and it was, it was crazy. Like, yeah, you were talking to the mayor. It was this crazy tmux user interface. I'd remember all the tmux shortcuts and there were pull cats and there was a refinery and it was all just kind of working. And it was the first time since like, you know, the first release of cloud where I was like, oh, wow, this is actually way, way faster. I tried to describe it as like, why tell Claude what to do when you can tell Claude to tell Claude what to do, right? Um, like, uh, we do many things at once, all of this fun stuff. Uh, and then it completely self-disrupted. Um, it was this, uh, the design of it was this like almost, uh, crazy mess of self-mutating prompts where all of the components like eventually found where all of their prompts were living on disk and some little blip happened and they all tried to repair it and they just completely destroyed GASTOWN. Um, but it was like an amazing four hour, uh, run. Um, and then I made my own slimmed down version of Gastown and I called multi-cloud that I was just using every day for, you know, the next six months instead of regular cloud, I was just in there on multi-cloud. Um, where I kept like a couple of parts of Gastown, the, the TMUX interface and the message passing, you know, kind of the, the wake loops around agents. And then cloud just kind of rolled enough of those features into cloud itself. The multi-agent loops got good and it's got a wake and notification system and stuff like that. But now I'm just back, I'm just the normie again. I'm just cranking out, I can keep a couple terminals open at once and just check on them every once in a while. [42:06] Conor Bronsdon: [OVERLAP] Yeah, I have had a bit of a similar arc where I feel like I was trying to engineer the interactions a lot more. And now with the newer class of models, they're so good as team leads, they don't really need me to manage the agent orchestration, they just need me to suggest what to do. And I don't [42:23] Dan Lorenc: [OVERLAP] Yeah. [42:23] Conor Bronsdon: [OVERLAP] know how long that's going [42:24] Dan Lorenc: [OVERLAP] Yeah. [42:24] Conor Bronsdon: [OVERLAP] to [42:24] Dan Lorenc: [OVERLAP] They divide the work [42:25] Conor Bronsdon: [OVERLAP] continue. [42:25] Dan Lorenc: [OVERLAP] up when they should. They go to agents to minimize context when they should. Um, and yeah, the, the prompting, especially in the generation, uh, five class, Sonic five, Focus five, Fable five, that kind of thing. The simpler, the prompting, the better again. Um, you know, you don't have to be crazy descriptive. Uh, you just tell them what to do and they figure it out.

Conor Bronsdon 42:47 Yeah, one of the things I've been thinking about, I saw Boris Tierney from Anthropic mention this idea is paring back some of my established infrastructure harness. Like I have a several month old repo that is full of just session logs and memories. And I'm wondering if I've almost gummed up the works too much sometimes.

Dan Lorenc 43:09 Yeah, I hate the memory system. Yeah, just because of that, like the models add a lot of notes, but I change stuff constantly. I've got multiple branches. I find that always adds more confusion to future runs than anything, the memory part. [43:24] Conor Bronsdon: [OVERLAP] Interesting. So yeah, how how are you managing memory at this point? Are you just letting the models handle it themselves? Do you have an established system you're approaching it with? [43:31] Dan Lorenc: [OVERLAP] I feel like it's been getting worse. Like, you know, five times a day now I see it writing memories of things that I don't want it to remember. We were just trying out something crazy. I'm sure there's a setting to turn it off, but normally I just go through, like, the directory by myself. [43:45] Conor Bronsdon: [OVERLAP] Interesting. Yeah. I, I took the opposite approach. I built my own memory repo that I force it to use. And then I have it check with me at the end of every session. I'm like, okay, great. I have an established regimen for, or discipline for how we call that a session. And it's can propose memories that I can then get to approve or say, Oh, adjust. But it does create overhead for me where I'm like, I know this is slowing me down a bit. So I'm curious to see how it goes long term. [44:10] Dan Lorenc: [OVERLAP] Yeah, whenever I'm in the web for writing and researching and stuff too, I have the memory feature turned off across all of these. It drove me crazy one day, I turned it off and I left it there. It was like writing a blog, I got a draft, I didn't like it, I opened up a new tab and started writing a blog and it was like, oh, you already wrote this and it just kept pulling from the one I didn't like that I just opened another tab.

Conor Bronsdon 44:32 Yeah, I think I need to refactor and maybe just throw some stuff out to see what happens, if nothing else. [44:40] Conor Bronsdon: [OVERLAP] Because I've built up such a corpus now where it's like, okay, I just need to try it with less and kind of see how that approaches things. [44:47] Dan Lorenc: [OVERLAP] Yeah. Might be the, like a year ago when prompt engineering was a thing and there were all those Twitter and LinkedIn, uh, click clickbait posts of like, Oh, here's the workflow. You start with this prompt, which is like six pages long and. [45:01] Conor Bronsdon: [OVERLAP] I've always thought prompt engineering was overrated. Yeah, [45:04] Dan Lorenc: [OVERLAP] Copy that deep research. We'll copy that over here. And like the results are so much better. Everyone. Um, yeah, you don't need any of that anymore. [45:12] Conor Bronsdon: [OVERLAP] I are you still using typing typically for prompts? Are you transitioning to more voice? What's what are you doing today? [45:19] Dan Lorenc: [OVERLAP] I type. Yeah, I've never been a big fan of voice and any computer interactions, but I know a lot of people like it.

Conor Bronsdon 45:26 Now I'm with you on the typing. Honestly, it's just largely because I like listening to music while I code, so I don't really want to have to switch that off and turn voice on. I'll use voice occasionally if I really like. I'll use it on the go. Actually, I find more for mobile if I'm like, oh, I'm having a thought. Let me just throw this into an instance so we can get something started and I can pick it up later. That's where I find these voice more interesting.

Dan Lorenc 45:46 This was fun. I'm glad we ended on a more exciting note than the [45:51] Conor Bronsdon: [OVERLAP] Yes. [45:51] Dan Lorenc: [OVERLAP] terrifying start you gave us.

Conor Bronsdon 45:53 Dan, do you have any closing thoughts for the audience or anything you didn't get to say you want to bring out?

Dan Lorenc 45:57 Um, no, this is, uh, even when it's terrifying, it's still an exciting time to be in software and in security and, uh, just have fun riding the wave.

Conor Bronsdon 46:07 Sometimes you might feel like you're drowning, but just get up, get back on the surfboard, and you're going to catch some cool waves for sure. Dan, it's been so fun talking to you. I really appreciate the wide ranging discussion. I appreciate the pros and cons. And thanks so much for coming on Chain of Thought. I think our listeners are really going to enjoy this episode. Where can folks go who are listening and want to check out Athena or check out Chain Guard or just follow you?

Dan Lorenc 46:29 Yeah, uh, chainguard.dev is our site. That's where all this stuff is. I've got a good blog. I do a lot of writing there, um, or LinkedIn. Um, yeah, that's where I do most of my posting. Um, Dan L O R E N C. You should be able to find me.

Conor Bronsdon 46:42 amazing. Dan, thanks again for coming on the show. And for everyone who has made it through the entire episode, if you enjoyed Dan's conversation with us, make sure you are subscribed to our newsletter at chainofthought.show or newsletter.chainofthought.show for more episode extras. Dan, thanks again for joining us.

Dan Lorenc 46:59 Yeah. Thank you.