How Cisco Secures Personal AI Assistants for 90,000 Employees | DJ Sampath

Conor Bronsdon 1:12 Is actually our pleasure, I'll say, because I think Cisco's rollout of personal agents is so fascinating. It's one of the Most scale rollouts in the entire world. And I want to understand. What has to be in place for a company like Cisco to be comfortable with this kind of rollout given your security background?

DJ Sampath 1:31 Listen, first of all, I think you need a a a fundamental you know, mindset shift in terms of saying You can you know, leverage AI to be able to do valuable work. You know, from a mission statement perspective, what we're trying to articulate here is to say, hey, every single thing that we do. is to be able to drive towards that mission of Hey, can we leverage AI to be able to do valuable work. And that, you know, we have a phenomenal IT team that said, listen, we're up to that challenge. And let's go out and let's try this out. Historically, you'd you've see IT teams kind of struggle with, like, oh. This feels hard, or this feels like we're not ready. We should wait and watch. But but at Cisco we have uh an unbelievably forward thinking IT team that said You know what, game on, let's make this happen. Um but they did need

Conor Bronsdon 2:18 did meet

DJ Sampath 2:19 a a certain sort of requirements. They they wanted to make sure that hey um We need to secure every single prompt that's coming in and out. We need to make sure that there's the appropriate sandbox. that we provide to these agents so that these agents are not you know, going rogue. And um and we needed to make sure that Hey, um Your agent is separate from my agent? You know, it's funny, we call it MyAgent. It's got Cisco MyAgent.

Conor Bronsdon 2:43 Ah, okay.

DJ Sampath 2:44 But um but we want to make sure that each one of our agents are separate and they're multi-tenancy from the get-go. So things are not getting unnecessarily you know polluted by the context and memory that you are you know uh having uh that that that you may not necessarily be sharing. We also had to reimagine, like, how do we start to think about agents collaborating with each other? So so so a number of different steps we had to get through, but I think the most important one was to make sure that Each individual brings their skills in. If they are looking at, say, a GitHub repo and they're grabbing a skill and they're throwing it in. You gotta make sure that those skills are appropriate. The skills are not malicious, they're not vulnerable.

Conor Bronsdon 3:23 audible.

DJ Sampath 3:23 So you needed a skill scanner, right? You also needed to make sure that you're scanning the tools that you're using, because every single one of these agents are calling tools to be able to. Access an API, get credentials approved. And identity is sort of a very, very important thing when you sort of think about these agents are going, how they're starting to operate. So the important thing that the team did was to be able to lean on a bunch of different frameworks that we've already put in place. So we sort of saw this coming in some ways. Like in 2024, we announced this product called AI Defense. So Cisco AI Defense is sort of a product that looks at the entirety of your supply chain that looks at models, understands you know Um if models are vulnerable because models could potentially have been trained on something that's malicious. Um so you know, you want to algorithmically red team these models because guess what? There are no C V Es for models, right? I mean, you can look at a a vulnerable package. By the way, yesterday was a Oh my God, what a day. You know, so CVEs are going to start showing up every single way you'd look at it. Um but with models it's it's so hard because there are you know billions, if not trillion, parameter models. And so the only way that you figure out what's really going on is to algorithmically red team these models and figure out what those gaps are. But beyond that, as part of AI Defense, we said, hey, we're going to have to build God rats. At runtime, you wanna make sure that, These don't get accidentally prompt injected. You know, one of the things that happens when you think about a personal agent. Um or even if it's an agent that you're bringing from the outside, the the very first integration that you end up doing is to connecting it to your emails. Right. Uh do you use Muse or Instinct?

Conor Bronsdon 4:58 I have tried Muse, I've tried Instinct, uh I've tried Grokbot, um I've tried a little bit of Dots, Right. Nothing's stuck with me yet except for Grokbot has, I have to Ooh, admit. So okay. yeah.

DJ Sampath 5:08 All right, I'm a Muse fan.

Conor Bronsdon 5:09 Okay, interesting. Yeah, And yeah.

DJ Sampath 5:11 more recently, ever since Dots got announced, because of the fact that it seamlessly connects to my ecosystem inside stuff. of Codex, it's starting to really stick. But I think one of the interesting things that you probably noticed is as soon as you got your Grokbot, you probably connected it to your emails and your Gallup. calendars. Yeah. Right. And so think about it. I come from an email security company. I sold an email security company to Cisco. I was a founder C of Armor Blocks. And I think about emails a lot, a lot more than a normal person would. And so inside of an email, if I were to send you an email right now. that had a prompt injection. It just simply said Hey Conor, great to meet you. And in complete like white colored text at the bottom of my signature. I put in a prompt injection attack.

Conor Bronsdon 5:54 Yeah.

DJ Sampath 5:54 Your agent's going to read that.

Conor Bronsdon 5:56 It's true. Okay.

DJ Sampath 5:57 So You know, it becomes incredibly when you start thinking about You know, these agents, what kind of guardrails do you have in place to be able to secure them? so that was a big part of the conversation with our IT team as well, saying, hey, we built this thing at Cisco AI Defense. We've deployed this inside of Fortune 100 companies. Um they're giving us a ton of feedback about the kind of guardrails that you need, whether it's PII, PCI data, whether you're operating with like, you know, specific types of tool poisoning or somebody's trying to like, you know, hijack your context window. They're trying to do some memory attacks. By the way, we've seen crazy amounts of attacks

Conor Bronsdon 6:31 AMIL

DJ Sampath 6:31 in the wild that are Targeted at these AI systems and then compromising and getting these AI systems to respond back in ways that. Um Yeah. world's not fully understood, you know, the extent to which these systems can be made to do things. So

Conor Bronsdon 6:46 we

DJ Sampath 6:47 put we together our an AI research team. To be able to understand all of these things. Now, the very first thing we had to solve for before we even got here was to be able to come up with the right kind of taxonomy to be able to describe these books. issues. Hey, you know, what is tool poisoning called as? What is, you know, we've always had this with regular security stuff. You know, if you remember the OWASP top 10. These were like nice bullets of like, hey, here are the things that you care about, cross-site scripting, you cared about SQL injection attacks, and all of those things. We needed a taxonomy for the industry to be able to start talking about this effectively together. So we put out a website. If you go search for Cisco AI safety taxonomy, you'll see that it's it's public, it's open. And we've collaborated with NIST, MITRE, OWASP. That really gave confidence to our IT team to say, hey, wait a minute, A, you have the right taxonomy, B, you've built, you know. Um things that help us understand the supply chain. See you've got the guardrails in place. Let's take all of these pieces, and then on top of it, from an agentic perspective, we open source a project called Defense Claw. Yeah. Um if you haven't tried it, you should try it.

Conor Bronsdon 7:47 It I actually have. So Oh you yeah, have. yeah.

DJ Sampath 7:49 Oh, that's amazing.

Conor Bronsdon 7:49 Yeah.

DJ Sampath 7:50 Um

Conor Bronsdon 7:50 I I'm a big fan of a lot of Cisco's open source work. I have contributed to the AGNTCY project uh for multi-agent uh orchestration. So uh really have been impressed with some of the open source work you all have been doing. And I hear you have some new news on that too to maybe share with Ooh us.

DJ Sampath 8:04 yes, there's something really, really cool that

Conor Bronsdon 8:06 cool

DJ Sampath 8:06 we're working on. And it's sort of like ties all of these pieces together, right? I think we've been so Deep in terms of understanding, like, you know, what are the things that we need to do to make sure that these agents are behaving the way that they should, you know, making sure that you can delegate actions with to these agents in a

Conor Bronsdon 8:23 buster

DJ Sampath 8:23 trusted fashion. You know, we've been very thoughtful about some of that work. as you probably know already, you know, we made a number of acquisitions in the space. We acquired this company called Galileo.

Conor Bronsdon 8:33 I was at Galileo actually. So

DJ Sampath 8:35 No I I way.

Conor Bronsdon 8:35 left shortly before Cisco acquired Galileo.

DJ Sampath 8:38 Hang So. on a second, time out.

Conor Bronsdon 8:53 And I will say, by the way, I've I've been really impressed by the thought process that. Cisco has been taken in general. And like I said, I've enjoyed the open source work, I loved the metaphor that. Um due to Patel used on this show when he came on the president of CPO of Cisco. If you haven't watched that episode, check it out. Or he calls agents unruly teenagers that need guardrails and rules. And I think I hear that from what you're saying, not

DJ Sampath 9:17 Totally.

Conor Bronsdon 9:17 only the agents will go out and misbehave, but also because people will attack them and they may not have the knowledge of the world that they need to adjust to that.

DJ Sampath 9:25 That's exactly right. They don't understand consequences, right?

Conor Bronsdon 9:27 I so

DJ Sampath 9:27 mean, as humans, a lot of times there's a anthropomorphization of agents that says it's exactly like humans. And

Conor Bronsdon 9:33 lab

DJ Sampath 9:34 the reality is it's more like teenagers, you know, that you give a credit card to and say, you know. Don't spend it on crazy things. But what is the

Conor Bronsdon 9:43 is definition definitely

DJ Sampath 9:43 of crazy? It's very hard to define it with a teenager. And so it's a lot like that but with these agents. they've got very little amount of understanding of the consequences. Like you mentioned, from a context perspective.

Conor Bronsdon 9:54 So it's really

DJ Sampath 9:54 really important to Understand Are they doing the things that they said that they were going to do? And so evals becomes a really core important

Conor Bronsdon 10:02 for

DJ Sampath 10:02 part of this. And as we looked at the market and we said, hey, are the best people to be able to go out and solve for these things? Vikram, Yesh, and Athen were like phenomenal people that are now driving that inside of Cisco. And so when you look at that whole spectrum of hey, how do we secure AI? How do we start to think about the models, the red teaming part of it? We acquired a company called Robust Intelligence that brought in some amazing talent over there. And then we said, hey, how do we think about evals from as we evolve the guardrails? You know, and and that's how we acquired Galileo. But we also recognize that identity is a huge part of

Conor Bronsdon 10:36 a

DJ Sampath 10:37 this story, right? How do we make sure that these agents, when they're acting on behalf of you, they're doing the right types of things? When these agents are acting autonomously by themselves, are they using system accounts the right way? And so we acquired this company called Astrix,

Conor Bronsdon 10:52 uh you know

DJ Sampath 10:53 which is a phenomenal company, Astrix Security. You know, they've thinking about non-human identities for a very long time With agents, that becomes really front and center. And we've already had a good book of business from Duo and Identity Services Engine. So it made sense for us to be able to say, listen, we understand this really well. But when you start putting all of these pieces together, we have arguably the most comprehensive way of being able to understand an agentic view, like agent behaviors. And that's really what unlocked the kind of success that we're seeing with the rollout of 92,000, 90 plus thousand agents to our employees. Um but one one of the things that we've noticed is that the open source contributions that we've been doing They've been extremely well received. Like, we have over 5,000 plus GitHub stars on the AI Defense projects, you know, collectively. And so. We recognize that there's now a moment in time where we have to step up and say, hey, How do we start to describe the behaviors that you want an agent to take. In a very prescriptive way, that the agents themselves understand that that is sort of their scope of what they need to perform. So we basically said there needs to be a domain-specific language. that describes these Yeah, agentic behavior. And that's sort of what we're launching as an open source initiative. And that's the news that you were referring

Conor Bronsdon 12:13 to.

DJ Sampath 12:14 it allows us to be able to um do really

Conor Bronsdon 12:18 me

DJ Sampath 12:19 interesting things because when you describe that hey here's what you all love to do It suddenly changes the agent's behavior when it has a certain amount of token budget. and it says, okay, you know, I say for instance that the agent wants to self-sacrifice itself for the larger good.

Conor Bronsdon 12:34 Ah, like kind of Yeah. like that meteor investigation of OpenAI's agents into the Hugging Face

DJ Sampath 12:38 hack. That's right.

Conor Bronsdon 12:39 Yeah.

DJ Sampath 12:39 You can now basically define a policy in a DSL that says, hey, that is not an acceptable thing. And it's going to define how the agent looks at that and goes, wait a minute, these are the set of instructions that I have, in addition to the goal that I have being given right now to be able to go out and execute. my policy prevents me from doing this. So um we expect that you know the DSL is a way that you could describe these agent behaviors. We we're hoping that this basically is something that we can share with Other industry partners to be able to help define these things, researchers, security researchers, folks that are building operators. Get them all together to be able to say, hey. Provide that specification. and help everybody start to understand that your agents can be governed a lot better. uh when you start using this um domain specific language.

Conor Bronsdon 14:06 I'm really excited to check this out because You know, something I've talked about on my newsletter, on the show, is the preponderance of various coding agents I'm using across multiple model families. And governance and orchestration are huge challenges. It is really cool to see the advancements happening in the open here. So very excited to try this out. And I will definitely link this in the show notes so people who are listening and want to give it a whirl or just read a bit about what Cisco's up to here can consume it. But I'd love to frame this a bit more. So we're talking a lot about agents and how to govern them and why that matters. And obviously you're learning a ton of lessons, it sounds like, from this rollout of personal agents across Cisco. But What are these personal agents doing at Cisco? What's the actual work that they're achieving? You talked about being mission-driven and value-aligned. How are personal agents being used at Cisco today?

DJ Sampath 14:54 Yeah, one of the big things we discovered, you know, the the primary goal was just to be able to allow people to first retrieve information. Our initial hypotheses was that, people are going to want to retrieve a presentation or a document or a PRD, and so make it easy for them to be able to ingest this inside of an Agentic framework, ask questions, that they don't have to keep copy pasting these files into different harnesses.

Conor Bronsdon 15:18 Oh.

DJ Sampath 15:19 However, what you we learned very quickly where people started really using this. Was the minute I think the teams activated a cron job, the ability to actually execute a scheduled task. Um I know that sounds very obvious when you think about it, but it's sort of one of the

Conor Bronsdon 15:34 them

DJ Sampath 15:34 basic functions that we start using it for like daily briefs and like summaries at the end of the day, for meeting notes. I think one of the biggest unlock that happened was, A lot of times at work, we're in meetings, right? And in the case Cisco, we're doing a lot of Webex meetings, right? And so. Every single one of these Webex meetings now has the ability to get transcribed. And at the end of the day, when you have an agent that can hook into that over APIs. collect all of the notes, take all of the actions, summarize it for you, and keep track of those actions so that you know which ones you've completed, which ones you've not, reminds you as part of your daily brief. Man, that just changed the way we thought about productivity inside.

Conor Bronsdon 16:14 And DJ, I do have to admit here, as much as I am a fan of the security work being done at Cisco and the open source work with agent orchestration. I'm not a big Webex guy, I'm more of a Google Meet guy, hooks into Yeah, my yeah. Gmail, but yeah,

DJ Sampath 16:27 you'd be shocked how good what Webex is.

Conor Bronsdon 16:29 Okay, okay. I It's truly been a believe. couple of years, I haven't used See, it in a while, so yeah.

DJ Sampath 16:32 I think you should try it out. We

Conor Bronsdon 16:33 I have arguably argue

DJ Sampath 16:34 some of the best devices. Now I'm just gonna go ahead and pitch. Anytime somebody tells me this, I gotta.

Conor Bronsdon 16:38 All right, next question.

DJ Sampath 16:41 We have arguably some of the best devices. Like the quality of the devices, the microphones, you know, we use AI to be able to do, like, you know. Automatic framing of people depending on where you are into the conference rooms. Noise cancellation is. Off the charts. It's Okay. one of the best on the planet. And it's really simple to use. And by the way, our devices support. Teams and Slack and like, you know, Zoom. Yeah, no other device out there does that, so we're pretty cool, man.

Conor Bronsdon 17:07 Okay, and so you're connecting these devices and all the inputs and outputs from them to these personal agents so people can ingest meeting notes and have actions tracked. What are other use cases you're seeing from personal agents throughout the organization?

DJ Sampath 17:20 Well the the one of the biggest things that we're um we're seeing from the engineering side of the house is they're they're using this to be able to track their you know um uh the work output. Like, hey,

Conor Bronsdon 17:28 geotickets

DJ Sampath 17:29 Which Jira tickets are getting solved. They're hooking up these agents to their codex agents. And so as they're looking at like, hey, there's a pull request that's coming in. Hey, can you tell me like, you know, what are the biggest sort of changes? And so a lot of people initially thought that Codex was sufficient, right? I could just use Codex. But then came the issue of like, hey, how do I do this remote management of Codex? And as much as the remote options that Codex natively provides is good. we had concerns around like how do we make machines accessible over a VPN. MyAgent

Conor Bronsdon 17:59 identifier

DJ Sampath 18:00 solved a lot of that part of the problem. So so engineers are using that for that uh purpose. Non-engineering staff is using it for creating like presentations. the most interesting thing that we've seen people use it is to be able to create like dashboards um on the fly. Hey, uh what is just happening with the latest pipeline? Tell me where we are in this quarter, how many deals have closed, what kind of customers, give me a quick information about this customer that I'm about

Conor Bronsdon 18:24 to

DJ Sampath 18:24 to go out and meet. And so it's hooking into Salesforce at the back, it's pulling in information that tells them exactly. What is the last most recent touch, and what happened to that customer? And so I think we're starting to see the usage. the way I know this personally is like, well, this rollout was happening. My team is responsible for securing the use of it. And so the number of tokens that

Conor Bronsdon 18:45 be we were restored

DJ Sampath 18:45 starting to see flow through this on a weekly basis. Skyrocketed from like, you know, a small number of tokens to like 2 trillion tokens in a single week. And so they're using it for pretty much any use case that you could imagine. it's currently being used for that. And again, these are very obvious use cases, but unlocking them with company proprietary data, which I think makes all the difference.

Conor Bronsdon 19:08 I want to ask about Codex in particular, because as we speak, we're what, a block, two blocks from OpenAI offices. And my understanding is that Cisco was an early design partner on Codex, had early

DJ Sampath 19:18 That's right.

Conor Bronsdon 19:18 access, helped shape. this incredible product, which by the way, insane that it only came out in February. Yeah. Just mind blowing how much progress has happened this year. OpenAI's case study with Cisco wrote that more than 95% of new AI features at Cisco are being shipped with Codex. what does a human engineer's day look like now with personal agents connecting their Codex instances? How has this all changed over the year as you've gone from design partner and maybe helping shape the early pieces of product to Now securing this ever-evolving scope of agents that are also leveraging Codex. to code. you no.

DJ Sampath 19:54 know, when we got started with uh with the Open AI team, it was it was phenomenal. their willingness to like say, Hey, listen, this thing is super broly. early It's probably going to break, but would you like to test it out? The response from Cisco and my team has always been yes, and you know, we completely recognize that this is early, um, what that

Conor Bronsdon 20:13 What

DJ Sampath 20:13 resulted in in our conversations with them and working closely with them was that It was the lived experiences of the team members inside of OpenAI. How they were using Codex was extremely informative for us to be able to say, Oh, that's that's how they're thinking about it, that's how they've automated the CI/CD pipeline, that's how they're thinking about the tickets. Um, and that evolution that happened inside of OpenAI was so rapid and

Conor Bronsdon 20:34 in

DJ Sampath 20:35 Learning from that was extremely beneficial for a large part of our team members, going, Oh, wow, I didn't realize you could do that. Um you know all of the bottlenecks that there

Conor Bronsdon 20:43 and there

DJ Sampath 20:43 were we kind of had to take our priors and wipe them and say hey these priors no longer hold true and this is sort of how we need to operate.

Conor Bronsdon 20:49 I think we're all having to do that, frankly.

DJ Sampath 20:51 And we're having to do that so often

Conor Bronsdon 20:53 every couple of weeks.

DJ Sampath 20:54 every Yeah. couple of weeks, Yeah. right? And so this was one of those moments for us.

Conor Bronsdon 20:58 One of What are

DJ Sampath 20:58 the things as we brought Codex in, as people have started using it, pretty much to do all of the coding work. Um, you know, they're the pull request reviews at this point in time. I think

Conor Bronsdon 21:08 think

DJ Sampath 21:08 the the big aha moment was like with each and every single um um generation model that comes out. You also have to kind of wipe out, just like you were wiping out your priors, you have to wipe out your skills files because you've built a lot of these things thinking that, hey, I'm going to need help with some of these things. So they've built some. Capabilities around the harness itself to be able to solve certain problems with them. And and we've we had to like completely wipe that out and say, Hey, new model came out. this new model's more capable.

Conor Bronsdon 23:14 I admit I'm less convinced on this argument, so I I'm I want to unpack this a little bit because I totally agree. You need to reassess with each new model generation. The rate of intelligence, the cost of intelligence are both changing rapidly. But I don't think you should throw your skills out wholesale. I know a lot of people are giving that advice. Uh my opinion is you should be refactoring them every time.

DJ Sampath 23:36 it's a good good point that you bring up. I think you're right. I think refactoring is is definitely you know, one of those things. I I think that the thing that I, you know, struggle mostly is with People getting very um um uh How do I say this?

Conor Bronsdon 23:49 Like attached to a workflow.

DJ Sampath 23:50 Attached to their workflows.

Conor Bronsdon 23:52 You have Yes. to change, yes.

DJ Sampath 23:53 You've got to change because there is no point in getting attached to your workflows because You know, I've seen people go like, oh, you know, they give names to their, you know, just like way they name their personal assistants. they have their own bot that's running. Um that's customized and it's like, oh, you know, here's my personal PR bot. I'm gonna call it Jarvis. I think that's our problem right there. Um and so so what we're trying to tell people

Conor Bronsdon 24:15 is like, that

DJ Sampath 24:16 say, hey, listen, the things that you're building, you gotta refactor them. In fact, I'll give you a simple example, right? So A lot of times in the first iterations, It wasn't doing very specific types of security checks in the source code that you would have had. So you you've created the skill file, which is pretty large. And and so what ends up happening is that every time a pull request was happening, You are now loading this inside of your context window for every single back and forth that's going on because that is important for it to be able to go out and assess. that consumed a lot of tokens. It consumed a lot of back and forth that's happening. It reduced the attention on the, especially if your code base is starting to grow in sizes, or your pull request is a pretty

Conor Bronsdon 24:54 which large if

DJ Sampath 24:54 pull you're coding request.

Conor Bronsdon 24:54 with agents, you definitely have a growing code

DJ Sampath 24:56 That's base. right. And and you we have large pull requests coming in, like, you know, pull requests are like forty thousand lines Well, of code.

Conor Bronsdon 25:01 you can limit that if you have good rules for your agents.

DJ Sampath 25:04 Right, but

Conor Bronsdon 25:05 Yeah. But you might understand

DJ Sampath 25:06 want to start like tuning all of that stuff because guess what? The context window sizes for these models are growing with each and every single si i iteration.

Conor Bronsdon 25:12 But it doesn't mean that not having a lot of context does not make it less efficient at times.

DJ Sampath 25:17 That's right. So

Conor Bronsdon 25:18 To your points.

DJ Sampath 25:18 So, so you just have to come back in and say, Hey, what if I didn't need to do these things? Because next generation model already comes in with a lot of these security checks baked into it. It come comes with new types of unit tests that you never thought about because

Conor Bronsdon 25:30 was it's just

DJ Sampath 25:30 thinking through all those things. So, which meant that you can go

Conor Bronsdon 25:32 can

DJ Sampath 25:33 back in and say, Hey, I don't need these things that I put in there anymore. to your point, you're right. You know, don't throw them away, but it's worthwhile block coming in be and willing saying,

Conor Bronsdon 25:39 to, though, too. To your point, yeah.

DJ Sampath 25:41 There's some things that you might not have to throw away,

Conor Bronsdon 25:42 throw

DJ Sampath 25:43 some things you want to refactor, saying, Hey, you know what, here's the context that you need to go get, here are the tools that I'm using. There might be some custom dependencies that you want to still be able to acquire. But I think the most interesting part is People are starting to bring in workflows from like, you know, there are agents that are not necessarily just the My Agents, the thing that we roll out, but there there are other agents that they wanna

Conor Bronsdon 26:01 start to Like bring someone's in. bringing their Muse workflow in, for example.

DJ Sampath 26:04 Right, we haven't

Conor Bronsdon 26:04 we

DJ Sampath 26:04 quite in a unblocked people

Conor Bronsdon 26:06 by

DJ Sampath 26:06 bringing in muse into the workplace just yet, but historically they've had their own bots that they've built using OpenClaw and hooked up via APIs into either an agent SDK that came from. OpenAI or Anthropic. And they were using that to be able to perform a lot of the workflows that they needed. And because the existing bots, I mean, the MyAgent that we rolled out just happened a few weeks ago. they already had these three existing bots in place. And so they were starting to connect each one of these things together.

Conor Bronsdon 26:34 Now, Now,

DJ Sampath 26:34 there are agents inside of the SaaS software that we purchase, right? Now Always, they're they're yeah. agents inside of Salesforce or agents inside of Notion. And so we had to up with a more thoughtful way of saying, hey, how do we first of all recognize and discover these Shadow agents, if you And it's a very common problem that used to exist before called shadow IT. We're using the same moniker to be able to describe shadow agents. And these are agents that have come in via some other sources. So what you got to do is to make sure that A, you discover them. And then once you discover them, you want to apply certain rules of how and what they operate on. and then you gotta also provide, you know, hey, which identities are you allowed to Emulate to be able to get your tasks done. And so once you have those three things locked in, Then you go out and say, okay, listen, I'm going to make sure that I'm constantly observing the actions that these agents are taking. And if they start to deviate, you want to be able to send out an alert saying, hey, is this something that you intended for this agent to do?

Conor Bronsdon 27:27 How are you enforcing mechanisms against these shadow agents that may not be as tuned to the security, observability, and evaluation measures that Cisco has with my agent?

DJ Sampath 27:39 Cisco has a zero trust solution that we use internally. That we also sell. It's called Cisco Secure Access. And we have a mechanism of being able to, you know, say, hey, listen,

Conor Bronsdon 27:49 Here are are

DJ Sampath 27:50 the agents that we love, and here are the agents that we don't love. Um, these are

Conor Bronsdon 27:53 These

DJ Sampath 27:53 not as not fine-grained or granular as you would like, and we're constantly adding capabilities, and so stay tuned for. uh some pretty

Conor Bronsdon 28:00 Okay.

DJ Sampath 28:00 cool stuff that we're going to do over there to be able to look at how do you love nuanced actions that the agents are taking? Right now when we see that somebody's using an agent inside of a solution that is not approved, we just go ahead go and simply put in a policy to say, nope, that's blocked. And so when you try to access that feature, it just wouldn't pop up. It'd say, hey, sorry, this is being blocked. The core belief that we have internally for ourselves, and what we believe everybody should. have is they shouldn't

Conor Bronsdon 28:25 that see C

DJ Sampath 28:25 security as a roadblock. You got to be able to use security as an accelerator somehow. This is one of those rare occasions where You're going to notice that agent safety and security is actually going to accelerate adoption because it's going to give people the confidence to be able to use these tools. And what we're doing is to unlock that for every single agent that we're bringing into our enterprise, whether it's agents that um the enterprise is building themselves or agents that they're consuming from like You know, a SaaS provider or personal agents that they want to bring on board. And you'll notice that the industry is sort of moving the direction where they're opening up like OTel export. capabilities and endpoints where you understand exactly what the agents are doing. And you have the ability to ingest that and say, you know what, I can now ingest this inside of my data lake. I can ingest this inside of my Splunk instance and I can observe what's really happening. Because that increases the level of confidence that you get in terms of using these agents.

Conor Bronsdon 30:28 And I think this is a really key point we should drill in on a bit because The confidence piece, I think, is underrated. We've talked on this show before for the last couple of years about how important it is to have confidence that your agents are going to perform in production. And this was stopping a lot of agentic rollouts, particularly at enterprises, in 2025 and I think early 2026.

DJ Sampath 30:47 No, 100%, right? I going You're gonna to see this is evolution, right? We started out with chatbots. We're evolving to agents they're doing a bunch of things. we're very quickly going to start see physical AI become a real thing. And I think what you're you're seeing that is you're constantly increasing the amount of autonomy that you're giving to machines, starting with like agents sitting in an environment to like You know, agents that are going to run inside of, you know, robots that are going to move around, do things that are going to interface with you in the physical world. I think what's going to happen is that it's to become. increasingly important to understand that slider scale.

Conor Bronsdon 31:19 apply

DJ Sampath 31:19 of like, you know, how much of autonomy and agency are you providing to these systems to be able to effectively do tasks on your behalf. I think it's going to be the reinstatement of dependable systems. We've always done the done study of dependable systems. I can't think we're we're now going

Conor Bronsdon 31:34 want

DJ Sampath 31:34 to start to see security and safety unlocking how we think about dependable systems from the ground up.

Conor Bronsdon 31:39 So, I want to ask about dependability

DJ Sampath 31:41 ability here in

Conor Bronsdon 31:42 because one aspect of that, and a lot of our audience knows about this, is. the engineering side and code review, ensuring accuracy, ensuring you don't have bugs that get in. the approach that Cisco's taking to code review at scale, particularly when I know Codex is driving a lot of your development. But my understanding is you also work with Anthropic and other model providers as well.

DJ Sampath 32:01 We do, yeah. Um you know, I think we're we're constantly Assessing which models are best for which tasks. we're using Frontier models. We're using open source, open rate models. We're building models ourselves, which are post-trained models on top of some of the open source models like Nemotron. And we're using those to be able to perform very specific types of tasks that's trained on proprietary data that we have. And so we're always picking and choosing the the best models for the best uh type of task. Now in the case of pull requests, man um Yeah. PRs became a bottleneck very quickly as the agents started pushing out a lot of code. Um now every single um

Conor Bronsdon 32:40 uh

DJ Sampath 32:40 person has their own respective PR

Conor Bronsdon 32:43 gods.

DJ Sampath 32:43 bots. which provides a little bit of taste and judgment that each and every single engineer has in terms of like how they like to review code. And they've like put those agents.md files that actually describe

Conor Bronsdon 32:54 that

DJ Sampath 32:54 how they like to do the reviews. And so you will see that every once in a while, The review bot

Conor Bronsdon 32:59 block

DJ Sampath 32:59 comes with like requests that you've normally seen that same person like, you know, submit. But it's essentially talking to the agent that actually submitted the pull request. And so you will

Conor Bronsdon 33:08 You

DJ Sampath 33:08 even though you know, look at that and you go, Yeah, you know, that's interesting, but um the agents are the ones that are coming up with the PR and pushing changes back in and then starting to do the merges. So I think we've gotten to the a level

Conor Bronsdon 33:20 a

DJ Sampath 33:20 of maturity at this point that the pull request reviews are being done by agents, but with very specific set of instructions in terms of how you want to think about security, how do you want to think about code clarity? Are you thinking about Um in a modularity, are you making sure that it's checking through like packages that you've used before? Are you making sure that you're using open source packages with the right licenses? are a lot of these instructions that are constantly being shared amongst teams that make its way inside. And on top of that, there's always some amount of like, hey, I like using single ticks versus double ticks. It's a lot of those nuances start to creep in from a taste and judgment perspective, which is kind of cool. But that's sort of how we solved it, right? And I think again, Sometimes it's different models, like You know, if you're

Conor Bronsdon 34:02 if

DJ Sampath 34:02 using Astro to code it up, you know, your opus, you know Uh is probably doing the polar crust review. We try to change some of these models.

Conor Bronsdon 34:10 Multimodal family helps a ton because you catch bias across families.

DJ Sampath 34:14 them. And sometimes same there are some teams that actually switch between model providers. And sometimes they using open source, open-weight models, you know, to be able to do this.

Conor Bronsdon 34:23 Gotta give a self-plug here that if you're interested in great code review, I have a whole article on that on my newsletter at newsletter.chainofthought.show. So definitely check that out. And I am curious to hear from you, DJ. Is your team Also then taking the data from your pull requests. and creating evals based off of that to then Oh 100%. apply.

DJ Sampath 34:42 All the time.

Conor Bronsdon 34:42 know, we're, Uh A, we're if you were

DJ Sampath 34:44 looking at pull requests. B,

Conor Bronsdon 34:46 we we're were

DJ Sampath 34:46 also like you know, a lot of times what ends happening is this kicks off our C D workflows, right? You know, we're constantly doing continuous integration into our existing systems, continuous deployments. And so as we do that in our sandbox environment, we're collecting the traces of how these things are running. using that to understand like hey are they behaving the way that they should be As part of those test deployments, even before we move something into production. And so there's a lot of that. That's going on. We've built the pipelines. It took a good chunk of time to be able to get those pipelines right. And it's worth that investment. But then it starts to speed things up, you know, from all the way from like spec design to like. being able to see that work product that shows up at the end. One of the biggest things we've also unlocked is once a code delivery happens. is to go back

Conor Bronsdon 35:32 go and back

DJ Sampath 35:32 say, hey,

Conor Bronsdon 35:33 why the

DJ Sampath 35:33 rewrite the spec for this at this point in time. And so you can see the spec drift. It

Conor Bronsdon 35:37 That happens from

DJ Sampath 35:38 from like the initial starting point to like where it landed. And recognize and reconcile, like, hey, wait a minute, this is not what I asked for. Or, I didn't even realize that we've evolved at this point in time. So, retros

Conor Bronsdon 35:48 The

DJ Sampath 35:48 look a lot different because you're just looking at spec drift. And you're also starting started to look looking at: hey, how do I retool this entire process? You know Um, and so so we've done a lot of work uh on that front and I think we've had these big aha moments where we we've had to sit back and go, Wow, That's not the way I thought we were going to evolve. this is such a brand new paradigm I work. of how we work today.

Conor Bronsdon 36:09 And I've heard you say before that security today can no longer run at human speed. And that has so many implications, but for the security team members listening, for the CISOs listening. How should they be thinking about this next era? Listen, Was it Addie?

DJ Sampath 36:23 I think yesterday was a you know, very, very interesting day. Railway, Uh

Conor Bronsdon 36:27 PHP, so many other examples.

DJ Sampath 36:28 so many examples in a single day. And I think a lot of people sort of started to understand that this is sort of the new normal. Greg Brockman talked about the defender's window. these models became so powerful at some point that said, hey, there's going to be a finite amount of time that you have from Defender's perspective to be able to leverage these powerful models to be able to go out and figure out where you have vulnerabilities and go start to fix them. Cisco's been part of both the Glasswing and the Daybreak projects. And from both Anthropic and OpenAI to be able to Use these models to be able to find vulnerabilities and fix them and sort of be part of that community. Um I think one of the the big things we learned that whole part of the process is that, you know, and I'm we kind of mute knew

Conor Bronsdon 37:10 us

DJ Sampath 37:11 this even before. was the amount of time it takes, you know, for an exploit to show up in the wild. from the time that a C V E was announced, It used to be months, you know, and if not, sometimes in some cases, those are years. And now it's like sub-eight minutes and it's only decreasing. So which means the minute you publish something, like when the PHP, like one of the interesting data that came out that Matthew Prince was talking about it from with with Cloudflare

Conor Bronsdon 37:35 Pogfli errors.

DJ Sampath 37:37 They saw some amount of attacks on the PHP stuff, like, you know, and there was non-trivial amounts. But the minute the C V E was published and it became a thing, That number went into the millions where they became aware of this, bots started reading these things and instantly started exploiting, you know, any open websites that were so using PHPs. Right. So this is is sort of like like a very uh interesting thing to keep in mind when you think about attacks taxes are now starting to happen at agent speed. Your defenders need to start moving at agent speed as And to do that, you're going to need to invest in. You know, agents that understand your security posture, agents that understand the context of how your applications are being deployed. Being able to recognize the kind of access that you have, the identity that you have. And then coming up with a mechanism of being able to say, hey, hang on, this doesn't look quite right. you looked at the the Microsoft attack that just happened, you know, and you know, with with the whole um you know th the um the the swarm I think the th the thing that it did first was uh you know it attempted one hundred and twenty thousand logins. You know, there were failing logins, but it still attempted that. You know, I think this is a really interesting, exercise in recognizing that if you knew that there are failed logins that are starting to show up. you now have to start thinking about, hey, what do I need to do? to be able to completely retool my my systems so that Yeah. You just block them further and further, you know, farther and farther further away raw from the perimeter. So I think that is really

Conor Bronsdon 39:06 uh

DJ Sampath 39:06 a reimagination of how you can start operating at agent speed, right? you gotta get you know, really smart about I can't compromise this. You got to get really smart about specific kinds of breaches that you're going to start to see. You got to get smart about east-west traffic. As you start to think about how attacks start to propagate, you gotta get smarter about you know, workplace protection. And then a lot of these things will start to come together inside of an agentic SOC. Um and and that's really how you know we at Cisco have been thinking about um How do you start reacting at agentic speed is about Understanding each and every single one of these pieces and then stitching them together collectively.

Conor Bronsdon 39:43 I think it's so interesting to explore the incredible security rollout you've had to do for 90,000 personal agents across the organization with my agent. DJ, thank you so much for having us here to the Cisco offices

DJ Sampath 39:55 SNA

Conor Bronsdon 39:56 today to explore this topic. I do want to say a quick thank you to our presenting sponsors, Inngest. Who do incredible work around long-running agents for helping make this show happen? DJ, where can folks go to follow your work? Hey, uh

DJ Sampath 40:09 yeah, you should check out cisco.com. We put out a lot of interesting blog posts. Twitter handle is Cisco AI. And um I'm uh at DJ Sampath.

Conor Bronsdon 40:19 I'd love to hear from our listeners here as well. Are you using Codex today? Claude Code, T3 Code? Some other open code? What's the harness that you're using for your coding agents? Or have you converted to assigning work to coding agents? from your personal agents. We're very curious to hear from our listeners. So let us know in the comments on Spotify, on LinkedIn, Twitter, X. or YouTube, wherever you're listening, we'd love to hear from you. DJ, I'd love to understand from your perspective. As Codex and AI coding models have accelerated what you're able to deliver on the engineering side with Cisco. while also accelerating the pace and scale of attacks. What are you doing to enhance Cisco Cloud Control and enable defense around the entire enterprise?

DJ Sampath 41:04 Yeah, listen, you know, Cisco Cloud Control was this project just to ground

Conor Bronsdon 41:08 the audience operations

DJ Sampath 41:09 on this as well. You know, Cisco has over like a hundred and you know thirty plus queues over you know, sign it for different products over over, 17 different kinds of platforms, it was really important to start to bring all of these pieces together. And, you know, so one of the missions that my team embarked upon. was this thing about saying, hey, look, we're going to bring all of these pieces together. and we're going to build this thing called Cisco Cloud Control. Now at that point in time, this was uh you know, summer of last year. agentic coding quite wasn't real. It's kind of crazy to think about like You know, just 12 months ago, or a little over 12 months ago, everybody was vibe coding at that point in time, right? and agentic engineering wasn't thing. And uh and we started, you know, hammering away at it and um Roughly around October, November was when We started getting our hands on the previews of some of the agentic coding capabilities and Boy, we lucked out, right. I think it was, uh, was one one of those of moments where We

Conor Bronsdon 42:03 We'd

DJ Sampath 42:03 had set ourselves a a know a 12-month deadline to be able to do something that was next to impossible. And uh and we were

Conor Bronsdon 42:08 would

DJ Sampath 42:08 rescued by agentic coding because All of a sudden, we started seeing this throughput go up. In fact, we, in record time, in a little over eight months, were able to start stitching all of these different parts of the portfolio together and start to offer an experience to our customers where they go, oh my God, this thing is really, really cool. I didn't know that I could. Like troubleshoot and debug from like networks to like security to like observabilities in a single shot. And they started seeing these things come together in ways that. Fundamentally altered how you thought about the infrastructure that you're deploying inside of an office like our Narios Cisco campus right here. So that really

Conor Bronsdon 42:45 was

DJ Sampath 42:47 was something that was remarkable about what we could do with agentic coding. And then we took that harness and we said, listen, we're going to put these agentic coding inside a Cloud Control. So you. Ms. Customer or Mr. Customer, you can now deploy your own sets of capabilities. You can almost sort of think about Cloud Control as a secure harness. you to build agentic experiences on top this. that solves your needs, your your work, your problems in meaningful ways.

Conor Bronsdon 43:12 And I know it can be scary because we are seeing these incidents and there's concerns and we should address them. But this is also the most exciting time ever to be building because So much more is possible today. And I would just encourage our listeners and everyone who's out there coding today to be more ambitious because models are capable of more than you think. You can give it all the context you need and your your taste, your knowledge, your thought processes, your architectural thinking. is so much more valuable today because you can accelerate your execution with agents. And I I couldn't be more excited for the future. And DJ, thank you for sharing your thoughts on it today.

DJ Sampath 43:48 100%. You know, it's time to build.