AI Codes: Product Engineers Decide What to Build | Laurie Voss

Laurie Voss 0:00 Reverse a linked list in Java is not a thing anyone's ever going to need to do ever again. Would I advise somebody to spend 15 grand on a, on a bootcamp? I'm not sure that I would anymore. Agents write code faster than we could ever have written code. The type in the code is going to happen by somebody else now. So you have to, you have to get there in your thoughts. The thing that you used to find in senior developers, which we called product sense. which was not just how to build it, but what to build in the first place, is becoming absolutely central to the job. It used to be called a systems analyst in like the 60s. That was their job, was translating business requirements into software requirements without writing any software. That job is coming back, and we're going to call it a product engineer.

Conor Bronsdon 0:44 AI has made code generation cheap, but shipping trustworthy software is still expensive and difficult. Most teams are responding by either letting agents run a bit wild, or they're slowing them down until the productivity gains start to disappear. We're going to talk about it today on Chain of Thought. I'm your host, Conor Bronsdon, and joining me is Lori Voss, who many of you may know. Laurie is a co-founder of NPM, a developer with decades of experience, a well-known writer, I highly recommend his stuff, and head of developer relations at Arise. Laurie, it is great to see you. How are you feeling about this current explosion of code generation and the moment we're experiencing in AI?

Laurie Voss 1:21 Hey, thanks for having me.

Laurie Voss 1:25 So many thoughts about what's going on in the world right now. I think my overwhelming feeling is that it's positive, which is not a universally held sentiment, I think. I think there's a

Laurie Voss 1:43 lot of people for whom code was a craft, where getting the syntax just right and making the code look pretty and getting exactly the algorithm that they wanted was the whole job for them. And it's going away. And there's no sugarcoating it, right? Agents write code faster than we could ever have written code.

Laurie Voss 2:11 So that means we're in a different world now.

Conor Bronsdon 2:16 It's interesting. I've come to recognize that my acceptance of AI coding is very related to the fact that I've never been a great software engineer. I've always been obvious. I have never been a full-time developer who's extremely good at the craft of code. And so for me, I'm like, amazing. I can take on the TPM role, the semi-team lead role that I am much better suited for than being the frontline developer. And so it's easy for me, but I do see this challenge for a lot of folks where so much of their identity has been wrapped up in not just the problem-solving part of engineering, but in being excellent at these pieces of their craft. And it's definitely difficult. How do you recommend folks adjust to this new reality? Because I do think we need to adjust to it.

Laurie Voss 3:03 Hoof.

Laurie Voss 3:06 I was just talking yesterday to a very good friend of mine who's been in software development about as long as I have, which is a pretty long time. And he's one of those crafts people. And his solution to the change was he's going to retire.

Laurie Voss 3:23 And I was like, well, that is one way of solving the problem of thinking like you're not needed anymore is just not being needed anymore. That is not an option available to most people. I think the most positive way that I've seen it put is for a lot of people, the fun part of coding is the aha moment where you've got from I have this messy amorphous problem to I think I have figured out how to solve this problem in a precise and repeatable way. And some people did that by typing the code. And some people did that by thinking about it and then typing the code. And the change you have to make in your mindset is that the typing the code is going to happen by somebody else now. So you have to you have to get there in your thoughts. You have to figure out the preciseness. uh you know in language um you know and you can do it in pseudocode if that helps you um but it's not it's probably not going to be the fastest way to do it uh by you know doing essays of code and trying out various algorithms like the the agents are going to be better at that than you are

Conor Bronsdon 4:38 We're definitely going to talk a lot more about this topic, but I would be remiss if I didn't mention our presenting sponsors for this episode who are helping make this transition happen for many people. Ingest, Sphix, Walrus Memory, and G2i have been kind enough to sponsor Chain of Thought this season, and you're going to hear a lot more about them as the season goes on and probably later in this episode. But as we all navigate this change in how the craft of engineering is occurring, It's starting to go beyond cogeneration. We're seeing math problems get solved. We're seeing millennium prizes fall. I think there's still an argument that the models aren't that good at writing yet, and that many of these, call it less clearly solvable tasks, are going to hold up for a while longer. But it's very clear that as humans, we're having to interact with the tools we use, in this case AI, very differently than we did six months ago, let alone two years ago. How are you advising people who are either entering the industry today to or are looking to retool to build those goals? So you mentioned like thinking through the problems, focusing on that clarity of thought, the precision of the architecture. But what are the other things that we should be learning or doing?

Laurie Voss 5:49 That's an excellent question. And I don't think anybody knows yet. I don't think we have a great answer. I

Laurie Voss 5:59 think it's clear that, like, you know, university level courses in programming are already feeling very anachronistic. Like, you know, build, you know, reverse a linked list in Java is not a thing anyone's ever going to need to do ever again. So it depends what time frame you're thinking of. If you're thinking next two to five, or if you're thinking ten years from now, those are two very different sets of things, I think, that we have to look at. In the two to five time frame, there's a bunch of stuff that humans used to do that the agents are still not good enough at yet. Like right now, code reviews. I would say the best code review AI tools that I've seen have like an 80% hit rate. And an 80% hit rate is really good, but it's not enough to put stuff out into production, right? You can't mess up 20% of the time and ship that's not going to work out. So there's still scope for senior level code reviews, obviously AI assisted, you can get the 80% done and focus on the last 20%, which is very important given how much more code is being produced. But there's still human insight there. And then there's a bunch of stuff that the agents are just not good at at all right now. operations is one. Like, I don't see anybody saying, okay, I've written this piece of code. I'm going to ask an agent to deploy it. I'm going to ask an agent to figure out what hardware it should go on and how many pieces of hardware and connecting those pieces of hardware together. um that is still a thing humans are doing and it's not a thing that they are trained on very much um and when i've uh so i expect i expect the asians will get there i expect it will be a thing that they are good at 10 years from now but deploying the software, refactoring the software, architecting the software in the first place, and scaling it up.

Conor Bronsdon 8:13 Hmm.

Laurie Voss 8:15 Those are definitely tricky, subtle areas where you need a lot of domain expertise and you need a lot of stuff that we have never written down. which I've seen referred to as tacit information, which is a thing that agents are notably bad at, and they'll continue to be bad at it for the next five years. So we don't have to do everything overnight, I guess, is the positive news. You don't have to, like, completely reskill and all, you know, every programming job in the world isn't going to vanish overnight. But over the next five years, we do need to, a whole lot of people do need to do a whole lot of retooling.

Conor Bronsdon 8:51 [OVERLAP] I do think there's a challenge here as far as the gap between getting started in the industry and folks who are farther along. It's a lot easier for people who have, you know, tacit knowledge, tribal knowledge, who have experience to take advantage of some of the execution gains these tools provide and leverage models and help guidance their models than I think it is for many folks who are early in career and are just trying to figure out the basics. And it does feel like we have an increasing gap here. You mentioned university coursework and how it's seemingly falling behind. I'm not sure I would go to college today if I was 17, 18 years old. Would that be the best use of my time compared to trying to spend a lot more time building and maybe get some internships or something? I'm not sure. Now, am I going to land an internship without an authoritative logo behind my name? Who knows? It's tough for me to reckon with, like, what would I advise a very early career professional or someone who is trying to get to their career? And I'm curious

Laurie Voss 9:51 [OVERLAP] I

Conor Bronsdon 9:51 [OVERLAP] what you would say to those folks.

Laurie Voss 9:54 think it depends very much on what kind of course you've been doing. For the last 15 years, there have been a lot of boot camps. And I have always been a big fan of boot camps because they get people going relatively cheaply, relatively quickly.

Laurie Voss 10:13 But boot camps are very code focused. They're very much like, learn this particular framework. And that is much less relevant now. So, you know, would I advise somebody to spend 15 grand on a bootcamp? I'm not sure that I would anymore. University courses vary a lot. There are some universities which are very hands on, very like, here's how to reverse a linked list. This is how you write code. This is how you do stuff. And there are other more theoretical courses which are like, this is what, you know, this is what big O notation means. And this is what This is how you think about complexity, and this is how you do a mathematical proof

Laurie Voss 10:57 that your software is correct. Those high-level things are still valuable. They are not hands-on stuff that you use every day, but they give you an idea about how to think about the more complicated parts of our profession.

Laurie Voss 11:17 And as you say, there is a certain amount of privilege in being able to say, I don't think I would go to college, right? It's a thing that white guys can say, is what a friend of mine has put it to me, is like, because you can just walk into a job anyway, whereas everybody else needs a degree that says, hey, this person is legit before they can land an internship, before they can get their foot on the ladder. But your broader point about what does the ladder look like is very much in flux right now. I think all of the same skills that I was just talking about people retooling into, so architecture, operations, scaling, complexity, design, all of those things are things that can be taught. And if you look carefully, there are people already teaching them. But you have to look a lot more carefully than you used to.

Conor Bronsdon 12:16 When you say, you know, you have, you need to look more carefully at what's being taught and where are you thinking specific to coursework? Are you thinking about more of the free online learning that's out there? You know, how would you go through this process granularly, uh, if you were starting on your career today?

Laurie Voss 12:36 [OVERLAP] Gosh, it's so tricky. If I was literally just going to college for the first time, I would look for a high-level CS course that is more theoretical than practical. So I wouldn't be like, you know, the nuts and bolts of Java, which is what they were teaching when I went to the university 25 years ago. They probably teach other things now.

Conor Bronsdon 12:56 [OVERLAP] I think that was my first CS course I ever took. Yeah.

Laurie Voss 12:59 Yeah.

Laurie Voss 13:04 So I would go for a course that is deliberately more theoretical. And I would probably look into, there are courses that blend in humanities into computer science. at various universities where they're like, and we will teach you how to think about the philosophy of computation, and we will teach you how to think about the business of software. Those sorts of things that previously a programmer might have considered tangential, they are becoming the whole job. This is something I wrote about recently. I wrote a blog post called we are all product engineers now. And the thesis of that was basically the thing that you used to find in senior developers, which we called product sense. Which was not just how to build it, but what to build in the first place is becoming absolutely central to the job. So that is the skill set that I would be looking for is like what skills can I learn that will help me figure out how to decide what to build, rather than how to build it.

Conor Bronsdon 14:14 Season 4 of Chain of Thought is delivered by Sphix. We spend a lot of time talking about what agents need in production, and one of the least glamorous answers is events. Your customers want agent workflows that react to things happening inside your system, which means your API needs webhooks that actually work. Not just a post request and a prayer, retries, ordering, idempotency, replay protection. Sphix does that as a service, and they wrote standard webhooks, the spec that Anthropic, OpenAI, and Google bailed against. So if your API doesn't have reliable webhooks, that's turning into a lost deal. Join Brex, Dorada, Daytona, and many others on Sphix. Get started at link.svix.com slash c-o-t or go to the show notes to grab the link. Qualified startups will get $12,000 in credits. $50,000 for YC companies. I can't recommend Sphix enough. I'm a huge fan of their open source project. I've actually contributed a bit myself and they're so easy to integrate with. I think you'll really enjoy it. Check out Sphix at link.svax.com slash C-O-T. I feel like we've gone a little dark in the start, right, where it's like this really challenging moment in software to get started. But it's also a magical time right now where we're able to do things with a computer we have never been able to do before. We're able to turn ideas into code and see them appear before us with less physical effort than we ever have before. And I wonder if you see parallels to other periods or other experiences you've had throughout the industry.

Laurie Voss 15:55 The

Laurie Voss 15:59 parallel that I draw all the time is that AI right now is like the web in 1997. So if you consider the sort of birth of the web, when it exploded, to be 1994, which is sort of like roughly when the AOL exploded and everybody suddenly was online for the first time. Three years elapsed from 1994 to 1997 and a whole bunch of stuff had changed. A whole bunch of stuff had gone really crazy. You know, there was an incipient bubble. There were massive changes to what people were learning and what people thought that they could build and all that kind of stuff. But it was still super early. 1997 isn't even 1999, right? Like, 1999 was sort of like peak bubble. and there were two more years to go before we hit that. And so I feel that's basically where we are. Like, if you consider 2023 to be when everything really kicked off for AI, you know, late 2022 maybe, then we're in 1997 right now, and even though it feels like everything has changed, we are just getting started. We do not know where this stuff is going to shake out. We do not know who the big players are going to be. Like, who was the big internet player in 1997 versus who did it actually turn out to be? Those are quite different companies. If people are feeling like they've missed the boat and their skills are behind, now is still really early. You can absolutely jump on right now and have a really great time, you know, until the bubble pops, which it will.

Conor Bronsdon 17:41 Yeah, it's interesting. I, you know, I'm a huge believer in the technology. It's obviously reshaping the world and where we're at. Are many companies in a bubble of valuation? Sure. For sure. I mean, like there, there's definitely going to be some of that. And I'm, I'm very interested to see who is durable here. I do think the model companies are going to be durable for the frontier labs. Um, but far be it for me to, to know for sure by any means. And it does feel like we may be. overestimating what AI can do this year, today, right now, at times, while underestimating how it's going to reshape the economy in the next couple of years.

Laurie Voss 18:19 Yeah, I mean, what we're seeing right this week is, you know, three of the major model labs all going, whoa, whoa, whoa, we need to slow down and think about things. Which is a very strange thing to see from three of them simultaneously. um uh and that is because if you're deep inside of it um the rate of change the rate the pace of improvement is such that you see an exponential growth curve and exponential growth curves are counterintuitive they are humans are really really bad at extrapolating what is going to happen when exponential growth kicks off, even when they know it's exponential growth. And I had a lot of really direct experience with this when I was working at NPM, because NPM grew exponentially for 10 years. And every year we were like, well, this is the year that downloads will sort of taper off and we'll be able to get our scaling up to snuff because it can't possibly grow this fast forever. And it did it every year for 10 years. So, and that's the same thing people inside the labs are going, they're going like, this is really fast. And if you look to where we were a year ago versus where we are now and say that we're going to be, you know, five times better than that, 10 times better than that another year from now, then that is an even more dramatic change. So maybe we should stop and think about that. But I temper that with having been around long enough that I've seen lots of bubbles. And we are certainly in a

Laurie Voss 20:07 valuation bubble and a hype bubble when it comes to AI. Like, there's a whole bunch of stuff that they can do. And unless we literally invent AGI and a super intelligent robot takes over all of the economy and everything that humans do and turns us into sort of treasured pets, It's going to be a normal technology. It's going to be, you know, we're going to hit the hype cycle. We're going to hit the Pico hype cycle. People are going to go, oh, no, actually, AI was useless and it could never do anything. And valuations are going to collapse. And a lot of people will lose their shirts. And then we'll slowly climb up back to roughly where we thought we'd get to, but it will take us much longer than we thought. That is my Bayesian priors on how bubbles go, and it is how I think this one will go as well. People are extremely hyped right now, and I do not think that we are going to invent, you know, I don't know if you've ever read the culture novels by Iain M. Banks, but they're set in the universe where super intelligent AIs basically keep humans as pets, even though the humans don't actually have anything to do. And I don't think we're going to do that. I don't think I don't think predicting tokens one after the other is going to get us there. So I think it's just going to be a normal technology that changes a lot of stuff and makes things go a lot faster and does a lot of changes to the economy. But it's not going to be this super intelligence that that that ends the economy.

Conor Bronsdon 21:48 [OVERLAP] It's a really interesting question and I vacillate a bit on my expectations around this, but I do think part of the challenge here is that for a lot of people using the term AGI a ton. And I think if you showed me Codex or Cloud Code today, four years ago, I'd be like, oh, this is AGI. Like, my God, what can I do with code here? But it's not ASI. It's not super intelligence. And I think that is the really important thing to remember. We can get caught up in the definitions, but assuming we don't get to super intelligence that does turn us into pets or, you know, kills us all, who knows? Anytime soon. You know, we are in a situation where certain things are getting solved, like coding. And we are able to execute much faster. We're able to hand off work much more easily. And we're seeing this already affect how companies are structured. You know, we saw Block decide to lay off, what, 40% of their staff at the start of this year. And now we can comment on whether that was positive for them and how that went, and they've certainly done more hiring since then. But we're seeing vast restructures that are at least excused and explained from them from an AI perspective. We're seeing

Laurie Voss 22:58 [OVERLAP] Mm-hmm.

Conor Bronsdon 22:58 [OVERLAP] companies stay smaller for longer, and people try

Laurie Voss 23:00 [OVERLAP] Mm-hmm.

Conor Bronsdon 23:00 [OVERLAP] to found small companies that can do much more in revenue. And it does seem to be reshaping how companies work. And I don't know that anyone's quite figured out how a company is going to look in the next couple of years based off of how many agents we're all starting to run.

Laurie Voss 23:14 I've written about this as well, actually. On my blog, I did a bunch of research, or rather, I got an agent to do a bunch of research about employment stats and Bureau of Labor Statistics things. Really great stuff to point an agent at, by the way. Just like, look at 500 spreadsheets and figure out a trend, please. And what you said is true. Companies are staying smaller for longer. They are more capital intensive per human. So people are putting more money in, but hiring fewer people. And that is a, the optimistic reading of that is that that is a productivity gain. these companies are being vastly more productive per individual human employed. And people go, oh, that means everybody will have no jobs anymore. No, that's not how the economy works. Usually, if you have, if you can do things more efficiently, it means that things that wouldn't previously have simply not got done, get done instead. So those people who are not going into company A, they form company B, and company B tackles some problem that previously was not economical to tackle. So, you know, the net effect is more stuff gets done more efficiently, and that's better for everyone. That is generally, you know, that is the lesson of the industrial revolution, was that, you know, getting a machine to do what 500 people used to do didn't mean that there were 500 people around sitting kicking their heels. Those 500 people found other things to do, and the whole economy grew.

Conor Bronsdon 24:57 [OVERLAP] Season four of Chain of Thought is presented by Walrus Memory, the portable memory layer for AI agents. Your agent has learned your code base and how you work. Now you want to use that context in another tool. Exporting a file gives you a snapshot, but what happens when the context changes? Walrus Memory is a portable memory layer that lets your agent store context for later use across tools. Python and TypeScript SDKs plus native MCP support let you connect it to your agents no matter where they are. You said who can read and write, so sharing context doesn't mean opening up your whole memory store. If you're building across tools or model families, take a look at Walrus memory. Learn more at walrus.xyz slash COT. That's walrus.xyz slash COT. And I will say, I personally think we are in a new industrial revolution. I know the term revolution is being overdone with AI, but it does feel like we're in that kind of moment as long as we can get the inputs right around data, electricity, et cetera. And I will also, since you didn't plug it yourself, you said you mentioned it's on your blog, but seldo.com is Laurie's blog and is highly recommended. I will certainly link it in the show notes. Got to got to plug that. Speaking of that blog, you recently wrote about product engineering, which we've talked a little bit about. but you've also written about open source and how open source is being impacted by AI. I'd love to get some of your thoughts around what we're seeing in the push and pull between open source models and open source code in today's

Laurie Voss 26:28 [OVERLAP] Mm-hmm.

Conor Bronsdon 26:29 ecosystem.

Laurie Voss 26:29 So I'll set open source models to the side for a little while because that's a very interesting topic and I want to talk about it, but it's different from the effect on open source code. So open source code is the vast majority of software that gets run is open source. It's a thing that people don't think about very often. Like if you think about like instances of running software, the vast majority of it is in fact open source. And then there's like a little sort of icing layer of closed source running on top of it. And the closed source makes all the money, but the open source is doing all of the work.

Laurie Voss 27:06 What I went looking for when I was researching the post about open source, which was mostly about how you fund open source and where does the money come from, was that open source is sort of the canary in the coal mine of where software is going to go. Because there's so much of it and because so much of it is open, it moves surprisingly quickly. And so they felt the problems that other companies are just beginning to feel really early on. They were like, we are drowning in slop. There are a zillion billion AI agent PRs landing on our repo. We can't possibly review them all. And they felt that a solid six months before every other company was like, we've started generating code so fast that we don't know how to review it. And they haven't solved that problem, but they felt it first. And they also have experienced

Laurie Voss 28:11 the disruption of the cost of code going to zero, which is a really good example. I think it was Cloudflare released a sort of clean room re-implementation of WordPress. I forget what they called it, but they were like, here, here's WordPress without all of the cruft. And it runs perfectly. And we cloned it in a week and spent $100,000 worth of tokens on it. And that was at a time when there was a lot of turmoil in the WordPress project. And a point I made in the post is that this has always been a force in open source software. It's why MIT and other permissive licenses tend to win, is because if you try to come up with an open source project that is slightly less open, that is slightly less permissive, something that, for instance, charges you for access to binaries or something like that, it immediately loses to a fork of the same code that has an MIT license instead. And people, you know, rapidly aggregate around those most open alternatives. And that keeps open source as open as it can possibly be. It makes it very hard to fund open source, but in general it's a positive force. And that is going to start happening to closed source software. Again, it's not happening quite yet because there's, you know, operations and scaling and the other things that we talked about that are more than just, you know, I have cloned Salesforce in my repo in two weeks.

Laurie Voss 29:59 But they're absolutely going to get there. Closed source software is going to find that somebody with a weekend and a whole bunch of tokens is going to clone their idea without stealing any of their code. And that is going to disrupt what they can charge for and how they charge for it. And that is going to produce a lot more open source, is my prediction, as the cost of generation goes to zero.

Conor Bronsdon 30:26 I mean, I think we're seeing this for a lot of individual developers who are saying, oh, I don't want to buy a tool for this side project. I'm just going to build it. You know, there's been some hackathons. I participated in one of Sphix's latent space kill my sass hackathons where it's like, hey, try cloning this project. And, you know, we're given a weekend and it's like, okay, let's let's clone

Conor Bronsdon 30:49 our version of event management software. I'm not going to say that you can necessarily do a full clone that has feature parity that is as strong as software that's been worked on for ages, because as much as I think everyone wants a clown on SaaS, there is value behind this. And particularly if you're an enterprise, it's like, okay, I have security concerns, I have compliance concerns, I have role-based access concerns. But for simple use cases, like I'm not going to pay for SEO software for my podcast right now. I'm going to build something simple with free tools that can, you know, do that. I'm not going to pay for social listening software for my podcast and, you know, track all these forums out like what's getting mentioned. I'm, you know, there's free versions already I can build on top of and I can, you know, throw my codecs or cloud code agents at it. and this kind of personal software era is definitely developing for I think folks who have some engineering priors. I am going to be really interested to see how much it expands into the broader world though because The truth is that most people don't really want to spend their time building a personalized software stack. They probably just want to use Meta's Muse, which is much easier to pick up and can do most of the things as well as they need it to. And frankly, so do I sometimes. So I'm curious if you think we're going to see an explosion in personalized software outside of, you know, a core group of developers who want to throw code at things. as the models get better and as people get broader access to code development or is this going to be fairly

Conor Bronsdon 32:28 centralized right now in like a certain grouping of people?

Laurie Voss 32:32 We're getting into what I was writing about when I was writing about product engineers. There is a skill that some senior software developers have, which is the ability to talk to a stakeholder and figure out what it is that they actually need. And domain experts usually don't have that skill, right? They have the other skills. My example in the post was somebody who runs a bakery. You have no idea how to make the perfect croissant, but they have no idea how to write bakery software, and they are not interested in finding out. They just want something that does the things they want to do. They could tell you probably, if you interview them one question at a time, what it is that they want to be able to do and what they want the software to be able to do, but they're not going to have any sense of how to put that together and how to make it work and how to be repeatable. and all of the other things that somebody whose job is thinking about software is good at doing. So that's a job. It used to be called a systems analyst in like the 60s. That was their job, was translating business requirements into software requirements without writing any software. And that job is coming back, and we're going to call it a product engineer. Their job is going to be to figure out what the user's requirements really are, and then work with a team of agents to implement that software, even if they're not writing any code themselves, and that's still a job. In fact, that's possibly the only job left when it comes to software development.

Laurie Voss 34:09 Certainly in like a 10-year time horizon, as they get good at all of the other themes. So that's that is going to, to your question, lead to a lot of not necessarily personalized software, but certainly much more specific software, right? Like at the moment, if you're running a bakery, you're probably running like generic small business point-of-sale software, and that is going to be different from something that was specifically about a bakery, right? I don't know how it's going to be different because I don't run a bakery, But there's going to be, you know, things that you experience right now as friction, stuff that you experience as pain points in your use of that software, that if somebody had built that software just for you, would work out better. And we're going to see a lot more of that. The same thing that I was saying about companies being smaller, Like, the company that writes point-of-sale software is going to get a lot smaller, and in its place are going to be 10 other companies that write point-of-sale software for specific industries that do a much better job of nailing it for that specific industry. And that's going to make everyone's life easier. That's going to make the software experience better. It's just generally going to be positive. But it's a big change, right? Because there used to be one big company, and now there's going to be 10 small companies.

Conor Bronsdon 35:36 Yeah, I'm going to be really interested to see how this affects VC and funding as well because I think we're going to see A lot more folks are just saying, I don't really want to raise a round beyond maybe a couple angel checks from friends to get started to build my company because I am going after such a niche market right now. So I wonder if we're about to enter, or maybe we already have entered an era of accelerated bootstrapping for engineers who want to build companies because look, they can take on, if they have product thinking and some sales skills, the code side of it can really be handed off to agents and those agents can also help with a lot of logistics as long as they can help have someone to oversee them on operations.

Laurie Voss 36:21 Yeah, I mean, I do think we are seeing some of that already. There's a lot of bootstrapping happening. It does mean that there is an opportunity in the DevTools space, which is funny because I've obviously spent most of my career in the DevTools space. All of those agents are going to want to not reinvent the wheel, and they're going to want platforms and things to build on top of. So there's going to be an explosion of potential for those. And those are the things that will be VC sized opportunities, things that are sufficiently generic, that everybody writing software will want them. I'm sure you can think of some examples of the kind of stuff that that would be. Those are going to continue to do very well, while the individual implementations are going to get smaller, and to your point, less VC sized.

Conor Bronsdon 37:13 So, okay, we've got a few different trends we've talked about here. One, the idea that we're still in a bubble, even if things are progressing rapidly, like look, The industrial revolution was a bubble in some cases, the internet was a bubble in some cases. It still vastly changed how the world worked. And as that's happening, we're seeing this change in education and educational outcomes, where people should be focusing their time. We're seeing challenges as the latter gets kind of pulled up for juniors in some cases. And then we're also seeing

Laurie Voss 37:39 [OVERLAP] mhm.

Conor Bronsdon 37:39 [OVERLAP] smaller companies staying smaller for longer, companies staying private for longer. We're already seeing that with like the Databricks of the world. But I mean, it looks like open AI is going to delay their IPO. Anthropic may delay their IPO. And given the pause conversations, I'm not like stunned by that. And or and then, of course, we're seeing this kind of personalized software product engineering stuff that you're talking about here. As all these trends come together, as the stew of these changes occurs, what do you think this means for the model companies? You know, some people think the model companies are going to eat all of software, eat the economy. You seem less convinced of that in the short term.

Laurie Voss 38:20 I'd say in the short term, while there's still a lot of headroom for how good the models can get, the model companies are going to be able to get away with doing what they're doing right now, which is charging 80%

Laurie Voss 38:35 margins on tokens over the API. And that's why Anthropic has reported two straight quarters of profitability, which is amazing given how much money they're spending. And that implies that their API is extremely profitable. And that is because, you know, Fable 5.1 is notably better than Fable 5 was. And people are willing to pay for that.

Laurie Voss 39:03 But right behind them, and ever increasingly close behind them, are the open models. The models from the Chinese labs are, you know, they cost a tenth or a hundredth of what the Frontier labs are charging, and they do 90%, 95% as much. So a lot of people who are scaling stuff up are already switching to open models. So that puts a ceiling on how much money these models can make and how much market share they can really capture, and how much instead goes to, you know, the neo-scalers of the world. Where is the ceiling? How much better can these frontier models get before you begin to run into scaling

Laurie Voss 39:53 laws or diminishing returns on how smart they get? Because if we run into one of those, then the Chinese models will catch up and the window where you can charge for the better intelligence collapses, and so does their profit. And at that point, they lose their shares. So if I was betting on how does the bubble pop, why does the bubble pop, it's because they run into some kind of scaling law with LLMs. They can't get transformers to do any more work. the models begin to see diminishing returns, and the open labs catch up. And then their profit goes, you know, to approximately zero, and the valuation does too.

Conor Bronsdon 40:38 This episode is sponsored by G2I. I've said before on Chain of Thought that most teams still treat evals like unit tests. Write them once, check a box, and move on. That doesn't hold up once an agent is making decisions. G2I is about to close that gap. For over a decade, they vetted and placed engineers at other companies, from startups to FAANG. Two years ago, they turned that same judgment inwards, building their own bench to review our own environments, evals, and training data that models are trained on. These reviewers know the difference between code that runs and code that's actually good because they've shipped it themselves. If your team needs that kind of work and doesn't have the engineers for it, check the link in the show notes to bring them in. I think the other challenge they're facing here is also that Meta and Google, I mean, they can stay profitable longer than pretty much any other company in the world with the scale they're at. And, you know, we can all talk shit about Gemini models for all we want to, but they still have a very strong research team. I think Gemini 4 will be pretty darn good, honestly. And they have advantages around cost and speed and integration. Now, Google has plenty of problems. I'm not, you know, waving a flag for Google winning everything. But it still creates competitive pressure on a closed model side. And we're seeing this, I think, come back from meta as well with Muse and some of the work they're doing with Spark, where that model is extremely cheap compared to the frontier intelligence from Fable 5.1 or GPT-Astra. And if there is a slowdown that impacts how far ahead the frontier is, to your point, it does become a lot harder to charge premier prices if I can get by with Muse or something else that's much cheaper or a fully open model. I do wonder, how much distillation is impacting the frontier for Chinese models, because we've seen these accusations from Anthropic around Moonshot supposedly routing traffic to cloud models to essentially steal their traces. But personally, I see DeepSeek and the work they're doing, and it makes it hard for me to think it's all distillation-based. There are incredible researchers in those open model labs, and it seems pretty clear that we're going to continue to see gains out of China. It's hard for me to picture a world where the frontier labs in the U.S. are able to both pause and kind of slow down their push for recursive self-improvement while also staying a long ways ahead of the Chinese labs.

Laurie Voss 43:15 Yeah, I completely agree. I don't think you can do those two things at the same time. And so that's, that's what the The shape of the popping of the bubble will look like.

Conor Bronsdon 43:24 Assuming that Anthropic and OpenAI recognize this and stay private for longer, and I presume they're going to be able to raise a ton of money to do so, it looks like, I think as we speak, OpenAI was doing like looking at like a 1.2, 1.4 trillion dollar round supposedly, which, good lord, the figures on this. But assuming they decide not to IPO quite yet, There is some protection as far as how much they can lose their shirt in the short term, but what do you think this bubble popping looks like if there is a continued catch-up from open model labs and competitive pressure from closed labs that simply have the ability to stay solvent longer in meta and potentially SpaceX and definitely Google?

Laurie Voss 44:10 I think it looks like every trough of disappointment always looks like in a hype cycle. So valuations go to zero. A lot of people lose their shirts. The people who lose their shirts get sour grapes. They go, oh, this stuff was was you know it was just crypto all over again this was never good it never solved the problem it was garbage the whole time and then a much slower climb back to a plateau of productivity um where the things that we thought uh would happen in three years happen in 10 or 20 years like this very much happened in 1997. Like, there were definitely people in 1997 who said, the internet is going to change the nature of democracy and how media works and how everything that we do, everything is going to work. And they were like, but it's going to happen by 1999. So you need to invest in toys.com right now. And they were wrong, but only about how fast it was going to be, right? Like, Pets.com was like the poster child for the ridiculousness of 1999. Chewy.com is Pets.com. Chewy.com does great business.

Conor Bronsdon 45:27 [OVERLAP] Yes.

Laurie Voss 45:27 [OVERLAP] It's exactly the same idea 20 years later. It works very, very well. And the internet did change the way we consume media. And the internet did change, for better or worse, the nature of our democracy and how we talk to each other and every other thing. So the things that people say that AI is going to do, I believe that they are correct. I don't think that it's going to be 12 months from now.

Conor Bronsdon 45:51 Yeah, and I think this is the the challenging thing we're having in this moment is how to align your long-term expectations of how sci-fi things make it with the short-term of like, how do I react in this moment? And there's so much noise coming at us, you know, from people who want to hype up a certain model or people who just are uninformed on all ends of the spectrum. that it feels hard to stay focused on on what you should be doing in the moment and adapting to these these vast changes that are occurring. So as we kind of navigate this, you know, we've talked a lot about potential implications of code generation on how software engineers have to react to this. And we did touch on this a little earlier, but I think a really crucial part of success right now is being able to curate the information you're providing to the model, whether that's for your company or for yourself as an individual. You know, there are some of this comes down to the information that's not written down and be able to pull in that product thinking, thinking that you talked about earlier in product engineering. But assuming that there continues to be a ton of power and leverage you can create through curating the data and context we provide to the models right now, what's the approach that people should be taking around building their company brain, their second brain to use overused phrases?

Laurie Voss 47:18 [OVERLAP] The thing that has always generated really great companies and really great businesses is the combination of two previously unrelated domains. And the easiest form of that intersection has always been, I know a whole lot about Internet technology, and I also know a whole lot about, pick any other random domain. Right? If I can, it was given a whole name. It was called digital transformation by the consultants, and everything was going to be digitally transformed. And they were not wrong, right? Everything's going to have an internet version of itself. And those generate a lot of value. So if I was telling somebody like what to get really good at, I would say, you don't need to get really good at technology because the technology is already getting really good all by itself. What you need to get really good at is domain. Right. Like to use my earlier analogy, like, are you potentially a really good baker? Because if you

Conor Bronsdon 48:28 [OVERLAP] I'm not,

Laurie Voss 48:28 [OVERLAP] are.

Conor Bronsdon 48:29 [OVERLAP] to be clear.

Laurie Voss 48:30 Like, I don't know what your area of expertise is, but if you're like, I'm a really good baker, or I'm really good at agriculture, or I'm really good at auto parts, or like some other domain that doesn't have a lot of internet and doesn't have a lot of AI in it right now, getting really deep on that domain is probably going to be more productive than trying to beat everybody else who's trying to win on the technology side of that intersection.

Conor Bronsdon 48:55 But I could just learn graph engineering, Laurie. That should solve all my problems.

Laurie Voss 48:59 Yes, absolutely. I've yet to see anybody describe graph engineering to me who I wasn't sure was actually making fun of it. There have been various posts about it and I'm like, this person is making a joke. I'm like 80% sure.

Conor Bronsdon 49:20 Season 4 of Chain of Thought is presented by Ingest. Agents in production run long. They call models and wait on APIs and people. But the longer agents run, the more they break. Ingest handles that with durable execution. You build your agent as steps in TypeScript, Python, and Go. When a step fails, Ingest retries it with exponential backoff, and completed steps are saved and skipped. A run can wait on an event for minutes or for months. You can trace runs and replay past runs against new code, and Ingest has built-in observability, evals, and experiments so you can wrap your agent tools and steps and use it as a full-on agent harness. The dev server is open source and you can get started for free. Grab the ingest link in the description or visit inngest.link slash cot dash pod to get started. All right. So if product engineer becomes this new category and it's not a graph engineer short term, please, Lord. Should we all though be retraining in this direction of saying okay let's combine domain expertise and the context we can gather with it with execution? I mean I can totally see a version of the economy that looks like that over the next couple years and you know models keep scaling but they don't quite hit the exponential that people expect. Um, companies stay smaller because of this and are more agile. Maybe we see more continued growth in bootstrapping and open source,

Laurie Voss 50:43 Boop.

Conor Bronsdon 50:44 more and more people trying to use open source, uh, stars to get themselves a funding round too, as well as I'm sure it's just definitely something where we're seeing. And I will roll my eyes a little bit at that. Um, it, it does feel like we still are maybe missing a part of the ladder where getting people from, you know, educational knowledge into that, like junior engineering role still feels challenging. We're, we are seeing a drop off in hiring of juniors right now in many companies. We haven't yet seen enough growth in apprenticeship to really solve this issue at this point. How do we solve that as an industry? How do we make that connection? So there isn't a generation of engineers, of people who want to become product engineers that are getting kind of left behind by the industry and where it's moving.

Laurie Voss 51:33 It

Conor Bronsdon 51:33 [OVERLAP] I know

Laurie Voss 51:33 [OVERLAP] is

Conor Bronsdon 51:33 [OVERLAP] I'm

Laurie Voss 51:33 [OVERLAP] the

Conor Bronsdon 51:34 giving you really easy softballs.

Laurie Voss 51:37 question of our time. I think to get back to sort of the dark tenor of the initial five minutes of our conversation, I think the pace of the change is such that it is impossible that there isn't a cohort who have a really bad time. There's the people who started college three years ago. to pick a really great example.

Laurie Voss 52:05 They were locked in to what their course was going to be about. And in the intervening three years, every single thing changed. And what they learned and what that course was laid out to do is about a world that doesn't exist anymore. And those people are going to inevitably have a rough time entering the job market because they were trained for a thing that vanished while they were training for it. To your point about apprenticeships. Google famously started the APM program, which is a program that produces product managers, and it's being copied at a handful of other big companies. Those companies are doing the thing that we need to do, but they're doing it at the scale of, like, maybe 1,000 people a year, when we need to be doing it at a scale of 10 or 100,000 people a year. So there's some prior art there that we can look at and go, we know how to put product people together, we think, we hope. We need to scale that all the way up. That is what the new boot camps need to look like, is how do I level up at systems analysis? Because that is the skill that we need more than ever and is incredibly valuable to do all of this domain-specific stuff that we're talking about. And it's not what the domain experts are ever going to want to do. They're too busy baking good stuff and making auto parts.

Conor Bronsdon 53:32 Laurie, it is always fantastic talking to you. I really appreciate the wide-ranging nature of this conversation. And again, I'm going to plug zelda.com to read Laurie's writing and follow his work because I think his thoughts are both deep and broad, and it's always interesting to dive into them. I'd love, Laurie, if you don't mind sharing a few closing thoughts with the audience about navigating this turbulent time, or maybe if you want to end on a slightly inspirational note, given the dark tenor we've hit here sometimes.

Laurie Voss 54:01 I think it's not hard to find the positive in what we are talking about. I think long-term, it's going to be positive. The Industrial Revolution was long-term positive for the human species, right? Like, it lifted a bunch of people out of poverty. It improved standards of living. Like, no one wants to go back to pre-industrial life. Well, almost nobody. um you know and it produced a bunch of externalities and problems as well that are we should be open-eyed about and ai is doing the same thing but it is going to be the same shape of transformation a lot of things are going to get better they're going to get better faster than we've ever seen them happen before And it's 1997. You can jump on this wave and ride it to heights of productivity that your ancestors could never have dreamed about. That's very exciting. It's going to involve changing what you thought you'd have to learn three years ago or two years ago. But it's not too late. You can change your mind. You can retrain. You can figure out what it is you need. and you can do very, very well.

Conor Bronsdon 55:10 to pull from the name of another AI podcast, no priors. Yeah, I think it is time to update our priors and to really think hard about what the future looks like and be willing to change our mind rapidly as new information comes at us. But it is also like one of the most exciting and inspirational times. Like I said earlier, there are things we can do with computers today we have never been able to do. Computers are getting better, they are getting faster, we can do more things with code, we can do all this crazy 3D modeling with Astro suddenly that we couldn't do a few weeks ago. And there is going to be another jump in a month, I'm sure, where we'll talk again and I'll say, well, this thing has completely changed. And that is such a fun, exciting time. And 1997 was a fun, exciting time. There's a lot of money to be made. There's a lot of productivity to do. A lot of businesses to create. There's a lot of fun, interesting things to learn. And Laurie, I really appreciate you joining me today to talk about all of it. Thank you, everyone, for listening to this episode of Chain of Thought. If you enjoyed it, drop a comment, whether you're on YouTube, Spotify, wherever else. We always really appreciate hearing from our listeners. And Laurie, just thanks again for joining us. This has been great.

Laurie Voss 56:14 Thanks for having me. This has been a great conversation.