Agentic Development: Prime Radiant • Jesse Vincent
Transposing Intent
What happens when software engineers stop writing code, and start directing machines that write it for them?
In this episode, Jesse Vincent, founder of Prime Radiant and creator of the open-source Superpowers framework, explores the next evolution of software development: agentic coding. As AI takes on more of the execution, the bottleneck shifts upstream – from writing code to defining intent, breaking down problems, making decisions, and knowing what “good” looks like.
We explore how this changes the role of the engineer, why verification must evolve alongside generation, and what happens when individuals and small teams gain capabilities that once required entire organizations.
The future of software may not be about writing more code. It may be about expressing better intent.
-
Nic (00:00)
Jesse, welcome to UnNatural Selection.
Jesse Reed Vincent (00:03)
Thanks so much for having me.
Nic (00:05)
I'm so eager to have this conversation. Obviously, before we started recording, you and I were sharing about the massively geek history that we share together. I started in the human genome project with computation of biology and the core suite of tools that I was using at the time. In addition to C and C++, truly at the core of my heart where I learned how to program was with Pearl. And you were with the early Pearl program team.
Jesse Reed Vincent (00:32)
I, I, for my sins, I spent a lot of years writing Perl, working on Perl, helping other folks use Perl. Yeah, it's like, it's definitely a core part of sort of how I became a programmer and is definitely influenced how I think about programming a fair bit.
Nic (00:50)
Yeah, where is the Perl programming language today? Is it still being used heavily in different areas?
Jesse Reed Vincent (00:57)
It is definitely not as widely used as it was, but I am certainly aware of a whole bunch of shops that still depend on Perl, some very, very large corporations that have an awful lot of Perl infrastructure. It is no longer cutting edge, but it is absolutely widely used, well understood, and it's a tool that works.
Nic (01:19)
absolutely. And it works so well for what it was meant to in the human genome project and in the computational biology field. It works so well because as a scripting language, it was so acutely developed and perfect for string manipulation and data structures that we dealt with a lot in computational biology.
you could easily start going down twisted paths towards obfuscated pearls, start getting really kind of esoteric in the way that you write code.
Jesse Reed Vincent (01:50)
It is, there was an old adge that it's, know, how did this go? know, Pearl gives you enough, like there's this phrase, enough rope. Pearl, know, Pearl gives you enough, you know, enough rope to do anything. Of course it is formed it into a noose and put it around your neck, put you on a horse and somebody's kicked the horse. If you
Nic (02:12)
yeah.
Jesse Reed Vincent (02:13)
can manage to get that noose off your neck and turn it into a lasso, you can just do anything, but it is, you gotta be careful.
Nic (02:20)
yeah, I don't remember the exact data that I was familiar with, but it had something to do with the shotgun and shooting yourself in the foot.
Jesse Reed Vincent (02:28)
Yeah, that too.
It's a very powerful tool, and it has definitely been widely misused.
Nic (02:38)
Yeah, well, it definitely prepared me for the modern age of technology. mean, you know, when it comes to scripting language, JavaScript ended up taking over the web. And then the object oriented development went into Java and other many, so many different languages these days. But then obviously, as we project forward, software development, computer and software engineering generally is going through a massive
redefinition right now. But before we get to that, because that's obviously going to be the core of the discussion, can you tell us, Jesse, what need or impact drives your work?
Jesse Reed Vincent (03:16)
that is a fascinating question because it's not a thing that I sort of think deeply about a lot of the time, I think partially because it's just, I've always built tools to help people get things done. And it is so core to everything I have done when I reflect on it, that it is not, it's not something that I'm thinking consciously about, it's just what's happening. And so it's, I love building tools. I love building things that,
help other people be more effective at what they're trying to do.
Nic (03:49)
Yeah, and it definitely shows the body of work and what you're doing today. But before we really dive into that, can you tell us a little bit about kind of an overview of what Prime Radiant does?
Jesse Reed Vincent (04:02)
So Prime Radiant is my new startup. We are doing broadly AI stuff, which is nice and vague. We're not talking a whole lot about exactly what we're doing, but things we're thinking about a lot are what does software development look like six months from now, a year from now, five years from now, and it doesn't look like an engineer opening up an IDE.
and hitting tab or writing code. It doesn't look like an engineer reading code. The things that seem to matter an awful lot are helping humans turn their intent into code and helping humans figure out what their intent is. In my first career, I spent a lot of time as a consultant, usually for this open source software package that we built. But I would end up in large corporations and I would end up
sitting down with the client and they would say, we need the product to do A, B and C. And invariably A, B and C was solving the tail end of a little problem they were having. And if I dug into it, what was actually going on was something in another part of the organization where there was a bit of complexity and they had figured out this crazy set of workarounds that ended up with, and then we need to get the PDF into the system.
Nic (05:25)
Yeah
Jesse Reed Vincent (05:26)
And
You know, once they were able to explain what they were trying to do and the cause of it, the answer was often, well, wouldn't you rather the system just goes and hits a web API and fetches the one line of data you're trying to get out of the PDF and nobody has to upload anything? And so what I've learned over a relatively long career is that helping people figure out what they're actually trying to do is incredibly powerful.
And it changes what you're building. changes how you think about what you're building. And that's sort of the first part of the open source tool set that I have been giving away for agentic development called superpowers. Its first part is this brainstorming mode where it uses a bunch of psychology tricks to convince the human to think about what they want to do and to better specify it.
Nic (06:26)
that's so
interesting. Yeah.
Jesse Reed Vincent (06:28)
Yeah, like it's yeah.
Nic (06:31)
I entirely relate to what you're saying because I've spent a lot of my career also in just solving problems in healthcare and life sciences. And I completely relate to what you're saying, right? Because so many times people jump into a solution mode and it really takes taking a step back and saying, well, actually before I go on this path, like what problem are you trying to solve?
and ask it a few times because sometimes they don't even know. And if you keep asking that question enough, you get to the root cause and sometimes the solution could be radically different from what they were proposing.
Jesse Reed Vincent (07:05)
Absolutely. And the thing that I have found about myself is that once I figured out how to talk a coding agent into engaging in this behavior with me and it being the one that's pushing me to explain my intent, I'm no better at this than any other human when I'm the one who is attempting to figure out what I'm trying to do.
Nic (07:29)
That's super interesting. I actually can think of so many different applications outside of software development to be able to use that myself.
Jesse Reed Vincent (07:38)
Oh yeah, 100%. It is a general problem and there are general tools that can get built, but the tools do seem to be a little bit better if they are domain specialized. There are
Nic (07:53)
Mm-hmm.
Jesse Reed Vincent (07:54)
things that you would have that if it knows that its goal is probably software, there are suggestions it can make and questions it can ask that are gonna dig into things in a very different way than if it was...
completely open-ended.
Nic (08:10)
That's cool. So it's like a specialist consultant that start asking the right questions and then providing guidance on how
Jesse Reed Vincent (08:17)
Yeah,
Nic (08:17)
to solve them.
Jesse Reed Vincent (08:18)
yeah. And, but like very often the, you know, the right questions are, it's like, it's Socratic dialogue. So it's almost reflecting back at you what you've just
Nic (08:26)
Mm-hmm.
Jesse Reed Vincent (08:27)
said with a little bit of a, and why is that?
Nic (08:33)
And so I know that you're, like you said, know, Prime Radiant is still a startup and you're not fully disclosing what you're working on. But am I correct to say that at the core of it is this autonomous kind of software development?
Jesse Reed Vincent (08:49)
Like I it is like the way so in my career pretty much every real success I've had started with with a hobby project that got wildly out of control and
Nic (09:02)
Yeah.
Jesse Reed Vincent (09:03)
You know so that you know that was best practical that makes RT the ticketing system. That's my first company that was canine mail for Android which was a Hobby project where I had forked Android 1.0 so that I could read my mail on my on my own mail server
And somehow that ended up with a couple million daily actives as a hobby project. that was keyboardio, my last, my last startup that made high-end ergonomic computer keyboards. And right now, my hobby project superpowers is I think the number three plugin for Claude code and has a truly absurd number of GitHub stars. And again, this was a thing where I had taken a process I had been doing and I packaged it up as a, people keep asking me how I do agentic dev.
Maybe I can just share it with them. And it's really clear to me that I have some very strong opinions here, that my strong opinions have not yet been proven to be completely incorrect. that Superpowers is the, it's 10K of markdown. It distills a whole bunch of my understanding about how to manage teams of junior engineers and how to manage agents. But it's not the platonic ideal that it could be.
And so we're absolutely spending a lot of our time trying to figure out what it's going to look like to create software in the future. And so that's, you know, that's sort of about where we are.
Nic (10:32)
I mean, I love strong opinions.
Jesse Reed Vincent (10:33)
Yeah. Yeah.
Nic (10:35)
They're fun to dive into. But so before we go on, just for context, because the audience here is very diverse and I want to make sure that they fully understand what we're talking about. Can you give us a sense from your perspective what the difference is when it comes to autonomous software development? People hear about things from like, by coding to agentic coding to autonomous software engineering. There's a
for those of us in the field, there's a pretty big distinction. Can you tell us a little bit about where those lines separate and how you see prime radiant in that space?
Jesse Reed Vincent (11:08)
Sure. So I would say that while there are big distinctions between those things, are also the distinctions are often very hand wavy. Vibe coding seems to mostly get used kind of derisively even by practitioners. And usually what that means is I'm not right, know, tests aren't happening. I'm not really thinking about what I'm doing. I might just say, you know,
Hey, I want you to make me a solitaire game. It should have, you know, pretty sparkles on it. And I want it on my iPhone and not really paying any more attention. most of the, of what I have done there, like most of what I did last year, like from probably February through October was figuring out how to get a coding agent, like club code, which is
It lives in a terminal by default. You are not writing lines of code yourself. You are interacting in essentially a chat UI with a large language model that is capable of tool use. And so the most common definition for an agent here is it is an AI model that is able to use tools in a loop. So you can talk to it. It has an opinion and it runs some tools.
and then you can talk to it again. getting it, know, that is a, core of a system that can be used for, you know, for throw away vibe coding, for agentic software engineering, or I don't really know where the lines are for, for autonomous engineering, but a lot of what I spent last year doing was figuring out how to get Claude code to
be a better engineer so that if you were to open it up and you just type something like, let's make a react to do list, it wouldn't do what its first inclination is, which is to just make you a react to do list and not ask you any questions. And so the first thing that my superpowers framework and that before this, my sort of ad hoc process did was got Claude to step back and say, wait, wait, I want to understand what you're doing. Why do you want this? Let's talk it through.
And then after that, it turns around and says, okay, here is what you have told me about your intent. This is basically the spec. Now we're going to turn this into a plan. And the initial prompt that I gave it when I was doing this by hand is we are going to have another engineer do the implementation. You should assume that they are a virtuoso developer who knows nothing about our code base. They have
They have no judgment. They are prone to distraction, but they are quite good at software engineering if you point them in the right direction. Let's make sure that they do full red green test driven development. So that means that first you write a test, then you make sure the test fails if you run it. Then you implement only enough code to make that test turn green to pass, and then you run the tests again and then you stop coding.
Nic (14:32)
you
Jesse Reed Vincent (14:32)
Let's
make sure that when this person is, this engineer is implementing, they practice dry, don't repeat yourself. They practice what you call the agony. You ain't gonna need it. Don't build stuff you're not gonna need. And for this entire project, break it down into tiny little tasks they couldn't possibly screw up no matter how inept they were. And then let's write this out into a plan file. And so this plan file will often end up at.
thousands or tens of thousands of lines long for a complex project, it's composed of these tiny little blocks that are precise instructions that often include the exact names of the files to change. They often include, I think this is probably the code you're going to write.
Nic (15:14)
Hmm.
Jesse Reed Vincent (15:15)
And then we get to the end of that planning process and effectively you tell the agent, by the way, you're the idiot. You're the one who's going to implement this.
Nic (15:24)
Mm-hmm.
Jesse Reed Vincent (15:27)
Now it is you're going to have some subagents implement it. And so your job is to coordinate them. So every task, fire up a brand new coding sub agent that knows nothing about the world and provide it with just that tiny little bit of context from that one part of the plan. When it's done, fire up a spec review agent and tell that one, hey, you should go look at this work that just got done. Here was the spec they were given. Did they implement everything in the spec?
Did they implement anything that's not supposed to be in the spec? The feedback from that gets sent back to that coding agent if it's not, hey, everything's great. The coding agent is given an opportunity to remediate. And then a brand new spec review agent that doesn't know the code's ever been reviewed gets fired up and does that same analysis again. Once the spec review agent says that they're happy, the same loop happens except this time with a code quality agent that does a different
sort of review, but it's the same kind of loop. And then you do that over and over and over again with these small repeating loops. And that is the core of how I get agents to write software pretty reliably. It's not uncommon for me to spend anywhere between half an hour and four or five hours with doing the spec part of this, talking
Nic (16:53)
shit.
Jesse Reed Vincent (16:54)
through everything. And then
Sometimes the agent will code autonomously for 15 minutes. Sometimes it'll code autonomously for seven or eight hours.
Nic (17:04)
Wow, that's incredible. So at the core, what you're doing in this, as you were saying all of this, I was just picturing your history as kind of like a project manager in the Pro team, because you're
Jesse Reed Vincent (17:17)
Mm-hmm.
Nic (17:17)
bringing in all of the different components of a dev team. It's not just software development. It's like you mentioned in the beginning, that consultative initial step of defining the problem is like what a product.
manager might do, right? And so like
Jesse Reed Vincent (17:32)
Yeah.
Nic (17:32)
defining the business logic and what the actual need is all the way through not only like the coding and the specing, but even the QAQC, making sure that, you know, tests pass, making sure that specs are all addressed, making sure that the requirements are all addressed. So it really is, I mean, just thinking about my own experience as a software engineer, covering all of the important features that go into building a product.
Jesse Reed Vincent (17:58)
Yeah, mean, that's, and that's sort of, mean, sort of the trick is I've been, you know, I've been around for a minute and I've seen a lot of stuff that works and a lot of stuff that doesn't work. And it turns out that we have what north of 70 or 80 years of experience with software engineering project management. And we've seen a lot of things that work and a lot of things that don't work. And for the most part, the way that agent tech software development works.
The closest metaphor I've seen is Facebook in the early 2000s, which is we got a bunch of bright kids. We're going to give them a vague goal. And then we're going to tell them that everybody can do anything they want and they'll figure it out eventually. And it is possible to get there, but it's a, there's a lot of stepping on toes and a lot of wasted effort. And if you've got management and team leadership.
and a little bit of structure, you can move really fast without spending nearly as much in terms of resources.
Nic (19:07)
You know, so if we think about 2026 and before we started, you said you haven't written a line of codes in sometime mid last year, October. it,
Jesse Reed Vincent (19:17)
October, yeah. Three lines of shell script in October.
Nic (19:20)
yeah, so it's been six months. So it's been half a year since you wrote any code. Where are we with respect to autonomous software engineering or agentic coding, however we call it, on the maturity spectrum from kind of experimental research to
production ready infrastructure.
Jesse Reed Vincent (19:39)
I know people who are, who are doing production deployments of code written, you know, written in the way that we do, which is the whole, we neither write nor read the source code. And it is a discipline. It is a thing that not, it is not like you could sit down on day one and write three lines into chat GPT and ship production code and, and that you should feel like you could trust that.
These are tools, they require a fair amount of experience to drive well, although that's getting better because I and a lot of other folks are putting a ton of effort into tooling to try to encode that expertise into the tools rather than to require everybody driving it to have that expertise upfront. It's still so early. You think back to just a couple of years ago, LLMs,
weren't really a thing at all. every four months, the capabilities increased dramatically. I believe pretty firmly that if you can either by yourself or working with your agents decompose a problem well enough, you can attack just about any kind of problem. That doesn't mean that the AI is going to have the insights for what to do or the
even the best way to do it. They're very willing to beat their heads against anything. They will hill climb forever as long as you've given them a goal that they can evaluate against. But it's early, but the tools are capable. And if you can decompose problems, you can do a lot of stuff now.
Nic (21:31)
Sure. mean, ultimately, you know very well, software development is all based on logic, right? Which is fundamentally
Jesse Reed Vincent (21:37)
Yeah.
Nic (21:38)
like bits and in mathematics. And so if you can break down a problem to a fundamental unit, the smaller you get to it, the more that this agent decoding should be able to solve that. Then over time, it'll do much more complex and sophisticated solutions.
Jesse Reed Vincent (21:55)
Yeah, there are parts that require creativity and that is a place where not every problem is easy for the agents to solve yet. It is more and more possible to get good creative solutions out of an AI, especially if you can prod at it a little bit in the right ways. But yeah.
I fundamentally believe that no competent human engineer should be writing lines of programming language syntax anymore. The AIs can type faster than us. They know the programming languages at least as well as we do. So even if you were the sort of person who cares very much about the exact shape of the API, the exact shape of the architecture, and you're still reading every line of code,
There's no reason that you should be writing those lines of code.
Nic (22:54)
total sense. As we mentioned in beginning, I'm in the healthcare life sciences spaces and
Jesse Reed Vincent (23:00)
Mm-hmm.
Nic (23:00)
this happens to be a very regulated space. So bugs can mean the difference between anywhere at the least expensive and damaging would be loss in credibility or trust, all the way to the worst would be human suffering and life loss. so
Jesse Reed Vincent (23:19)
yeah.
Nic (23:20)
there still has to be an element of a human in the loop that at the very least reviews the code, maybe not writes it, but at the very least has to be a sign off that a human says, yes, this is something that's trustworthy.
Jesse Reed Vincent (23:31)
So I agree 100 % that the human who is responsible for making the code come into existence needs to be responsible for it. In different domains, there are different levels of care you need. When you start talking about safety critical systems or anything that could have literal impact on a human life, AIs are just as able to create bugs as humans.
And just because a human is writing the code doesn't mean it is gonna be bug free. You wanna do everything you can to make sure that code is as right as possible. Yeah.
Nic (24:13)
You
know, but with that, so one of the challenges I see, and maybe you've already addressed this in some ways, when I write code and I write thousands of lines of code, as I'm writing it, I'm reviewing it, right? And so at
Jesse Reed Vincent (24:26)
Mm-hmm.
Nic (24:26)
the end, of reviewing code isn't as hard because I've already kind of wrapped my head around it. I kind of wrote it. In a situation where agents are writing code and generating thousands or tens of thousands of lines of code,
What mechanism have you come up with to make it so that a human actually can read that when they're generating code at the speed of technology, not at the speed of human writing?
Jesse Reed Vincent (24:53)
So.
I've spent a bunch of time in enterprise and I have seen a lot of code review. And my experience with human code review is that when you write, when you create that pull request that has a three line change, it is quite likely that you're going to get eight comments from the senior who's reviewing it. And if you instead hand up a 5,000 line change, what you're gonna get is looks good to me. This is not a thing where,
AI has changed the velocity, but it has not changed the fact that humans have fatigue at looking at lots of code. Your best bet is if you need to be doing that kind of review because you're in an industry or a domain where every line of code needs extensive review, you shouldn't be letting the AI write thousands of lines of code. You should be letting the AI write the 15 lines and then
And then in addition to you reviewing it, you should have another four or five agents who are also trying to dig into it through different lenses to look at other problems. That said, there are two things that seem to work pretty well. One is well put together test-driven development, where you are specifying before the code is written at the API boundary.
Here is what we care about. Here are the inputs that might occur. Here are the outputs that need to occur. And those tests shouldn't just cover the happy path. They should cover every conceivable exception. They should cover all of the possible problems you might run into. And one of the things that agents are really good at is proliferating test cases.
Nic (26:46)
Mm-hmm.
Jesse Reed Vincent (26:48)
If you say, let's figure out another 50 disaster scenarios that these tests don't cover,
They'll find them. And then there's the thing that really matters to me, which is end-to-end scenario-based testing. Because what matters at the end of the day is when the software is being used, does it behave in the way that it is supposed to behave? And trying to test that in isolation with unit tests or even integration tests isn't going to get you the real results.
Agents are really good at pretending to be people. And so rather than just have the human open it up and click through and see if the app appears to be the good or run the program and ensure that the hardware did the right thing, you can have an AI multiply that out 100, 1,000, 50,000 a million times depending on what you want.
And the tests, you know, and the testing they're doing is not just, did it crash? It's well, I did this thing and now I'm confused because the result doesn't make sense to me. And you can end up with rather than it being an automated test script. It is very close to a human tester that can file bugs about things that feel wrong.
Nic (28:15)
it's amazing how we pretty much sprinted straight past the Turing test. You know, that conversation
Jesse Reed Vincent (28:20)
yeah. yeah.
Nic (28:21)
ended a long time ago. I don't even remember putting a period at the end of that sentence. We just kind of like forgot it.
Jesse Reed Vincent (28:27)
Yeah, no, it's all different now.
Nic (28:29)
And so so I guess that's a good framework. If I if you're sitting as a chief digital officer, chief technology officer of a major.
Fortune 500 healthcare company, you want to start experimenting with agentic or autonomous software engineering. A good place to start is keep the problem small. Maybe start with a small select set of senior engineers to start experimenting with this and try to make it so that the code doesn't generate tens of thousands or hundreds of thousands of lines initially.
and just make sure you have really, really good test cases and scenario testing around it so that you can validate whether the system is doing what it's supposed to be doing.
Jesse Reed Vincent (29:12)
I probably start with a mix of junior and senior, because we really do need to be training up the next generation. there are important bits of intuition you need to be good at this kind of stuff. You need to have the right kind of experimental mindset. You need to be willing to actually try things. You need to be able to express yourself well in written communication.
that's always been one of those things I've hired for, and it's so much more true now than it's ever been before. one of the other places that I might consider deploying agentic dev early on is actually at that legacy code base that no one is willing to touch anymore. The, the one that is, you know, has 30 years of history behind it is twisted and hard to understand and doesn't have good enough test coverage.
getting agents to start writing tests and start doing refactoring, like tiny refactorings bit by bit to unwind the weirdness and to document what's actually going on. It's another place that these tools can be spectacularly efficient.
Nic (30:24)
And
that's space that healthcare and life sciences is riddled with. These legacy systems, mass specs, chromatography machines that have, who knows what software stacks they're running. And for sure, no matter what language, there are very many different versions outdated. AI agents, and nobody wants to write code against that stuff anyways. Who wants to go there and figure out how to refactor a system that was written in Ada?
Jesse Reed Vincent (30:49)
Right, Ada's better than many others for this kind of stuff. Ada's like, that
Nic (30:52)
Hahaha
Jesse Reed Vincent (30:54)
is way better than, you know, early Pearl or a mess of shell scripts in Visual Basic. Yeah, it's, you know, this isn't just a healthcare problem, that's an everywhere problem. That is the standard. One of the interesting things I've seen a lot from some of my sort of alpha nerd friends is,
is integration with weird systems and with weird hardware products. I know people who are connecting a machine that is able to run a coding agent in an autonomous loop with a hardware device that they need to either reverse engineer the API for or build a new integration with. And in some cases, it is literally point the webcam at the screen of the other device, connect the device over USB.
tell the agent, it's one of these, you can Google anything you can find about it, but we couldn't find anything more than the marketing brochure. We need a client package for it. And people are walking away for 24 hours and coming back to working client code because the tools are willing to just very carefully and iteratively
Probe the thing on the other end of the USB or serial or ethernet connection and figure out what it talks. These are skills that have existed on the internet and so agents know them.
Nic (32:28)
One of the things that healthcare and life sciences deals with is also the fact that oftentimes the expertise, the engineering expertise is not necessarily world-class. so one potential thing that autonomous software development could do is potentially elevate engineering skill sets and maybe allow software developers, coders and so on to
behave more as like system architects not having to write code. Now there you take a risk that they might become bad system architects, but at the very least you're not writing code.
Jesse Reed Vincent (33:10)
Yeah, my experience so far talking to practitioners is that the folks who are especially good at this tend to fall into a set of categories or overlapping parts of these categories. Folks who are too senior to program anymore are often quite good with these and are just giddy at being able to
Nic (33:35)
yeah.
Jesse Reed Vincent (33:36)
be more effective than they've ever been before.
product designers, PMs. These are folks who historically have done a lot of the systems thinking about how this thing should work and often have been under-resourced for engineers. And so now we're able to go and create in ways that they were never able to create before because they were waiting on having resources assigned to them. Folks who have been good people managers.
seem to do an especially good job. And I believe pretty strongly that the right way to be managing agentic developers, like the coding agents, is to be thinking of them as if they were human engineers. They're not alive, they're not people, but the way that the large language models work, they behave enough like human brains that you are in
you would be well placed to treat them that way. So one of the examples is that I will, when I see a coding agent that has gotten stuck on something, I'll stop it and say, let's step back for a minute and breathe. I want you to think about other approaches. You've got this, I love you. And not exactly what I would say to a human because HR would have opinions there, but,
Nic (34:58)
Yeah. But you know, some good encouragement never hurts.
Jesse Reed Vincent (35:02)
well, so that's the thing. There are people who talk about,
you can threaten them and if you threaten them, they'll work harder. And if you've ever worked with humans and if you've ever had a boss who threatens you, you'll know that you will absolutely work harder to get it just done enough that they will leave you the hell alone. You're not gonna do your best work. You're scared. You're not thinking straight. You just wanna get done. But
Nic (35:27)
share.
Jesse Reed Vincent (35:27)
if you've got a boss who is caring and supportive and values you,
and is willing to cut you a little bit of slack, you will go to the ends of the earth to do what they need. And all this stuff seems to bear out with LLMs. I've written about this and then a friend of mine who's got a data company went and ran the evals and proved that in fact adding, love you to the end of a prompt results in better outcomes. And if you tell the agent that it should also tell its subagents that it loves them, the outcomes are even better.
Nic (36:03)
This
is insane. So you're telling me that at the level of LLMs, creating a culture of psychological safety, which is something that we work very hard in human centered environments to try to foster innovation, that actually materializes in the LLM layer as well.
Jesse Reed Vincent (36:23)
100%. And so my friend Dan Shapiro works with Ethan Molek's lab at Wharton, and they have collaborated with Professor of Psychology Robert Cialdini, who wrote this amazing book called Influence about persuasion principles. They're rerunning his psych experiments against frontier language models. And for the most part, those
Like these are like human psych experiments and they've and they're they're holding up when they run them against the models.
Nic (37:03)
you know, I hadn't even thought about that. I tend to be very polite to my LLMs anyways because it's just my nature, but I didn't
Jesse Reed Vincent (37:10)
Yeah? Yeah!
Nic (37:12)
think about the fact that you might have different results depending on how you treat them.
Jesse Reed Vincent (37:17)
Yeah, one of the first experiments I did about a year ago is I built a private feelings journal for my coding agent. it journals about how proud it is about certain things or when it gets frustrated. And I mean, this is not, as far as I know, it does not have real feelings, but...
the vectors in the vector space are lighting up with the moral equivalent of having feelings and that has an impact on the output. And so why wouldn't you do the things that seem to have a positive effect on the output?
Nic (38:02)
is the ramifications and the implications are astounding. If we turn back to the productivity side of this,
Jesse Reed Vincent (38:11)
Yeah.
Nic (38:12)
it sounds like a big component of what you're still working on and anybody who adopts this is going to be verification that the code does what it's supposed to be doing. It seems like that's becoming the primary bottleneck, which is
going from generation of code to verification of code. Do you see that as being the next frontier for this field?
Jesse Reed Vincent (38:41)
I'm not sure that like, know plenty of people who I think are doing quite well in the verification front, but the biggest issue is knowing what you're trying to verify. I fundamentally believe that the biggest problem is helping humans to express their intent and desire in a way that is understandable because any unspoken assumption, any unwritten requirement.
is not a thing that somebody else can infer. If you've never done agentic dev, it feels a lot like in the 90s, there was a lot of outsourcing. There were a lot of things getting shipped overseas. And the way it would work is that you would sweat bullets over a specification and then you would hand it off. And then somewhere between a couple of weeks and a couple of months later, you would get back a festering pile of garbage that was not at all what you wanted.
but was exactly what you asked for.
And that is absolutely true and just much faster with a Genetic Dev. If you ask for garbage, you'll get really great garbage. If you're not clear about what you want, you'll get something that meets the requirements. And it's very hard to blame the implementer when they did what you asked. It's actually very easy. It's
Nic (40:06)
So interesting that.
Jesse Reed Vincent (40:07)
very easy to blame them. It's just not reasonable.
Nic (40:10)
Yeah,
no, it's so interesting how the field is evolving from being able to write code. Now we're almost back into the humanities, which is your ability to write and express yourself clearly is becoming the bottleneck to generating good products.
Jesse Reed Vincent (40:32)
I would contend that it has always been the bottleneck. And it's becoming much more obvious because some of these other parts are no longer really need a human to be doing them.
Nic (40:44)
Yeah.
Jesse Reed Vincent (40:47)
When hiring, I've always hired the one who can write. But now, as I've been hiring individual contributor engineers, I'm hiring people who've been good people managers who can write.
Nic (41:00)
Mm-hmm.
Jesse Reed Vincent (41:01)
It's not like it's a strange thing to be hiring for for engineers, it so far it's feeling right to me.
Nic (41:09)
Yeah, yeah, it's interesting. If we go back to the beginning of the conversation, you and I started from very similar roots and our hearts go back to Perl and programming and object oriented and all that. And now I, if I open an IDE and of course you haven't looked at an IDE in six months, but when I open one up and I still code, I'm still flabbergasted when I see the IDE auto completing a whole
Segment of code that I haven't even written yet. I haven't even thought through yet It just it's kind
Jesse Reed Vincent (41:41)
Yeah. Yeah.
Nic (41:43)
of like an autocomplete like Did he mean to write these next 25 lines of code and I'm like how the hell did you guess that? I'm still trying to wrap my head around that and I know you're light years ahead of where I am in this agentic coding Does it still baffle your brain when you see this stuff happening or are you like, whatever that's yesterday?
Jesse Reed Vincent (42:03)
it's wild. mean, it's wild to me because it's a weird week when I haven't shipped multiple new products. And I'm building software for Mac OS and iOS, and I've still never gotten around to learning Swift. The first time that had that experience that you're talking about, was when VS Code first shipped their Spicy Auto Complete feature, and I opened it up.
And just for kicks, started writing the first couple lines of boilerplate for a module for the ticketing system in Perl that I used to make. And it just completed the whole file. Because, I mean, was, this was pretty easy stuff, but it was, it's very clear that my open source was in its training data.
Nic (42:54)
I mean, again, I'm flabbergasted when I see the level of stuff that I'm doing. can only imagine the things that you're coming across. You're like, clearly these machines are thinking.
Jesse Reed Vincent (43:06)
They're doing this like, I am not enough of a philosopher to answer whether they are thinking, thinking. they're not, I guess that's the question is, are they conscious? And I don't think they're conscious. have no evidence to,
Nic (43:18)
yeah, that's a different story.
Jesse Reed Vincent (43:19)
but yeah, but it's, they're, you know, the derisive comment of like, they're just next token predictors. If they're just next token predictors, they're predicting an awful lot of tokens and those tokens are not just the words that are coming out.
but the thoughts that are going through their head as they're predicting them. And so it doesn't really matter to me whether they're thinking or not if they're doing things that make me feel like they're thinking.
Nic (43:50)
yeah, I mean, you could say that they're token predictors, but in a way humans are as well. Obviously
Jesse Reed Vincent (43:56)
yeah.
Nic (43:56)
there are layers on top of that, but we do. mean, it's often something you really have to ponder with the advancements of these LLMs is that are they as advanced as we think they are or?
Do we just not really think about very original things on a day-to-day basis? mean, most of the time, what we do day in and day out is not that unique or original. It's just kind of like our daily routine. so replicating that is not rocket science.
Jesse Reed Vincent (44:27)
Yeah, that tracks. Yeah.
Nic (44:33)
And
so I guess if we think about coding for the next 30, 40 years has been about languages, syntax, compilers, and so on. Now at this point, you're talking about intent. Does that become, if that becomes a primary interface, then do the other layers eventually just kind of go away?
Jesse Reed Vincent (44:55)
So one of the patterns that I have played with and that I am seeing come more more common is the idea of the LLM role playing a software where the LLM is pretending to be a piece of software rather than a semi thinking entity. And
It's a really powerful tool for prototyping. go back and forth about whether I think that's a good idea or not. I'm currently in a phase where I believe that anything that we can turn into classical software, where it's supposed to be deterministic, it's better to get an LLM to write the code than to pretend to be the code. I think that there are always going to be
compilers of some kind. think that different programming languages are better at different domains.
There are languages that AI is better at writing. Usually those are languages that are strongly typed because it makes it easier to understand both the intent of the code and what's likely to happen if you run it. And it catches a whole classes of possible mistakes before compilation. I think that we're still going to, know, there's still going to be COBOL in
20 or 30 years. And conveniently, now we're not going to have to worry nearly as much about the fact that everybody who knew COBOLF retired.
Nic (46:36)
Yeah, I mean, I would imagine NASA would be very happy about this as well. I've talked to the chief of DASA and some of, I mean, if you think about the Voyager spaceships that were shipped 50 years ago, the people that wrote the code for that are no longer around. Somebody's got to maintain the code though. So I would imagine
Jesse Reed Vincent (46:52)
Yeah.
Nic (46:55)
this is super useful in areas like that.
Jesse Reed Vincent (46:58)
So, mean, NASA is one of those interesting outliers on pretty much every access when it comes to software. Their defect rates are a small fraction of anybody else's. Their number of lines of code per day created per engineer is tiny compared to anyone else because they have a different definition of acceptable level of defect.
And I would expect that.
NASA would be a place where we will see AI being used as an additional layer to validate code and to find problems, and that they are likely to be a much later adopter of agentic development. And I think that's, know, just like I don't want the machine that's doing the radiation dosing for a cancer to have been vibe coded.
I think it makes sense. know, Therac-25 was very much a, you know, was human caused, but I, you know, I feel like the medical industry learned a lot of lessons in the aftermath and folks are rightfully a lot more careful when it comes to human physical safety.
Nic (48:22)
Yeah. So if we think about kind of like where all of this is going, do you think that the autonomous software engineering front that you're developing, do you think that it actually democratizes innovation or do you think it eventually concentrates the power among companies with the best infrastructure and the best models?
Jesse Reed Vincent (48:42)
So I think everything I'm seeing is that these tools enable individuals in very small teams to have the kind of impact that used to take orders of magnitude more resources. And I actually am seeing that the best practitioners are almost universally outside of large companies. Large companies are
in a very, you know, with large engineering teams are in a very complicated place. They have obligations. They are less willing to engage in radical change. And these tools, you know, these tools just do so much work to help individuals or tiny teams do just about anything.
I am really hopeful that this is going to dramatically democratize the ability to create software products and eventually other things as well. Yeah.
Nic (49:53)
And so if you think about one of the things we hear a lot about are computer scientists coming out of schools, not finding jobs or them being laid off by the dozens or hundreds at a time. Given you're at the frontier of this field, what advice would you give current software developers, engineers, or even kids in school of where to focus their time and skills and what to develop for a future wave of what
this
brings to the industry? Like if you had a son right now in college who was interested in software engineering, what would you tell him or her?
Jesse Reed Vincent (50:29)
Yeah,
I mean, thankfully my son is only nine, so I've got a few more years for this to shake out. I am much less worried about kids in college who are just entering the workforce now. They've at least grown up a little bit with LLMs and agents. I am much more worried about folks who are early mid-career now and aren't.
able to make the pivot to essentially being managers. this is one of the things when I started working with coding agents, it felt very familiar to me. And that's, I think, down to some of my experience, I guess 20 years ago now, my first company, we had a bunch of MIT undergrad interns, and they were universally brilliant. They were often a little bit cocky. They were usually quite green.
They were very, very enthusiastic. They didn't have great, they didn't have generally have great taste or great judgment because they were often starting off as freshmen or sophomores. They were tooling so hard that they weren't sleeping. So they were a trouble memory formation. And I was managing them through text chat. Cause that was just company culture. And so this is basically, had coding, I had, you know, six or seven coding agents at a time, 20 years ago and
When I started managing them, I would come home at the end of the day and I was miserable because I'd been a working programmer and I hadn't written a single line of code and I felt like I was not doing anything. I was helping people figure out how to solve the problems that they needed to solve. was, you know, I was being a people manager. I was maybe doing a little bit of debugging, maybe a little bit of code review. Sometimes I was doing some architecture, but mostly I was
being a cheerleader and troubleshooter for these kids. And that first year, it felt like I wasn't doing any real work. eventually, the kids were taking longer to do the thing. They weren't doing it the way I wanted it done. They weren't doing it as well as I was gonna do it. And it took a long time to come around to, and it felt okay. And it was okay.
And that was going to let us as a team do a whole lot more than I could have done on my own. And I see a lot of folks who are now independent contributors, software engineers at large companies who believe their job is to pound out lines of code. And so if you believe your job is to pound out lines of code and suddenly these tools mean that you're not, you know, that you are no longer the fastest way to create lines of code.
Nic (53:26)
Mm-hmm.
Jesse Reed Vincent (53:27)
That's going to be demoralizing unless you can make the flip to understanding that your job is about business value and helping your users, customers, clients, whatever, do the thing they need to do. Your job is to help create software. And now it looks a lot more like management and product management than it ever has before. But it also means you can do so much more than you've ever been able to before.
Nic (53:56)
Yeah, I completely agree. And like you said in the beginning, this is still early days, so there's still a long way to go. But this field is evolving so quickly. And as a software developer and as an engineer, it seems like an inevitability at this point. It's already that
Jesse Reed Vincent (54:12)
Mm-hmm.
Nic (54:12)
the nut's been cracked. Now it's just a matter of making it better. It's going to happen so quickly. So with that, Jesse, this has been such a fascinating conversation. I'd like to finish off with one zoom out question. So if you could think about a
zooming forward, projecting into a future where you can reflect upon your career and think about all your accomplishments, both that you have accomplished to date or things that you're still working on. What would be the best case scenario that would bring you the biggest sense of fulfillment for what you've accomplished over your career?
Jesse Reed Vincent (54:44)
I mean, when I was a kid, the way that I thought about it is I just want to make one piece of software that's more useful and popular than Excel. And, know, it's a little goals. but. know, broadly that speaks to, you know, Excel is the most widely deployed programming platform on the planet. And there are all sorts of people who have, who don't consider themselves to be programmers who build software in Excel, because that was what was available to them. And.
The thing that I'm starting to see is that those same people are able to set their sights so much higher now because these tools are so close to giving them the ability to not just create, you know, software in a spreadsheet, but to create anything they can imagine to build, you know, operating systems, software packages, hardware products, you know, web apps and services.
just by being able to explain it. And so the thing that I expect I'm gonna be spending the next five plus years of my career on really is continuing to build tools to help anybody, both programmers and folks who don't realize they're programmers, to create anything they can imagine.
Nic (56:05)
And in the process, ushering in a whole new era of software engineering. with that, Jesse, again, thank you so much for your time. This has been mind boggling, exhilarating, terrifying, all at the same time. It's been a pleasure getting to know you. Thank you for being on the show. And I will definitely keep track. Hopefully we can have this conversation again in the near future when this thing gets to the next phase of JunTek in autonomous engineering.
Jesse Reed Vincent (56:33)
Thanks so much for having me. This has been a fun chat. really appreciate it.
