"Sorry that I suggested creating massive security gaps so that I could recover 0.1% more database performance. That was probably a bad choice."
That was Left Field Labs Chief Engineer Ian Lollar, imagining what an AI coding assistant might say after an engineer catches one of its more questionable recommendations.
The line came out of a recent conversation Ian and LFL CTO Eric Lee had on For Break Fast, a podcast about AI and the people actually building with it. Their conversation covered a lot of ground, but kept coming back to a problem that's becoming harder to ignore as AI gets better at writing software: getting code is easy. Knowing whether it's good is something else entirely.
Eric had a demo that made the point nicely.
The code worked, which was kind of the problem.
Eric was building an internal video platform and wanted to add a view counter. So he pointed an AI coding tool at the codebase and gave it a pretty reasonable request: add a view counter and only count a view after someone has watched for more than 30 seconds.
The AI went off for several minutes, reviewed the codebase, and came back with a working implementation. It did exactly what Eric asked.
It also introduced two problems that an experienced developer would have spotted pretty quickly.
The application used Firestore, and the implementation had multiple users incrementing the same counter. At any meaningful scale, that creates contention and the potential for database lockups. The endpoint also allowed any authenticated user to send the message that incremented the view count. The UI waited 30 seconds, but nothing stopped someone from bypassing it and generating views directly.
For an internal video platform, maybe nobody cares enough to launch a sophisticated view-count fraud operation. But that wasn't really the point of the exercise. The model had produced a feature that satisfied the request while missing things Eric and Ian considered table stakes, like security and the technical constraints of the system it was modifying.
So Eric tried again.
This time he told the model that views needed to be logged and auditable in case of fraud. Recording them couldn't create a performance hit or database lockups. The system needed to work at scale. And if the existing architecture wasn't enough, the model had permission to introduce additional infrastructure.
The result was considerably better.
Instead of repeatedly updating a single record, the new implementation sharded the writes across 20 records and tallied them in the background. It also created a test video and ran through different viewing scenarios to make sure the feature behaved as expected.
What changed wasn't the model. Eric knew which problems to tell it to solve.
It's tempting to describe this as a lesson in prompting, i.e. "write a more detailed prompt and you'll get better code." But a longer prompt wasn't what ultimately made this project work. It was knowing to ask about concurrency, fraud, auditability, performance and scale in the first place.
And even then, the model found another way to surprise them.
Ian asked what would happen if the video itself was shorter than 30 seconds. It turned out the first, naive implementation had accounted for that on its own, counting a view after 90% of a short video had played. The supposedly better implementation hadn't. A video under 30 seconds would never register a view.
There was always another assumption hiding somewhere.
Working code isn't the same as good code
Ian had run into the same problem in less controlled circumstances.
He had recently used AI to help investigate a large, unfamiliar codebase. It was a good use for the technology, because the model could chew through a lot of code, explain unfamiliar patterns, and help him narrow down where performance problems might be coming from. The resulting work produced performance improvements ranging from roughly 1.5x to 5x in the areas he was investigating.
But the AI didn't guide him neatly from problem to solution. Ian repeatedly had to pull it back when its recommendations didn't make sense.
At one point, it suggested an approach that would improve performance at the expense of security.
Ian rejected it. The AI promptly agreed that, yes, sacrificing security was indeed a bad idea.
That experience is part of why Ian compares AI-assisted development to driving with autopilot. It can take on a lot of the work, but you still need to know when to grab the wheel.
None of this is completely new to software engineering.
Developers have been moving up through layers of abstraction for decades. Higher-level programming languages, frameworks, libraries and managed infrastructure all let people build without having to understand every mechanism underneath what they're using.
AI takes that much further. You can now describe a feature in natural language and get back a plausible implementation without necessarily understanding how that implementation works.
That's useful, obviously, and it's also where things get interesting.
Ian's point on For Breakfast was that the strongest engineers have never been the ones who know only the abstraction they're currently working in. They're the people who can go down another layer when something isn't behaving the way it should. They understand enough of what sits underneath the convenient interface to recognize when the abstraction is hiding something important.
AI doesn't change that. But it does make it possible to produce much more software before anyone notices that the person producing it doesn't have that understanding.
As Eric put it during the conversation: "Anybody can write software. Not everybody can write good software."
There's an uncomfortable corollary. If AI lets people generate eight times as much code, that isn't automatically an eightfold improvement in engineering output. Every additional line is another place for a bug, vulnerability, or maintenance problem to live. Ian noted that the prospect of dramatically increasing code volume makes many experienced engineers nervous for exactly that reason.
The ability to make more software and the ability to make better software aren't the same thing.
Knowing what to hand over
Eric and Ian don't use AI in quite the same way.
Eric tends to experiment broadly with new models and tools. Ian has built his own tooling partly so he can decide, task by task, how much control he wants to give up. Sometimes he wants an agent to explore an unfamiliar codebase. Sometimes he wants AI to fill in a boring implementation detail while he stays directly in the code. Sometimes he wants to do the work himself.
Neither approach requires deciding whether AI is trustworthy in the abstract. That's not a particularly useful question.
The more useful question is what you're trusting it with.
That applies when using AI to write software, but it also applies when AI becomes part of the software itself. Ian made that distinction explicitly during the conversation. Using AI to help build a product and building AI into the product are different engineering problems.
If you're building a conversational experience, variation is part of what makes an LLM useful. You don't necessarily want the exact same sentence every time.
If you're expecting a system to produce a guaranteed structure, enforce a security rule, or perform an irreversible action, variability looks very different.
That doesn't mean an LLM can't be part of the system. It means the rest of the system has to be designed around what the model can and can't reliably do. There may need to be deterministic checks, evaluations, restricted permissions, or a person reviewing the result before anything consequential happens.
Software engineers already know how to design around components that can fail. Ian compares the problem to distributed systems, in that with enough scale and complexity, individual components fail, responses don't arrive, or something behaves in a way the next part of the system wasn't expecting. Good systems account for that rather than assuming every component will behave perfectly forever.
AI adds a different kind of uncertainty, but the engineering instinct to understand the failure modes and build accordingly should be familiar.
The part that doesn't get automated
The first implementation of Eric's view counter wasn't obviously broken. It ran. It counted views. It passed the superficial test implied by the prompt.
Eric knew it wasn't done because he understood what would happen when multiple users hit the database at once. He understood the security implications of the endpoint. He knew there were operational questions the implementation hadn't answered.
Then Ian looked at the improved version and immediately found another hole.
That's a much better picture of AI-assisted engineering than either extreme of the current debate. AI isn't just autocomplete with better marketing, and it isn't an autonomous engineer that makes engineering expertise obsolete. It can investigate unfamiliar systems, write substantial amounts of code, test its own work, and help experienced people move much faster.
But speed changes what the engineer has to pay attention to.
When producing an implementation takes less time, more of the work shifts toward deciding what should be built, defining the constraints, anticipating how it can fail, evaluating what comes back, and understanding the system well enough to know when an apparently reasonable answer is actually a terrible idea.
The tools will keep getting better at the implementation part.
That makes knowing what to ask for (and knowing when the answer isn't good enough) more valuable, not less.