Vibe Coding in Production: What AI-Generated Code Actually Looks Like After 6 Months

Six Months In, Nobody’s Laughing Anymore

A startup we worked with last year shipped their entire MVP using nothing but Claude 3.5 and Cursor. Forty-seven screens, a REST API, Stripe integration, the works. They were done in six weeks flat. The founders were ecstatic. Their CTO sent me a Slack message that said something like “we may never hire a senior dev again.”

I kept that message. I look at it sometimes when clients ask me whether they should build software with AI from day one. Because fourteen months later, that same team spent three months doing almost nothing except untangling what the models had written. One engineer described the codebase as “a coral reef, beautiful on the surface, structurally insane underneath.”

This isn’t a hit piece on AI tools. I use them every day. But there’s a real gap between the demo and the eighteen-month maintenance reality, and the software engineering community is only now starting to quantify it. So let me share what we’ve actually seen.

What “Vibe Code” Even Means in Practice

The term was coined (or at least popularized) by Andrej Karpathy in early 2025, describing a mode of development where you describe intent in natural language and mostly accept what the model produces without scrutinizing every line. It’s not just “AI helps me write code faster.” It’s closer to “I stop fully reading the code and trust the output.” That distinction matters enormously for what happens six months later.

In practice, we see vibe code produced in a few recognizable patterns. Developers prompt GPT-4o or Claude for entire modules at once. They accept suggestions from Copilot without running the logic through their head. They ask Cursor to “add error handling” and move on. None of these are inherently reckless, but collectively they create a codebase where no single human holds a coherent mental model of what’s running.

That’s the real risk. Not any individual bug. The loss of the mental model.

The Data: What Our Audits Actually Found

Over the past year, our team ran static analysis audits on eleven production codebases that were self-reported as “primarily AI-generated” (meaning the development team estimated over 60% of lines were written by an AI tool with minimal manual editing). We also audited eight codebases from the same period that were traditionally engineered with AI used only for autocomplete or isolated function generation. The comparison is imperfect, I’ll say that upfront. Different team sizes, different domains. But the patterns are consistent enough to be worth sharing.

We used SonarQube for bug density and code smell counts, Radon for cyclomatic complexity on Python services, jscpd for code duplication rates, and manual test coverage reports where CI pipelines existed. Here’s what the aggregate numbers looked like:

The test coverage gap is probably the most damaging long-term. AI models will write tests if you ask them to. The problem is that most developers in “vibe mode” don’t ask, or they ask once and the tests generated are shallow assertions that pass trivially without actually validating behavior. This is documented in academic literature too: a 2024 study published on arXiv examining GitHub Copilot outputs found that AI-generated test suites had significantly lower mutation scores compared to human-written tests, meaning the tests existed but weren’t catching real logic errors. (Schafer et al., 2024)

The cyclomatic complexity number is worth pausing on. A score of 14.2 per function on average means these functions are genuinely hard to reason about. The threshold where code becomes “complex” by most standards (including Carnegie Mellon’s SEI guidelines) is around 10. We were well above that in nearly every vibe-coded project.

Where Technical Debt Actually Accumulates

The debt doesn’t show up evenly. It clusters in specific places, and once you’ve seen it a few times you start recognizing the signature.

State Management and Side Effects

AI models are genuinely good at generating isolated functions. They’re noticeably weaker at reasoning about shared mutable state across a system that grows incrementally over dozens of prompting sessions. In React applications especially, we see state management become an absolute nightmare. Context providers nested six levels deep. useEffect hooks with dependency arrays that are either empty (wrong) or contain everything (also wrong). The model produced each piece correctly in isolation. Nobody designed how they’d interact.

Error Handling Theatre

This one frustrates me personally. When you ask an AI to “add error handling,” it will add error handling. It catches exceptions. It logs them. It returns null or an empty array. What it almost never does is think about recovery strategy, user feedback loops, or whether the error should propagate up. So you end up with codebases where errors are technically handled but functionally swallowed. The app doesn’t crash. It just silently fails, and you find out three weeks later when a client reports missing data.

Dependency Sprawl

Models suggest packages. That’s what they do. In one Node.js project we audited, the package.json had 211 direct dependencies. The application was a fairly simple SaaS dashboard. Some packages were duplicates of each other with different names. Several were abandoned projects with no maintenance since 2021. Each one was suggested by a model at some point and accepted without review.

The Cost Reality: Maintenance Eats the Savings

The speed gains from AI-augmented development are real. A 2023 paper from Microsoft Research found that GitHub Copilot users completed tasks 55.8% faster in controlled experiments. (Peng et al., 2023) Nobody’s disputing that.

But the research that gets less attention is what happens to maintenance costs. A 2022 study in the Journal of Systems and Software found that code maintainability issues account for roughly 42% of total software development lifecycle costs. (Besker et al., 2022) When you inflate complexity and reduce test coverage, you’re essentially borrowing from that budget at a high interest rate.

The fintech client I mentioned at the top calculated that their three-month refactor cost approximately $180,000 in engineering time. Their original AI-accelerated build had saved them an estimated $60,000 compared to a traditional timeline. The math isn’t complicated.

I want to be fair here: this isn’t a universal outcome. We’ve seen teams that build software using AI tools and maintain genuinely clean codebases. The difference is almost always discipline around a few specific practices, not the tools themselves.

What Actually Works: AI Software Engineering With Some Structure

The teams doing this well aren’t avoiding AI. They’re treating AI generation as a first draft that requires a real review pass. Here’s what separates them:

• They write .cursorrules or equivalent project configuration files that constrain the model’s output style, preferred libraries, and testing requirements before a single prompt goes out.
• They run SonarQube or CodeClimate on every pull request and treat new code smells as blocking, not advisory.
• They review AI-generated code with the same rigor as any external contribution, which means actually reading it, not just running it.
• They prompt for tests explicitly and then manually review whether those tests would catch a plausible regression, not just whether they pass.
• They enforce a complexity budget. If a function comes back from the model with cyclomatic complexity over 8, they ask the model to refactor it before merging.

None of this is magic. It’s just applying normal engineering standards to a new input source. The teams that get into trouble are usually the ones who treated “AI wrote it” as equivalent to “it’s fine.”

There’s also a model selection dimension worth mentioning. In my experience, Claude 3.5 and GPT-4o produce noticeably different code patterns for the same prompt. Claude tends toward more verbose but more defensively written code. GPT-4o is terser and often more elegant but more likely to skip edge case handling. Neither is universally better. Knowing which to use for what task is part of real ai software engineering practice now.

What This Means for Teams Choosing to Build Software with AI

If you’re a CTO or engineering lead deciding how much to lean on AI generation for your next product, here’s my honest framing: the six-week savings are real, and they’re worth capturing. The twelve-month maintenance liability is also real, and most teams dramatically underestimate it at the start.

The question isn’t “should we use AI to build software.” That ship has sailed. The question is whether you’re building the code review culture and static analysis infrastructure to handle the specific failure modes that AI-generated code introduces. Because those failure modes are different from what traditional engineering produces. They’re not worse in every dimension. But they’re worse in the dimensions that hurt you at month nine or month fourteen, when the original momentum is gone and someone has to understand what was built.

One practical benchmark: if your team can’t answer “what does this module do and why is it structured this way” for every major component, you’ve already accumulated the kind of debt that tends to compound. AI-augmented development makes that question easier to ignore. It shouldn’t.

FAQ

Is vibe coding ever appropriate for production systems?

Yes, conditionally. For early-stage MVPs where speed matters more than longevity and the team has a clear plan to revisit the codebase before scaling, AI-heavy generation can be a reasonable trade-off. The mistake is treating MVP code as foundation code. If you know you’re building a throwaway prototype, vibe code away. If you’re building infrastructure you’ll run for five years, you need architectural review regardless of how it was generated.

Which AI tools produce the cleanest production code?

Based on our audits, the tool matters less than the workflow around it. That said, Cursor with a well-configured .cursorrules file tends to produce more consistent style than raw API prompting, simply because the constraints reduce model improvisation. For security-sensitive code specifically, we’ve had better results using GitHub Copilot Enterprise with organizational policy controls enabled compared to unconstrained models. The models themselves keep improving rapidly, so any specific ranking I give will be outdated in six months.

How do you measure technical debt in AI-generated codebases specifically?

Start with the standard tools: SonarQube’s technical debt ratio, Radon for Python complexity, and jscpd for duplication detection. Beyond that, the metric that correlates most strongly with future pain in our experience is test mutation score, not raw coverage percentage. A codebase with 70% coverage but a mutation score below 40% is in worse shape than one with 45% coverage and a mutation score of 75%. Tools like Pitest (Java) or mutmut (Python) will give you this. Most teams don’t run them. They should.