So Here’s What Vibe Coding Can't Teach You: Why Fundamentals Still Matter

Written By Rob Kleiman

Feb 26

By Rob Kleiman DevRel, community builder, and host of Megashift

There's a new way of building software: prompt an AI to write your code, get a working prototype, and call it done. Andrej Karpathy coined it "vibe coding" in early 2025: "you fully give in to the vibes, embrace exponentials, and forget that the code even exists." The tweet got 4.5 million views. Collins Dictionary made it their Word of the Year.

And honestly, some of what people produce that way is impressive. But as Simon Willison pointed out, there's an important distinction: vibe coding specifically means generating code without reviewing it. That's different from AI-assisted development where you understand what's being written. The gap between those two approaches is where architecture decisions, tool familiarity, and debugging instincts still matter. Skip them, and you're shipping risk.

I wanted to put that to the test. What does it actually look like to ship a fullstack product end-to-end using modern tools and AI-assisted development, not JUST vibe coding, but actually understanding the code? So I built one. This post walks through the tools I chose, the patterns I relied on, where AI coding helped, where it created problems, and what the full development lifecycle actually looked like building BizSpotNY.


A Bit About Me

I've spent a few years in developer relations and community building, working across AWS, Google, Adobe, and Major League Hacking, supporting developer workflows and helping people build things. I've gotten more technical as my career has progressed, from earning a master's in technology management from Columbia to working hands-on with AI tools daily.

I also run Megashift: an interview series where I sit down with business leaders, technologists, and builders to unpack the biggest shifts happening in business right now, from AI tooling and developer experience to go-to-market strategy and emerging tech. As Caroline Lewko of DevRel Agency said on the show, the field is shifting fast so "who's not an AI developer now?" and the DevRel professionals who thrive are the ones with both technical aptitude and business instincts. That rang true for me: the more I understood what I was building, the more useful I became to the developers and communities I supported.

A recurring theme across Megashift conversations has been the tension between AI-powered speed and the depth of understanding you actually need to ship. I needed to see for myself how far this abstraction away from the basics takes hold.

So set out to build a fullstack application from hypothesis to production over a few months, with Claude Code as my AI pair programmer throughout the process.

The Hypothesis

I keep wondering the same thing as I walk past restaurants in NYC: will AI search spell doom for local businesses that don't have marketing budgets?

When someone asks ChatGPT for "best espresso bars in the East Village," the answer comes from content crawled from the web: Google results, Yelp pages, company websites. Most small businesses don't have the time or resources to invest in SEO (search engine optimization, the practice of making your website show up higher in search results). They've got a name, maybe an address, maybe a paragraph on a page no AI system can parse into a useful answer. Will local businesses become invisible to AI? I hope not.

This question is becoming more relevant, not less. ChatGPT started running ads in February 2026, and the line between organic AI answers and paid placements is starting to blur. As Garrett Sussman from iPullRank put it on Megashift, "Companies don't know if they're showing up in ChatGPT or AI Overviews, nor how to monitor visibility across conversational queries, the biggest money left on the table."

The idea I wanted to test: what if you could give a local business a structured listing that AI systems can actually crawl, show businesses which AI platforms visited their page, and let them optimize their online presence through a simple interface? This idea became BizSpotNY, live at bizspotny.com, with real listings and real AI crawler data coming in.

Then 230 commits over around three months. What started as a weekend proof of concept became a fullstack production application that kept growing. Here's the journey through building it, roughly in the order things actually happened.

Choosing the Stack

Every tool choice is a bet. Here's what I picked and why:

LayerChoiceThe BetFrameworkAstroSome pages are pre-built for speed (like category pages), others are generated on demand when a user visits them (like individual business listings). Astro.js lets one framework handle both.UIPreact + TailwindSmall interactive components without shipping a heavy JavaScript bundle to the browser.Database + AuthSupabaseA managed Postgres database with built-in user authentication, security rules, and vector search. One platform instead of four separate services.PaymentsStripeSubscription billing with webhooks (automated notifications sent from Stripe to my app whenever a payment event happens) to keep the app in syncSearchpgvector + Hugging FaceAI-powered semantic search running inside the database itself, no separate search service neededDeploymentVercelCloud hosting with edge delivery (serving content from servers close to the user) and built-in support for AstroAI Pair ProgrammerClaude CodeAI pair programmer throughout the build process

These choices determine where the complexity lives. Consolidating authentication, database, and search into Supabase meant fewer moving parts, but it also meant when Supabase had a problem, everything had a problem.

Astro's hybrid approach meant I could optimize pages for both AI crawlers and human users, but it introduced routing bugs that no tutorial warned me about.

Here's the thing: an AI assistant can generate boilerplate for any of these services. It can't tell you whether consolidating five services into one platform is the right trade-off. That decision requires understanding how your app deploys, how much downtime you can tolerate, and how you plan to debug when something breaks.

Building: Where Things Broke

Two weeks into this fascinating "weekend project" I was rolling right along. The stack was chosen, the data model was in Supabase, listings were rendering. Then everything started breaking. Not all at once, but in sequence, each fix revealing the next problem. Three different parts of the application broke in three different ways, and each one required understanding something that no AI tool could figure out for me.

Astro Routing

The first real problem. The app had two types of pages that both used dynamic URLs (URLs that change based on what content you're viewing): [section].astro for categories like /restaurants and [slug].astro for individual business listings like /joes-coffee. Both worked locally. In production, every listing page returned a 404 error. The framework couldn't tell if a URL like /services was a category or a business.

The fix: pre-build the category pages at deploy time so they're resolved first, and keep business listings generated on demand. Astro resolves pre-built pages before falling through to on-demand rendering. Two dynamic routes colliding? Ok: Make one static. I had to dig deep into the Astro documentation to find this behavior explained.

Client-Side State

With routing fixed, navigation broke in a different way. Astro has a feature called transition:persist that keeps interactive components alive when users navigate between pages, instead of destroying and recreating them, the component stays mounted. The problem: the data passed into the component updates silently, but the component doesn't re-render to reflect it. In practice, my FilterBar.tsx category navigation pills wouldn't highlight the active category after clicking through pages.

Eight commits to find a five-line fix. Whoops. Sound typical? You can follow the journey in the commit history: event listeners, forceUpdate hacks, removing state entirely. The actual solution was straightforward: tell the component to watch for changes in the incoming data and update itself:

const [activeSection, setActiveSection] = useState(currentSectionId)
useEffect(() => { setActiveSection(currentSectionId) }, [currentSectionId])

This is a common React pattern: syncing local component state with incoming props. Knowing how component state management works is what got me there. not prompting.

Payments

Stripe's happy path is well-documented. The edge cases aren't. Here's one that bites people: when a payment notification (a webhook) fails to deliver, Stripe automatically retries it. If your app doesn't check whether it already processed that payment event, it processes it twice. A customer gets charged once but your system records it twice, or worse, triggers duplicate actions. Most tutorials skip this entirely.

The fix takes 20 minutes: a database table that stores every payment event ID, a check before processing (webhook.ts looks up whether the event was already handled, and skips it if so), and an insert after. This pattern is called idempotency: making sure that processing the same event multiple times has the same result as processing it once. Twenty minutes of work that prevents an entire class of production bugs you'll never catch in development.

What These Bugs Have in Common

Three bugs. Three different systems. But the same underlying lesson: each one required a fundamental skill that AI couldn't provide right out of the box. The routing collision needed framework knowledge: how Astro resolves pages. The state bug needed React fluency, how components re-render. The payment bug needed distributed systems instincts: how webhooks retry.

In my chat with Ash Ryan Arnwine, a former DevRel lead at Adobe and Nylas who's now building Collxn as a solo founder with AI coding agents, put it this way on Megashift: AI coding agents are "absolutely a double-edged sword" they put more emphasis on the developer to be "an effective product manager and an effective software architect." The ability to push back on why something is built a certain way doesn't go away with AI. It becomes the whole job. As Ash said, "a nice prototype built in a chat prompt is a huge unlock. But if we start shipping all those things to production and getting real users on them, putting real data in; there could be a reckoning at some point."

Every bug in this section was a small reckoning. And every fix came from fundamentals, not prompting.

Deploying: When "It Works on My Machine" Isn't Enough

By now the app worked locally. Auth, payments, search, bot tracking, all running. Time to push it live and move on with my life.

That is not what happened.

This is the part of shipping software nobody puts in their portfolio. An actual stretch of my commit history:

fix: Handle missing Supabase env vars gracefully during build
fix: Use Proxy to make supabase export build-time safe
fix: Convert dynamic imports to static imports for Vercel build
fix: Remove .ts extension from bot-detector import (Vercel build fix)
fix: Add .ts extension to bot-detector import for Vercel build

All "works locally, breaks in production." All different root causes. Here's what was happening: when the app gets packaged for deployment, the framework pre-builds the static pages. During that build process, some code tries to connect to services like the database. But the connection credentials (stored as environment variables: configuration values that live on the server, not in the code) only exist at runtime when the app is actually running, not during the build step. Every static page that touched database code failed, and the error messages never pointed to the actual cause.

Fix: a Proxy wrapper: a stand-in object that intercepts requests and only creates a real database connection when the app actually needs data, not when the code is first loaded. More on this pattern below.

The rule: anything that runs at build time needs to handle missing runtime config gracefully. Run npm run build locally before every push. It catches 80% of these before they become multi-commit detours.

This is a clear example of where tool familiarity matters. Understanding how the build pipeline packages your code, how the hosting platform resolves file imports, and how the bundler analyzes your dependencies: that's not something you can prompt your way through. The AI suggested toggling an import extension back and forth. I had to read the Vercel docs and understand the actual module resolution before the fix stuck.

Patterns That Survived

Some architectural decisions held up under all of this. These are patterns I'd bring to any project:

Security enforced at the database, not the application. Supabase has a feature called Row Level Security (RLS) that controls who can read and write data at the database layer. Instead of relying on your application code to check permissions (which means every API endpoint needs to get it right), the database itself enforces the rules. Every query automatically scopes to the logged-in user. Even if there's a bug in an API endpoint, another user's data can't leak because the database blocks it. Write your security policies before your API routes.

Lazy initialization for build safety. The Proxy wrapper that fixed the deployment builds is a pattern worth explaining:

export const supabase = new Proxy({} as SupabaseClient, {
  get(target, prop) {
    const client = createSupabaseClient()
    return client[prop]
  }
})

Instead of connecting to the database when the code is first loaded (which fails during builds because credentials aren't available yet), this creates a lightweight stand-in object. The stand-in does nothing until the app actually tries to use it, at that point, it creates the real connection on the fly. Zero changes needed to the rest of the codebase. This pattern works for any service client, not just Supabase.

Feature gating with a kill switch. When I introduced paid tiers, existing free users needed to keep their current features. The solution was a two-level safety net: a global toggle to turn restrictions on or off, plus per-user overrides for grandfathered accounts. Launch day: bulk-grandfather existing users, flip the switch. Zero breaking changes.

Working With AI: Where It Helps, Where It Creates Risk

167 out of 230 commits have Co-Authored-By: Claude in them. But this wasn't just chatting with an AI and pasting the output.

The setup mattered. MCP servers (Model Context Protocol, a way to give AI tools direct access to external services) connected Claude Code to my payment and database environments so it could read real data, not guess. Custom skill files maintained structured context across coding sessions…essentially cheat sheets that told the AI about the project's architecture and conventions. A CLAUDE.md project memory file enforced coding standards every time a new session started. Supabase CLI handled database migrations. Stripe CLI handled payment testing.

MCP is still early, and the governance questions around it are real. In a Megashift conversation with Andrew and Stephen, the founders of Keyboard, we talked about exactly this: MCP servers can unlock serious productivity by letting AI agents interact directly with your tools, but as Stephen put it, "when you hire a junior employee, you don't give them access to all your data." The same principle applies to AI agents: you need governance around what they can touch. For BizSpotNY, that meant giving Claude Code read access to Stripe and Supabase data, but keeping write operations under my control.

The tooling around the AI matters as much as the AI itself. Here's what I actually learned about that workflow across the full development cycle:

It's fast for well-defined tasks. Database migrations, test scaffolding, CSS cleanup. One commit cut a component from 1,008 to 468 lines. When I could define exactly what "done" looked like, and the MCP integration gave Claude Code the context it needed, execution ran at 3-4x my solo speed.

It's unreliable for ambiguous debugging. The navigation state bug? Claude Code kept generating increasingly complex solutions. I kept saying "simpler." It kept getting more complex. The deployment issues? Some of those fix commits were AI suggestions that addressed symptoms, not root causes. A systematic review out of Carnegie Mellon analyzing 518 practitioner accounts of vibe coding found the same pattern: hallucinations and compounding technical debt are the most common failure modes.

Architecture decisions can't be delegated. The choice to pre-build some pages and render others on demand. The decision to enforce security at the database layer. The tier gating kill switch. Those required understanding the tools, the trade-offs, and the deployment model. An AI can draft an implementation once you've decided the approach. It can't decide the approach. The tool gets faster, but the judgment calls stay with you and if you can't make those calls, speed just means you ship the wrong thing sooner.

The real risk of vibe coding at this level: if you don't understand why the Proxy wrapper fixes the build, it might be hard to debug it when the hosting platform changes its behavior. If you don't understand database-level security rules, you it does make it tricky to audit whether your security model actually holds. If you don't understand how payment webhooks retry, you can't reason about double-processing. A benchmark study from Purdue University found that only 10.5% of vibe-coded solutions were secure despite 61% being functionally correct. The AI accelerates what you already understand. It obscures what you don't, and that's where production bugs hide.

What I Actually Got Out of This

Three months and 230 commits after a weekend proof of concept, the platform is live with real listings and real crawler data. Users can see their GPTBot, ClaudeBot, PerplexityBot, and others are hitting their listing pages, and the bot tracking middleware logs every visit.

Wheter the business case is for this exists is still an open question. But building it was worth every commit. What ended up in the repo: 35 database migrations. 17+ API endpoints. 6 serverless edge functions. A 9-step submission wizard. A 5-email onboarding sequence. An AI readiness scoring engine. A bot tracker watching 10+ crawlers. None of this was in the weekend plan.

That's the kind of breadth you don't get from tutorials or side projects that stop at "it works on localhost."

What I'd do differently next time:

  • Lock the design system on day one. Three color palette migrations across 38 files and 143 class replacements. All avoidable.

  • End-to-end tests from the first deploy. The route collision would have surfaced in automated testing, not production.

  • Recognize scope creep earlier. A weekend POC doesn't need an AI readiness scoring engine and a 5-email drip campaign. Build the measurement first, optimize later.

If you want hands-on experience with modern tools, build something real. Not a todo app. Something with users, payments, edge cases, and deployment problems. The mess is where the learning is.

The code can be viewed here: github.com/rkrevolution/nyc-directory-poc

For the product thesis and why AI discoverability matters for small businesses, read the companion post: Answer Engine Optimization Is the Next Frontier.

These questions: what it takes to ship, how AI changes the builder's role, where fundamentals still win, are what I explore every episode on Megashift. If this post resonated, the conversations that shaped it go deeper. And if you're shipping on this stack or thinking about the line between AI-generated code and production-ready software, open an issue or find me on LinkedIn.

Further reading:

Rob Kleiman
Next

Woah. Was I Actually Vibe Coding Way Back in 2014? Reflections on DIY tech projects