Rob Kleiman

• Written by Rob Kleiman

Installed Isn't Adopted: Rolling Out an AI Coding Assistant to Amazon's Engineers

Amazon wanted its engineers productive on generative AI tools, and fast. The push had visibility all the way to the top of the company, and at its center was Amazon Q Developer, a command-line-first AI coding assistant. I owned the program management of the rollout: the developer success strategy, the adoption standards and the usage metrics.

The challenge

Shipping a tool to tens of thousands of engineers is easy. Getting them to change how they work is not. Developers rarely complain about a tool they’ve given up on. They just stop opening it. We needed a way to get engineers to a real win in their first session, and a way to notice when they quietly drifted away.

What I did

  • Hands-on first. I built workshops around a sample to-do app, so engineers left their first session with something working instead of a slide deck.
  • A steady drumbeat. Announcements, a contributor guide and an editorial review process kept useful content coming. A newsletter covered milestones, new features and tips, and an internal “billboard” showed off the CLI’s slash commands, designed around the jobs engineers were actually trying to do.
  • A playbook other teams could run. I partnered with Prime Video to co-develop their own adoption journey, so the approach could spread beyond my team.
  • Instrumentation for the quiet part. With our insights team I built a churn dashboard: usage by product, clear thresholds for what counts as an active user, and drop-off trends shared with leaders.

How I measured it

I think about platform health in four layers:

  1. Awareness. Do the right people know it exists?
  2. Activation. Did they do something meaningful, like run a first prompt or slash command?
  3. Retention. Are they back next week? Drop-off in the first two weeks was the signal we watched most closely.
  4. Expansion. Are they going deeper, and spreading it team to team?

Installs live at the top of that funnel. The number that matters is weekly active use.

What changed

More than 30,000 engineers installed Q Developer, with strong continued usage. Over the same period, Amazon reported an 18.3% year-over-year increase in weekly production deployments per builder as friction came out of developers’ days.

What I took away

The difference between installed and adopted is whether a tool sits in someone’s critical path. Design for the first two weeks, and instrument for the silence.