When Engineers Stop Trusting an AI Agent

Right after the Amazon Q Developer CLI launched to Amazon’s engineers, a pattern showed up in the feedback: answers were inconsistent and sometimes generic. Adoption was at risk, and not because developers didn’t want the tool. The tool wasn’t earning their confidence.

The challenge

Developers are an unforgiving audience. If a tool gives you a bad answer twice, you stop asking it. I needed to find out why the answers were unreliable and get a fix prioritized before trust problems turned into churn.

What I did

• Gathered the signal from everywhere. Slack, the download page, documentation comments and direct interviews with developers.
• Tested it with the people affected. I recruited an engineer advisory board to try search and retrieval and help pin down where it broke.
• Found the root cause. The agent was pulling from the entire internal documentation ecosystem. With that much surface area, the answer depended on whatever it happened to retrieve.
• Made a specific recommendation. Limit retrieval to a curated, more predictable set of documentation.
• Framed it for engineering as a trust problem, not a bug. Developers won’t adopt a tool they can’t rely on, and that put the install base we’d built at risk.

What changed

The product team put the scoping fix on the roadmap. Narrowing what the agent could draw on made its answers more predictable, and it addressed the feedback that had been piling up, supporting continued adoption instead of drop-off.

What I took away

Trust erosion looks like a marketing problem from a distance, but it’s a product problem. Marketing just hears about it first, because we’re closest to the feedback.