Stakeholder Communication & Lifecycle Management

Stakeholders arrive with a solution, not a problem. "We want a chatbot on our documentation" is a proposed answer; the question underneath it might be that support is overwhelmed, or that the docs are unfindable, or that one specific team asks the same thing every week. Those have different best solutions and only one involves a chatbot.

What to establish before designing anything:

• The actual problem, separated from the requested feature.
• Who is affected and how often: volume decides whether automation pays for itself at all.
• What happens today, in detail. The current manual process is your specification, your eval set, and your baseline.
• What "better" means numerically. If nobody can say, you cannot demonstrate success later, and the project will be judged on vibes.
• Constraints already fixed: data residency, regulated data, systems you must integrate with, deadlines tied to something external.
• The cost of being wrong. This drives more architecture than any other single answer: it sets your human-in-the-loop placement, your guardrails, and your accuracy target.

Asking "what happens when it gets one wrong?" early tends to reset expectations more effectively than any later conversation about model limitations.

Explaining an architecture to people who will not read the diagram is a distinct skill from producing the diagram.

Lead with the decision and its consequence, not the reasoning that produced it. "We're using retrieval rather than putting the whole handbook in every request. It costs less per query and stays current when the handbook changes" lands. A walk through the alternatives you rejected does not.

Name what you gave up. Every architecture has a cost, and a presentation that describes only upside reads as either naive or evasive to anyone senior. Stating the trade-off is what makes the rest of the account credible.

Match the frame to the audience. Executives are deciding whether to fund it: cost, risk, timeline, what it enables. Engineers are deciding whether they can build it: interfaces, failure modes, dependencies. Legal and security are deciding whether it is allowed: data flows, retention, access control. The same design, three genuinely different documents.

For anything with a probabilistic component, say plainly and early that the system will sometimes be wrong, and describe what happens when it is. Discovering this after launch is how projects lose their sponsor.

You can commit to availability and latency the ordinary way. You cannot commit to correctness the same way, and pretending otherwise is the trap this objective is built around.

What can be committed to:

• Availability and latency percentiles: ordinary infrastructure commitments.
• A measured accuracy range on a defined evaluation set, with the set specified. This is a real, defensible number.
• Process guarantees: every output above a threshold gets human review; every failure is logged and investigated; the eval set is re-run on every change.

What cannot: a guaranteed correctness rate on arbitrary future input. Your measured accuracy holds on inputs resembling your eval set, and the honest form of the commitment says so.

When a stakeholder pushes for a hard accuracy number, the productive move is to offer the measured range plus the process guarantees, and to be explicit that the number is conditional on the input distribution. Under-committing here costs you a little at signing and saves the relationship later, which is the trade a professional-level architect is expected to make.

Documentation that another team can build and operate from covers more than the diagram:

• The architecture, with data flows: including which data crosses which boundary.
• Why the significant decisions were made. Six months on, nobody remembers why retrieval was chosen over long context, and someone will "simplify" it back.
• The prompt and its version history. Prompts are production configuration; treat them as versioned artifacts, not strings pasted into a file.
• The eval set and current baseline numbers.
• Known limitations and failure modes: written down, so the next team inherits them rather than rediscovering them in production.
• Runbooks: what to check when quality drops, how to roll back a prompt, who to escalate to.

Phases, briefly: discovery (the problem and the constraints), design (architecture and evaluation approach together: an evaluation plan added afterward is usually shaped to make the system look good), build, handoff, and then monitoring and iteration, which is the longest phase and the one that gets no plan.

The tail is where LLM systems differ most from ordinary software. A conventional service that isn't changed keeps behaving the same way. An LLM system's inputs drift, its dependencies update, and its retrieval corpus goes stale, so a deployment with nobody watching it does not stay level, it degrades quietly. Handoff must include who owns the eval set, who reviews quality metrics, and on what cadence, or you have handed over something that will fail without anyone noticing.