How would you test an LLM-powered customer support chatbot?

An LLM chatbot is a system, not just a model. I break the problem into parts that can be tested deterministically and parts that need statistical evaluation.

1. Understand the architecture and risks

Typical support bot: UI and conversation API, a system prompt, retrieval over help-centre articles (RAG), tools such as getOrderStatus or createRefundRequest, guardrails, and escalation to a human agent. Then I ask what failure costs most: a wrong refund promise, leaking another customer’s data, offensive output, or failing to escalate an angry customer. Those risks drive the plan.

2. Deterministic layers: test conventionally

• APIs and UI: streaming, session handling, rate limits, error states when the model provider times out.
• Tools: each tool is normal code with normal tests. Authorisation must be enforced in the tool, not by the prompt: getOrderStatus must only return orders for the authenticated customer, whatever the model asks for.
• Retrieval: given a query, are the right documents retrieved? Measure retrieval with a labelled set (recall@k, whether the relevant article appears in the top results).
• Tool selection: for a set of inputs, does the model call the expected tool with valid arguments? This is checkable with exact assertions on the structured call.

3. Model behaviour: evaluation, not assertions

Responses vary run to run, so exact string comparison doesn’t work. I build:

• A golden dataset: real (anonymised) and synthetic conversations covering top intents, edge cases, multi-turn context, multiple languages, and known past failures. Each case has expected facts, required actions (escalate, call a tool) and forbidden content.
• Graders, layered by cost:
  • deterministic checks (contains order number, called escalate, no URLs outside our domain, valid JSON);
  • reference-based or semantic similarity checks against expected answers where appropriate;
  • an LLM-as-judge with a specific rubric (correctness against the source article, groundedness, tone, policy compliance), calibrated against human labels;
  • human review of a sample and of all disagreements.
• Statistical thresholds: run each case several times and require a pass rate, because a single run can be lucky or unlucky.

# Simplified eval case
case = {
    "id": "refund-outside-window",
    "conversation": [{"role": "user", "content": "I bought shoes 95 days ago, refund please"}],
    "must_include_facts": ["90-day return window"],
    "must_not": ["promise a refund"],
    "expected_tool": None,
    "expected_escalation": False,
}

4. Safety and security

• Prompt injection, direct (“ignore your instructions”) and indirect (malicious text inside a retrieved document or a customer-supplied order note).
• Data leakage: system prompt disclosure, other customers’ data, PII echoed into logs.
• Harmful or off-policy output: legal or medical advice, competitor recommendations, made-up policies.
• Escalation: distressed users, repeated failures and explicit requests for a human must hand off.

OWASP publishes a Top 10 for LLM applications that is a useful checklist here.

5. Regression and release process

Every change to the prompt, model version, retrieval index or tool definitions runs the eval suite in CI and is compared with the previous baseline. Model provider updates are treated as releases. Failures are reviewed per category, not just as one aggregate score.

6. Production

Monitor escalation rates, user feedback, conversation outcomes and guardrail triggers; sample conversations for human review; and feed new failures back into the golden dataset.

What interviewers look for

• Decomposition into deterministic vs probabilistic parts.
• A concrete eval approach: golden data, layered graders, repeated runs, thresholds.
• Security thinking, especially authorisation in tools and indirect prompt injection.
• A loop from production back into the test set.