docs

Agents

Retrieval & grounded answers

How the agent finds and uses your knowledge, and how to tune it.

What retrieval does

For every customer message, the agent searches your knowledge base for the most relevant chunks and answers using them, rather than answering purely from what the underlying model already "knows." This is what makes answers specific to your business and keeps them consistent with your documentation.

Settings (in an agent's Settings tab)

  • Grounded answers only — when enabled, the agent refuses to answer ("I don't have that information") rather than guess, if it can't find supporting evidence in your knowledge or tool results. Turn this on for agents where a wrong answer is worse than no answer (policy, legal, pricing). Leave it off if you'd rather the agent use general knowledge as a fallback for questions outside your knowledge base.
  • Chunks used (top-K) — how many knowledge chunks are handed to the model per answer (1–20). Higher gives the model more context per question but can dilute focus; lower keeps answers tightly scoped but may miss relevant detail spread across chunks.
  • Candidates per leg — how many candidates the retrieval step considers before narrowing down to the top-K (1–100). Higher improves recall on large knowledge bases at a small latency cost.
  • Rerank threshold — a 0–1 relevance cutoff; chunks scoring below it are discarded even if they'd otherwise make the top-K. Raise it to be stricter about relevance, lower it if the agent is missing answers that exist in your knowledge base but aren't being surfaced.

Tuning workflow

If the agent gives a wrong or generic answer to a question you know is covered in your knowledge base:

  1. Open the ▶ Test drawer and ask the same question.
  2. Check the Activity trace panel — it shows exactly which chunks were retrieved and their scores. See Playground & execution trace.
  3. If the right chunk wasn't retrieved at all, the source may need better headings/structure, or the rerank threshold may be too strict.
  4. If the right chunk was retrieved but the answer still missed it, try raising chunks used (top-K) so the model has more room to use it.

Tuning retrieval settings is almost always more effective than switching models — see Choosing a model.

Writing a system promptEscalation to a human