Retrieval (RAG)
How agentty's search_docs and search_code tools find the right docs and code, how to turn on semantic search in two commands, and how to check it's working.
on this page
agentty gives the model two local search tools. search_code finds code by what it does ("where are retries handled") when you don't know the name. search_docs searches your docs folder, your installed skills, and facts you asked it to remember. Both work with no setup using keyword search, and everything runs on your machine.
For exact names and strings, the model uses grep instead.
Turn on search by meaning
Keyword search needs no setup. To also search by meaning, run Ollama and pull an embedding model:
ollama pull qwen3-embedding:0.6b # best for code, ~640 MB
# or: ollama pull nomic-embed-text # smaller, ~270 MBThat's all. agentty finds Ollama on 127.0.0.1:11434 by itself. It prefers qwen3-embedding:0.6b and falls back to nomic-embed-text if that's the one you pulled.
The first search in a repo builds the index. This takes from about 30 seconds to a few minutes on a large repo. After that, the index loads in about a second and only changed files are updated.
Check that it's on:
agentty diagnosticsThe retrieval (embeddings) section should say probe: ok. Each search result header also says hybrid (keyword + meaning) or bm25 (keyword only).
To use a different model or a remote server, type /settings and open Proactive retrieval (RAG) (the Retrieval pane), or set:
export AGENTTY_EMBED_MODEL=nomic-embed-text
export AGENTTY_OLLAMA_HOST=gpu-box.lan:11434If results look wrong
| Symptom | Cause | Fix |
|---|---|---|
Results say bm25, never hybrid |
No embedding server, or the model isn't pulled | ollama pull qwen3-embedding:0.6b, then agentty diagnostics |
diagnostics says invalid: host ... |
A typo in the Retrieval pane (e.g. 127.0.0) |
Fix the host there, or delete the rag keys in ~/.agentty/settings.json |
search_docs finds nothing |
No docs folder | Put docs in ./docs or ./.agentty/knowledge, or set AGENTTY_DOCS_DIR |
| First search is slow | The index is being built | Only the first time per repo and model |
| Wrong file comes first | The query matches a sibling file better | Ask with words from the file's purpose; grep for exact names |
When a search returns no hits, the result says why. no matches (bm25) means it ran, nothing was relevant, and meaning search was off. no matches (no docs indexed — set AGENTTY_DOCS_DIR …) means there was nothing to search.
How well it works
agentty measures its own search on a judged set of real questions about its own code and docs (tests/rag_eval, run with agentty rag-eval <file>). "Right file first" means the file that answers the question is the top result:
| Search | Setup | Right file first | Right file in top 8 |
|---|---|---|---|
search_code |
keyword only | 57-60% | 92-95% |
search_code |
+ nomic-embed-text |
60-63% | 88-90% |
search_code |
+ qwen3-embedding:0.6b |
63-78% | 93-97% |
search_docs |
+ embeddings | 70-85% | 93-97% |
Each range covers the main and held-out halves of the set (60 + 41 code questions, 30 + 20 docs questions).
What helps code search:
- Identifiers are split into words, so
onClipboardReadmatches "clipboard read" (AGENTTY_RAG_SPLIT_IDENT=0turns this off). - Source files rank above tests and docs unless you ask about tests or docs.
- A header pulls its
.cppup with it. - Results spread across files instead of returning one file eight times.
AGENTTY_RAG_CODE_RERANK=0 turns off these code-specific ranking rules.
How a search runs
Each search passes through these stages in order. Cheap stages are on. Slow or corpus-sensitive stages are off until you turn them on. Defaults are set in src/rag/adapter.cpp:715-761.
| Stage | What it does | Default | Env var |
|---|---|---|---|
| Chunking | Splits code at functions and classes, docs at headings | On | - |
| Contextual chunks | Adds context about where a chunk sits to the chunk | On | AGENTTY_RAG_CONTEXTUAL=0 |
| Keyword search (BM25) | Matches words | On | - |
| Meaning search | Matches embeddings, if the startup check found a model | On when a model answers | AGENTTY_EMBED_MODEL |
| Fusion | Merges the two lists. Convex by default; rrf is weighted reciprocal-rank fusion |
convex |
AGENTTY_RAG_FUSION=rrf |
| Adaptive fusion | Picks the keyword/meaning balance per query | On | AGENTTY_RAG_ADAPTIVE=0 |
| Reranker | Deterministic rerank, plus code rules for search_code |
On | AGENTTY_RAG_CODE_RERANK=0 |
| Diversity (MMR) | Spreads results across files | On, lambda 0.65 | AGENTTY_RAG_MMR=0, AGENTTY_RAG_MMR_LAMBDA |
| Dedup | Drops near-duplicate passages | On, threshold 0.92 | AGENTTY_RAG_DEDUP=0, AGENTTY_RAG_DEDUP_THRESHOLD |
| Autocut | Cuts the list where scores drop sharply | On, sensitivity 2.0 | AGENTTY_RAG_AUTOCUT=0, AGENTTY_RAG_AUTOCUT_SENSITIVITY |
| Stitching | Joins adjacent hits from the same file | On | AGENTTY_RAG_STITCH=0 |
| Trimming | Fits passages into the size budget (see Token budget) | On | AGENTTY_RAG_OUTPUT_BYTES |
Every result starts with a short funnel line. It shows how many candidates each stage kept and whether the run was bm25 or hybrid. You can see when a query found little, or when the relevance floor cut a weak tail.
The off-by-default stages are listed in Optional stages. Turn one on only after measuring it on your own files with agentty rag-bench and real questions.
What gets indexed
| Source | Default | Setting |
|---|---|---|
| Docs folder | Found automatically | AGENTTY_DOCS_DIR, then ./docs, then ./.agentty/knowledge |
| Installed skills | On | AGENTTY_RAG_SKILLS=0 turns off |
| Learned memory | On | AGENTTY_RAG_MEMORY=0 turns off |
| MCP resources | Off | AGENTTY_RAG_MCP=1 turns on |
| Workspace source code | On demand | Indexed separately by search_code |
MCP indexing is off by default because reading a resource can mean server I/O. MCP indexes are not saved between sessions, so stale resources can't leak into a later run.
Meaning search and fallback
At startup agentty runs one quick embedding check. If the server or model is missing, the session stays keyword-only. This avoids a failed network timeout on every search. The check waits up to 30 seconds for Ollama to load a model. A server that isn't running fails at once.
With the backend on Auto (the default) and the model left at its default, agentty tries qwen3-embedding:0.6b first (src/rag/embed_backend.cpp:321) and uses nomic-embed-text if that isn't pulled. A model or backend you set yourself is used as is.
Models trained with query/document prefixes (nomic, e5, mxbai, arctic) get them automatically. AGENTTY_EMBED_PREFIX=0 turns that off. Other backends (an OpenAI-compatible endpoint, llama.cpp, or an in-process GGUF/ONNX model) are picked in the Retrieval pane.
AGENTTY_RAG_BM25_WEIGHT and AGENTTY_RAG_DENSE_WEIGHT (both default 1.0) set the keyword/meaning balance in rrf mode. The default convex mode adapts the balance per query instead.
Proactive retrieval
By default agentty does not search before each turn. The model calls search_docs or search_code only when it needs to, which costs fewer tokens.
To search automatically, type /settings and open Proactive retrieval (RAG). Pick one mode:
| Mode | What it does |
|---|---|
| On | Retrieve and inject context before every turn |
| First turn only | Ground the first turn of a thread, then stay quiet |
| Off | No injection before turns. The search tools still work |
The choice is saved in settings.json. A forked thread can have its own mode, chosen in the fork picker. AGENTTY_RAG_PROACTIVE=1 is the env-var equivalent of On:
export AGENTTY_RAG_PROACTIVE=1When it's on, agentty runs one retrieval in the background, shows retrieving context…, and starts the model when retrieval is done. It injects only when retrieval confidence is at least AGENTTY_RAG_PROACTIVE_MIN (default 0.35, adapter.cpp:760). The block is capped at about 6 KiB (~1,500 tokens), tighter than the tool budget, because you didn't ask for it:
export AGENTTY_RAG_PROACTIVE_BYTES=6144 # clamped 1 KiB - 32 KiBInjected context shows as its own retrieved-context card, not as your message. The card lists the sources and shows a confidence bar. These blocks never appear in ↑/↓ history recall. They ground only that turn, and older ones are dropped from later requests so you aren't charged for them again.
Optional stages
These are off by default. Each adds latency or can hurt results on some corpora.
| Stage | Turn on with | Default | Code |
|---|---|---|---|
| Pseudo-relevance feedback | AGENTTY_RAG_PRF=1 |
Off | adapter.cpp:735 |
| Corrective grading | AGENTTY_RAG_CORRECT=1 |
Off | adapter.cpp:736 |
| GraphRAG | AGENTTY_RAG_GRAPH=1 |
Off | adapter.cpp:737 |
| Multi-query expansion | AGENTTY_RAG_EXPAND=1 |
Off | adapter.cpp:740 |
| HyDE | AGENTTY_RAG_HYDE=1 |
Off | adapter.cpp:741 |
| Learning from file opens | AGENTTY_RAG_LEARN=1 |
Off | adapter.cpp:2832 |
PRF
AGENTTY_RAG_PRF=1 expands the query with words from the top keyword hits. It can help when your words differ from the file's words. It can also drift toward a wrong first result.
GraphRAG (off by default)
AGENTTY_RAG_GRAPH=1 builds links between documents, similarity edges, and communities, then expands results along the graph. Building the graph can take time quadratic in the number of documents, so it is not a default. Base and graph results are merged by rank because their scores use different scales.
Corrective grading
AGENTTY_RAG_CORRECT=1 grades each retrieval using its scores plus keyword support. A meaning match is not rejected just because it paraphrases the query. Grading keeps the number of results you asked for, so k=10 stays 10.
Multi-query and HyDE
AGENTTY_RAG_EXPAND=1 and AGENTTY_RAG_HYDE=1 use a small local model (AGENTTY_RAG_GEN_MODEL, default qwen2.5:0.5b) to rewrite the query. They help with vague or research-style questions but add model latency.
Learning
AGENTTY_RAG_LEARN=1 turns on an experimental loop that learns from which files you open next. It is stored in .agentty/rag_feedback.tsv. It's off because opening a file afterwards doesn't prove a skill, memory, or passage was relevant.
Saved indexes and freshness
Indexes are saved and reopened, not rebuilt on every start. They live with the project, so each checkout has its own:
<project>/.agentty/cache/rag_docs.<id>.ragdb # docs, skills, memory
<project>/.agentty/cache/rag_code.<id>.ragdb # search_codeAGENTTY_RAG_DIR moves them (see Environment).
<id> names the embedding model, so switching models keeps the old index for when you switch back. Each index stores the size and mtime of every file. When files change, only those files are reindexed, so one edited file costs seconds. Changing the chunking, the docs folder, or the model starts a fresh index. A large change set also triggers a clean rebuild.
The freshness scan:
- skips
.git, build outputs, dependency trees, virtual environments, and caches; - looks only at file types and sizes the loader can index;
- includes relative paths, so a rename updates the index;
- fingerprints skill and memory content, not just their count.
search_code keeps its own index, separate from docs. It splits source at definitions, so a result is usually a whole function, class, or declaration. It indexes at most 4,000 source files of up to 256 KiB each.
Token budget
Retrieval is the one place agentty spends model tokens for you, so its output is tightly budgeted. These settings shape how many tokens the ranked passages use. None of them change the ranking. All are on by default.
| Setting | What it does | Default |
|---|---|---|
AGENTTY_RAG_OUTPUT_BYTES |
Total size of all returned passages, clamped 2 KiB - 64 KiB | 12288 (~3,000 tokens) |
AGENTTY_RAG_RELEVANCE_FLOOR |
Keeps a passage only if its score is within this fraction of the top hit. The top hit is always kept. 0 keeps all |
0.30 |
AGENTTY_RAG_BUDGET_GAMMA |
Splits bytes in proportion to score^gamma, with a per-passage floor. Higher gives more to the top hits | 1.5 |
AGENTTY_RAG_CONF_BUDGET_FLOOR |
A low-confidence retrieval gets only this fraction of the total budget, rising to 100% at full confidence | 0.45 |
AGENTTY_RAG_EXTRACTIVE |
Compresses long prose passages. 0 forces a plain line window |
On |
AGENTTY_RAG_PROACTIVE_BYTES |
Separate cap for proactive injection | 6144 |
Defaults are in src/rag/adapter.cpp:216-290.
When a prose passage is too long, agentty keeps the sentences that best match the query (rare query words count more). It prints them in their original order with … marking gaps. No model is involved. Code and config keep a single block of lines with the most query words, because code is hard to read with gaps.
The requested k is still honored when enough distinct passages exist.
Measured effect: on a 161-file corpus, these settings together cut retrieval output from ~1,845 to ~1,605 estimated tokens per query (−13%), with identical ranking (recall@10 1.000, MRR 0.968, nDCG@10 0.976). Small chunks that already fit see little change. Retrieval is a small part of a turn's tokens, since the system prompt, tools, and history dominate and are mostly cached. Measure on your own corpus instead of assuming.
Measuring retrieval
Two built-in tools, both offline.
agentty rag-eval <cases.json> scores search against questions with known answers, using the real tools. agentty's own sets are in tests/rag_eval/ (code and docs, each with a held-out half kept out of tuning). A cases file is a list of {"q": "...", "gold": ["path", ...]}. Wrap it as {"tool": "search_docs", "cases": [...]} to test docs search. It prints hit@1 (right file first), hit@8, MRR, and every miss:
agentty rag-eval tests/rag_eval/agentty_code.json
AGENTTY_RAG_EVAL_ALL=1 agentty rag-eval my_cases.json # every case's rankWrite twenty questions about your repo, each with the file that answers it. Then you can check whether a model or setting change really helped. The rag_eval_guard test runs agentty's sets keyword-only on every build and fails if search gets worse.
agentty rag-bench benchmarks search_docs on your docs folder. It reports indexing time, recall@k, MRR, nDCG@10, and milliseconds per query:
agentty rag-bench docsTo measure token cost, add AGENTTY_RAG_MEASURE=1. The queries then go through the real output path (relevance floor, budget split, confidence budget, compression). It reports bytes and estimated tokens per query, plus a machine-readable MEASURE line:
AGENTTY_RAG_MEASURE=1 agentty rag-bench docsTo measure one setting, run twice with it toggled and compare the tokens= field:
AGENTTY_RAG_MEASURE=1 AGENTTY_RAG_EXTRACTIVE=0 agentty rag-bench docs # off
AGENTTY_RAG_MEASURE=1 agentty rag-bench docs # onNo rebuild is needed. rag-bench builds its queries from the documents, so it tests the mechanics. rag-eval with real questions tests what you actually care about. If a metric drops after turning on a stage, leave it off or tune it.
Provenance
Every passage keeps its source and path:
docs:guide/auth.md:20-44
skill:release-checklist
memory:fact-id
mcp:file://resource-uri
code:src/auth/token.cpp:80-126Use it to cite, open, or check what was retrieved.
Why it works this way
- Cheap by default. Slow or risky stages are off, so every search has predictable latency and token cost.
- Fallback, not failure. One startup check decides between hybrid and keyword-only, so a missing model never slows each search.
- Local. Indexes live in your project. Embeddings run on your machine unless you point
AGENTTY_OLLAMA_HOSTelsewhere. - Measured. Optional stages stay off until you measure them on your own corpus.
rag_eval_guardcatches regressions.