Retrieval (RAG)

agentty ships a complete, state-of-the-art retrieval engine behind two tools:

• search_docs — searches your knowledge base: a docs folder, your installed skills, your learned memory, and (optionally) connected MCP resources.
• search_code — semantic search over your source code by meaning, for "where is retry backoff handled" style questions where you don't know the identifier.

The whole engine is local, dependency-free, and degrades gracefully. With no embedding model reachable it falls back to keyword (BM25) search and keeps working. Nothing is sent to the cloud; the only optional network hop is a localhost Ollama server for embeddings.

The one-paragraph version

You ask a question. agentty retrieves a wide candidate pool with hybrid search (keyword BM25 + dense embeddings, fused), re-ranks it for precision, diversifies away near-duplicates, compresses each hit to its most relevant span, stitches small chunks back into their surrounding context, and expands along the corpus's document graph (author-drawn links + inferred entity relationships) to pull in the connecting context a flat index misses — then hands the model a short, high-signal, source-tagged set of passages. Every stage that costs nothing runs by default; anything that costs a model call is opt-in.

What gets indexed

Even with no docs folder at all, search_docs still searches your skills and memory — so it's useful from the first turn.

The retrieval funnel

Every search_docs call runs this pipeline. The default path uses no LLM calls — it's fast and predictable.

1. Hybrid retrieval — cast a wide net

Two independent retrievers score every chunk, and their ranked lists are fused with Reciprocal Rank Fusion (RRF):

• BM25 (keyword) — catches exact terms, proper nouns, code symbols. Porter-stemmed by default so run / runs / running match.
• Dense (embeddings) — catches paraphrase and semantic near-matches. Uses a localhost Ollama embedding model (nomic-embed-text by default); at scale, an in-memory HNSW index makes the nearest-neighbour search sub-linear.

Dense is weighted slightly above lexical (1.3×) because semantic matching usually wins on natural-language docs — both weights are tunable (AGENTTY_RAG_W_DENSE / AGENTTY_RAG_W_LEXICAL).

2. Pseudo-relevance feedback — recover the words you didn't type (default-on)

A bare query rarely uses the exact vocabulary the docs use. agentty runs an initial BM25 pass, treats the top hits as pseudo-relevant, harvests their most discriminative terms (feedback frequency × rarity), and fuses in a second, down-weighted BM25 probe over {query + those terms}. A chunk that matches both the literal query and the expanded vocabulary is reinforced.

This is the classic RM3 technique — deterministic, sub-millisecond, no model, no network — which is why it's on by default. Disable with AGENTTY_RAG_PRF=0; tune its weight with AGENTTY_RAG_W_PRF.

3. Contextual retrieval — a chunk knows where it lives (default-on)

Each chunk carries a breadcrumb of its document title and markdown heading path (guide.md › Installation › Linux). That breadcrumb is indexed on both the keyword and embedding sides, so a chunk that says "run the installer" is findable by "linux install" even though neither word appears in its body. (This is Anthropic's 2024 "contextual retrieval", built for free during chunking — no per-chunk model call.)

4. Re-ranking — precision at the top

First-pass fusion is recall-oriented and noisy at the very top, so the pool is re-scored:

• Feature-fusion reranker (default-on) — cheap, deterministic signals the first pass ignores: exact query-term coverage, phrase proximity, title/path match, and (when embeddings are present) calibrated cosine similarity. Pure C++, zero network.
• Late-interaction rerank (sentence MaxSim) (default-on when embeddings are available) — a chunk-level vector blurs a multi-topic chunk; ColBERT-style late interaction re-scores each candidate by its best sentence against the query (blended with the runner-up, so one lucky sentence in a sea of noise doesn't win). Costs the same single batched /api/embed round-trip as chunk-level cosine. AGENTTY_RAG_LATE=0 falls back to the chunk-level rerank below.
• Embedding cross-encoder rerank (fallback tier) — one batched /api/embed round-trip re-scores the candidate pool by fresh whole-chunk cosine. Degrades to the input order if the backend is unreachable, so it's safe to leave on.
• Generative cross-encoder rerank (opt-in — AGENTTY_RAG_NEURAL=1) — a graded 0–10 relevance judgment per candidate from a local generative model. The strongest reranker, but one LLM call per candidate, so it's off by default.
• Learned prior (default-on) — the learning loop's read side (see below): each chunk's score gets a bounded nudge (×0.85–×1.15) from its historical usefulness. Neutral with no history — a fresh workspace ranks exactly as before.

5. MMR diversification — no near-duplicates (default-on)

Maximal Marginal Relevance greedily keeps hits that are both relevant and different from what's already selected, so three overlapping windows of the same paragraph don't crowd out three distinct answers.

6. Compression — signal, not noise (default-on)

Rather than dump a whole 1600-char chunk into the model's window, each surviving hit is split into sentences, scored by query relevance, and trimmed to its best contiguous span. "20k noisy tokens" becomes "2k useful tokens."

7. Parent-document expansion — read it in context (default-on)

Small chunks retrieve precisely but read out of context. After ranking, each surviving chunk is stitched back into its adjacent sibling chunks from the same document, so the model sees the precise hit inside its surrounding prose — without widening the retrieval probe. Pure in-memory, no network. Tune the window with AGENTTY_RAG_PARENT_RADIUS; disable with AGENTTY_RAG_PARENT=0.

8. GraphRAG expansion — retrieval over the document graph (default-on)

A flat chunk index treats every passage as an island. Real knowledge bases aren't flat: documents link to each other, and documents about the same thing share vocabulary. agentty builds that structure into an explicit document graph and retrieves over it — the full GraphRAG recipe (entity graph → communities → community summaries → graph-aware retrieval), but with the expensive LLM ingredients made deterministic and free by default.

The graph (nodes = documents) is built once per corpus and memo-cached — invalidated only when the corpus itself changes. It has two kinds of edge:

• Link edges — resolved markdown links (](other-doc.md)). An author-curated relevance signal most engines throw away. Ambiguous targets (a filename owned by two docs) are skipped rather than guessed, so a wrong edge never poisons the graph.
• Entity edges — the LLM-free half of GraphRAG's entity graph. Each doc's salient entities are its top terms by tf·idf that are also rare corpus-wide (they discriminate). Two documents that share ≥2 such entities get an edge — related even when the author drew no link (quantizer.md ↔ codebook.md connect because they co-mention the same rare identifiers).

Authority. PageRank runs over the link graph only (being cited by an author is an endorsement; entity similarity is symmetric, not a vote). It supplies a deterministic authority prior — hub docs win ties.

Communities. Deterministic label propagation over the union of link + entity edges (a dependency-free stand-in for Leiden clustering) partitions the corpus into topic clusters.

Retrieval expands around the top hits through four tiers, in priority order, each ranked internally by PageRank, all scored below every direct hit so expansion is supporting material and never displaces a real answer:

All of it is deterministic, in-memory, and needs no model. Disable with AGENTTY_RAG_GRAPH=0.

Community summaries — the full GraphRAG global search (opt-in)

The one ingredient that genuinely needs a model is the community report: a short natural-language summary of what a whole cluster is about, which lets a corpus-level question ("how does the retrieval engine fit together?") be answered by a digest instead of one lead chunk. Enable it with AGENTTY_RAG_GRAPH_SUMMARY=1 (needs a local Ollama). When the community hub is selected, its passage carries a 2-3 sentence LLM overview of the cluster in place of its raw lead chunk.

The cost is paid once per community per corpus shape, not per query: summaries are keyed on the corpus signature plus the community's member paths and persisted to .agentty/rag_graph_summaries.tsv (override with AGENTTY_RAG_GRAPH_SUMMARIES_PATH), so every later session — and every later query touching that community — reads a cached line with zero network. Editing any document invalidates exactly the affected communities. Any failure (backend down, parse error) degrades to the plain hub chunk and is not cached, so a later session with the model up fills it in. Pick the model with AGENTTY_RAG_GRAPH_SUMMARY_MODEL (default llama3.2).

9. Corrective retry — a second chance on a weak result (default-on)

agentty computes a confidence signal for the result set. If the first pass looks weak, it strips stopwords, widens the pool, and retries — recovering hits when the original query was buried in conversational phrasing. Disable with AGENTTY_RAG_CORRECT=0.

Conversation-aware retrieval (default-on)

The query you type is not always the query you mean — especially mid-conversation. Two deterministic rewrites widen the probe set (never replacing your original query, so recall can only rise):

• Carryover — a recency-decayed salience pool tracks the content terms of your recent queries. A vague follow-up ("how does it handle errors?") gets the top carried-over terms appended as an extra probe, resolving the pronoun to the entities under discussion. AGENTTY_RAG_CARRYOVER=0 disables.
• Multi-hop decomposition — a compositional question ("how the auth flow works and how the sandbox blocks paths") has no single covering chunk. It's split on clause connectives into per-facet probes that ride the same RRF fusion, so each facet retrieves on its own strength. Gated conservatively (≥2 clauses, ≥2 content terms each) so ordinary queries pass through untouched. AGENTTY_RAG_MULTIHOP=0 disables.

The learning loop (default-on)

agentty is the rare retrieval engine that sees what happens after it answers — and it uses that. Every passage search_docs surfaces counts a use; when the agent follows up by reading the file a passage pointed at, that counts a win (an implicit relevance judgment — the passage pointed somewhere worth acting on). The Beta-smoothed win-rate per passage persists to .agentty/rag_feedback.tsv (human-inspectable TSV) and folds back into ranking as a bounded nudge — passages that repeatedly help rise, chronic noise sinks, and near-ties resolve toward what has actually worked in this workspace. Retrieval gets better the more you use it. AGENTTY_RAG_LEARN=0 disables; delete the TSV to forget.

Measure it: agentty rag-bench

A pipeline you can't measure is a pile of vibes. agentty rag-bench [dir] benchmarks the funnel on your own corpus, offline, in milliseconds. It synthesizes known-item queries from sampled chunks — each query is a chunk's most discriminative terms by tf×idf, so the chunk it came from is the known gold answer (deterministic and reproducible, no LLM needed). Then it runs the retrieval ladder — bm25-only → hybrid+prf → +feature-rerank → +mmr — over every query and reports recall@k, MRR, and nDCG per rung. Because each rung adds exactly one stage, a metric that drops at a rung points at the stage worth tuning, and every AGENTTY_RAG_* knob can be set against numbers instead of guesses.

Gold matching is coverage-based, not exact-identity: a returned chunk counts as a hit when it covers the gold chunk's region (same document + overlapping line span). This matters because the reranker deliberately collapses overlapping-window siblings into one survivor — which contains the gold region but may start on a different line. Scoring that as a miss would falsely flag the reranker as a regression; coverage matching measures the retrieval outcome the user actually gets.

Known-item synthesis exercises the funnel's mechanics; run it with a local embedder up to also exercise the dense (embedding) rungs, which is where semantic paraphrase wins show.

Opt-in recall boosters (they cost a model call)

Three of the strongest techniques need a generative model, so they add latency and are off by default. When enabled they help every knowledge configuration — docs, skills-only, memory-only, MCP, or any mix — because they feed extra probes into the same source-agnostic fusion.

• RAG-Fusion query expansion (AGENTTY_RAG_EXPAND=1) — the LLM rewrites your query into several alternative phrasings; each is retrieved and all results are fused. Vocabulary mismatch on any one phrasing stops being fatal. Variant count via AGENTTY_RAG_EXPAND_N (default 4).
• HyDE — Hypothetical Document Embeddings (AGENTTY_RAG_HYDE=1) — a question and its answer can sit far apart in embedding space. The LLM hallucinates a short answer-passage for your query; embedding that fake answer lands the probe near the real passages. The answer needn't be correct — only look like one. Composes with, and is independent of, query expansion.
• GraphRAG community summaries (AGENTTY_RAG_GRAPH_SUMMARY=1) — a cached natural-language report per topic cluster, so corpus-level questions are answered by a digest rather than one chunk (see §8). Unlike the two above, the model cost here is paid once per community per corpus shape, not per query — the rest of the graph is free and on by default.

The proactive path

Beyond the explicit search_docs tool, agentty can retrieve before you even ask. When your message looks knowledge-shaped, it runs the funnel pre-turn and injects a <retrieved-context> block (source-tagged, deduplicated across turns) into the prompt — grounding the answer in your docs/skills/memory without a tool round-trip. This is on by default (AGENTTY_RAG_PROACTIVE=1) and only injects above a confidence bar (AGENTTY_RAG_PROACTIVE_MIN, default 0.45).

Provenance

Every returned passage is tagged with its source (docs:, skill:, memory:, or an MCP URI) and its file + line range. agentty never discards where a piece of information came from — cite it, open it, or follow it.

Enabling the dense half

BM25 works with zero setup. To turn on the semantic half:

# 1. Run a local embedding model
ollama pull nomic-embed-text
ollama serve            # localhost:11434 by default

# 2. Point agentty at a docs folder (optional — skills/memory are always indexed)
export AGENTTY_DOCS_DIR=~/my-project/docs

That's it — agentty auto-detects the running Ollama server and upgrades from BM25-only to full hybrid retrieval. Override the model or host with AGENTTY_EMBED_MODEL / AGENTTY_OLLAMA_HOST.

Full knob reference

Every environment variable — defaults, ranges, and effects — is documented in the Configuration table under the AGENTTY_RAG_* and BM25_* rows.

Design notes

• No dependencies. BM25, RRF, HNSW, the reranker, MMR, compression, PRF, the chunker, and the GraphRAG document graph (PageRank, entity extraction, community detection) are all in-house C++/STL. The only optional network hop is localhost Ollama.
• Graceful degradation everywhere. No embeddings → BM25-only. Ollama unreachable mid-search → the affected stage no-ops and retrieval continues. Community-summary backend down → the plain hub chunk. Empty corpus, blank query, zero-k → empty result, never an error.
• Deterministic by default. Every default-on stage — including the whole GraphRAG graph, its PageRank, and its communities — is deterministic given the corpus, so results are reproducible and rag-bench numbers are stable. Only the opt-in LLM stages introduce a model.
• Cached, not recomputed. The document graph is memo-cached per corpus shape; community summaries and the learning-loop feedback are persisted to disk (.agentty/rag_graph_summaries.tsv, .agentty/rag_feedback.tsv) so expensive work is paid once and survives across sessions.
• Sensible defaults. Everything that's free and safe runs by default; anything that costs a model call is opt-in, so the default search_docs is fast and predictable.
• One seam for RAG and MCP. From the model's view a docs folder and an MCP server are the same thing — a KnowledgeSource behind one interface — so they fuse identically.