Akshat Khosla

How it works

• Three memory tiers, each chosen for a different access pattern: working memory in a process dict for zero-latency exact lookup, episodic memory in ChromaDB for semantic recall without exact keywords, and semantic memory in SQLite for deterministic fact lookup.
• Entity extraction runs on every message — spaCy NER plus regex relationship patterns, with a tech-term blocklist so "Python" never gets stored as a person's name.
• Calibrated confidence scores per source: PERSON detection 85%, GPE 80%, ORG and regex matches 75%, so downstream reads know how much to trust a fact.
• Exposed over MCP as a JSON-RPC server on stdio, which means it plugs straight into Claude Desktop as a tool provider.

Memory tiers

user message
      │
      ▼
extractor.py  spaCy NER + regex
      │
      ├─► working   dict      exact, 0-latency
      ├─► episodic  ChromaDB  semantic search
      └─► semantic  SQLite    structured facts
                          │
                          ▼
              context injected into
                 next session

How it works

• Provider-agnostic by design — Gemini, Ollama and any OpenAI-compatible endpoint sit behind one provider interface, so adding a new model is a subclass, not a rewrite.
• Two grades of scoring: LLM-as-judge with structured rubrics (0–10) for nuance, plus deterministic keyword and regex judges for fast, reproducible checks.
• Purpose-built judges for Faithfulness, Relevance and Coherence, each scoring 1–5 with written reasoning attached to the result.
• Async runner — per-sample tasks fan out with asyncio.gather, and each provider gets its own semaphore so a rate-limited API can be capped independently of the rest.
• Full execution tracing in an in-process OpenTelemetry-style span tracer, persisted as JSONB alongside the run and replayed in the dashboard.

Request path

CLI ──┐
        ├──► FastAPI POST /api/v1/evals
React ─┘         │ add_task
                  ▼
            run_eval_background
                  │
         asyncio.gather per sample
                  │
      ┌───────────┴───────────┐
      ▼                       ▼
target LLM            judge pool
semaphore-gated        gather(...)
      └───────────┬───────────┘
                  ▼
       PostgreSQL eval_runs
       + trace_data JSONB

How it works

• Ingestion extracts text from PDF/DOCX/TXT, chunks it into 512-token windows with 64-token overlap so context survives the boundaries, embeds with all-MiniLM-L6-v2 into 384-dim vectors, and upserts into a persistent ChromaDB collection.
• Retrieval embeds the query with the same model — non-negotiable for a shared vector space — and runs cosine ANN over an HNSW index, returning top-k chunks with scores.
• Three tools exposed to the model: search_documents, list_documents and summarise_document. Tool descriptions are written with the same care as a system prompt, since they are what the model reads to decide when to call.
• Multi-step reasoning falls out of the loop: asked to compare two reports, the agent lists documents, searches each one separately, then synthesises — rather than one-shot retrieving and hoping.

Agentic loop

while True:
  resp = claude(tools, messages)

  end_turn  ─► stream answer, done
  tool_use  ─► execute_tool()
               append tool_result
               loop ──┐
                      │
           ┌──────────┘
           ▼
  search_documents
  list_documents
  summarise_document
           │
           ▼
   ChromaDB · cosine HNSW
   384-dim · top-k + scores