Architecture

Indexing pipeline

This page describes how the index is built. What it exposes once built is the tools reference; which languages reach which depth of the pipeline below is the language capability matrix. For how persistent graph indexing reduces AI coding agent token costs, see how a code graph MCP server works.

trace-mcp uses a two-pass indexing pipeline:

Source files (PHP, TS, Vue, Python, Go, Java, Kotlin, Ruby, HTML, CSS, Blade)
    │
    ▼
┌──────────────────────────────────────────┐
│  Pass 1 — Per-file extraction            │
│  Language plugins (tree-sitter) →        │
│    symbols (functions, classes, etc.)     │
│  Integration plugins →                   │
│    routes, components, migrations,       │
│    events, models, schemas, variants     │
└────────────────────┬─────────────────────┘
                     │
                     ▼
┌──────────────────────────────────────────┐
│  Pass 2 — Cross-file resolution          │
│  Module resolvers:                       │
│    PSR-4 · ES modules · Python modules  │
│  Integration plugins resolveEdges():     │
│    Vue component references              │
│    Inertia render → page mapping         │
│    Blade template inheritance            │
│    ORM relationship resolution           │
│    Route → controller binding            │
│  → unified directed edge graph           │
└────────────────────┬─────────────────────┘
                     │
                     ▼
┌──────────────────────────────────────────┐
│  Per-project SQLite (WAL mode) + FTS5   │
│  nodes · edges · symbols · routes       │
│  + optional: embeddings · summaries     │
└────────────────────┬─────────────────────┘
                     │
                     ▼
┌──────────────────────────────────────────┐
│  Subprojects (auto, post-index)         │
│  Topology DB (~/.trace/topology.db)     │
│  Auto-detect services per project       │
│  Contracts · Endpoints · Client calls   │
│  Cross-service impact edges             │
└──────────────────────────────────────────┘

Incremental by default — files are content-hashed; unchanged files are skipped on re-index.

When AI is enabled, a background pipeline runs after indexing to generate summaries and embeddings for key symbols.

Storage

All state is centralized in ~/.trace/, and what goes in it is set by configuration:

~/.trace/
  .config.json              # global config + per-project settings
  registry.json             # project registry (all added projects)
  topology.db               # cross-service topology + subproject graph
  analytics.db              # session analytics (cross-project)
  activity.db               # tool-call history behind the Activity tab (7-day retention)
  savings.json              # cumulative token savings tracker
  index/
    my-app-a1b2c3d4e5f6.db  # per-project SQLite databases
    api-server-b2c3d4e5.db

Each project gets its own SQLite database, named <project-basename>-<sha256-hash-of-path>.db. The project registry tracks which projects are registered, their root paths, and last index time. Nothing is stored in the project directory itself. For how background memory usage, cache limits, and daemon footprint are bounded across SQLite databases, see daemon memory. All databases stay local to your machine; for storage boundaries and telemetry policies, see privacy.

The topology database (topology.db) is shared across all projects. It stores:

Subprojects (= services) — bound to projects, auto-detected or manually added
API contracts — parsed OpenAPI, GraphQL SDL, Protobuf specs
Endpoints — normalized API endpoints extracted from contracts
Client calls — HTTP/gRPC/GraphQL calls discovered in code
Cross-subproject edges — links between client calls and endpoints

Each subproject is bound to a project via project_root. A project can have multiple subprojects (frontend, backend, etc.), and the same subproject can belong to multiple projects.

The decision memory database (decisions.db) is also shared across all projects. It stores:

Decisions — architectural decisions, tech choices, bug root causes, preferences, etc., each with temporal validity (valid_from/valid_until) and optional code linkage (symbol_id, file_path, service_name)
Session chunks — chunked conversation content from AI session logs, FTS5-indexed for cross-session search
Mined sessions tracker — prevents re-processing already-mined session files

The activity database (activity.db) records every MCP tool call the daemon serves, keyed by project root: timestamp, tool, params summary, result count and tokens, latency, error flag, session id. It answers one question — what has been happening in this project — which is why it is on disk rather than in the SessionJournal: a journal belongs to one MCP session and dies with it, so the Activity tab read 0 calls in every window while agents were actively working (TRA-1071). Entries are batched (128 rows or 2 s) and pruned to 7 days, which is what lets the tab offer a 24h window honestly. The API reports recording_since so the UI can say when recording started instead of presenting a young install’s empty 24h window as a confident zero.

Decisions are auto-enriched into code intelligence tool responses (get_change_impact, plan_turn, get_wake_up) via the enrichment layer in src/memory/enrichment.ts.

Plugin system

Plugins are the core extensibility mechanism. There are two types:

Language plugins

Located in src/indexer/plugins/language/. Each plugin handles symbol extraction for one language using tree-sitter. Depth varies by plugin — see the language capability matrix for which ones resolve import, call and type edges, and supported frameworks for the integration layer on top.

Registered plugins: PHP, TypeScript/JavaScript, Vue, Python, Go, Java, Kotlin, Ruby, HTML, CSS.

Integration plugins

Located in src/indexer/plugins/integration/, organized by category:

Category	Plugins	What they do
`framework/`	Laravel, Django, Rails, Spring, NestJS, Express, FastAPI, Flask, Hono, Fastify, Nuxt, Next.js	Route, controller, middleware extraction
`orm/`	Prisma, TypeORM, Sequelize, Mongoose, SQLAlchemy, Drizzle	Model, relationship, migration extraction
`view/`	React, Vue, React Native, Blade, Inertia, shadcn, MUI, Ant Design, Headless UI, Nuxt UI	Component tree, prop, render analysis
`api/`	GraphQL, tRPC, DRF	Schema, endpoint, resolver extraction
`validation/`	Zod, Pydantic	Schema definition extraction
`state/`	Zustand	Store, action, selector extraction
`realtime/`	Socket.io	Event handler, namespace extraction
`testing/`	Testing	Test suite, fixture, coverage analysis
`tooling/`	Celery, n8n, data-fetching	Task, workflow, query hook extraction

Plugin interface

Every integration plugin implements FrameworkPlugin:

interface FrameworkPlugin {
  manifest: PluginManifest;           // name, version, priority, dependencies
  detect(ctx: ProjectContext): boolean; // returns true if framework detected
  registerSchema(): NodeTypes & EdgeTypes; // declares symbol/edge types
  extractNodes?(filePath, content, language): FileParseResult; // extract symbols
  resolveEdges?(ctx: ResolveContext): RawEdge[]; // resolve inter-symbol edges
}

Detection runs once on startup. Only plugins whose detect() returns true participate in indexing.

Plugins are loaded in topological order (respecting dependencies) and by priority (lower = earlier).

Module resolution

Three module resolvers handle cross-file imports:

Resolver	Languages	What it resolves
ES modules	TypeScript, JavaScript, Vue	`import` / `require` with tsconfig paths, barrel exports
PSR-4	PHP	Namespace-based autoloading per `composer.json`
Python modules	Python	Relative/absolute imports, `__init__.py` packages

Scoring & ranking

src/scoring/ contains algorithms for ranking search results and context assembly:

BM25 — full-text relevance via FTS5
PageRank — symbol importance based on the dependency graph
Hybrid scoring — combines BM25 + graph signals
Structured assembly — assembles context within a token budget, maximizing coverage

Alongside the code graph, trace-mcp keeps a second store of why the code is shaped this way — see decision memory. Writing a new plugin for either pass is covered in development.

Tech stack

Component	Technology
Parsing	tree-sitter (PHP, TS, Python, Go, Java, Kotlin, Ruby, HTML, CSS), @vue/compiler-sfc
Database	better-sqlite3 — WAL mode, FTS5, vector storage
Module resolution	oxc-resolver (ESM/CJS), PSR-4, Python modules
AI	Ollama / OpenAI — embeddings, summarization, reranking, inference caching
Validation	Zod — config + input validation
Error handling	neverthrow — Rust-style `Result<T, E>`
Logging	pino — structured JSON logging
MCP	@modelcontextprotocol/sdk
Build	tsup · vitest · TypeScript 5.7

Project structure

src/
├── ai/                     # Embeddings, reranker, summarization, vector store, inference caching
├── db/                     # SQLite schema, store, FTS5
├── subproject/             # Subproject layer (subprojects = services, bound to projects)
│   ├── manager.ts          #   Add/remove/sync subprojects, auto-discover projects, cross-subproject impact
│   └── scanner.ts          #   HTTP/gRPC/GraphQL client call scanner
├── topology/               # Cross-service topology layer
│   ├── topology-db.ts      #   Topology + subproject SQLite store (subprojects bound to projects via project_root)
│   ├── contract-parser.ts  #   OpenAPI, GraphQL SDL, Protobuf parsers
│   └── service-detector.ts #   Subproject discovery (Docker Compose, flat/grouped workspace, monolith fallback)
├── indexer/
│   ├── plugins/
│   │   ├── language/       # 81 languages — PHP, TS, Vue, Python, Go, Java, Kotlin, Ruby, Rust,
│   │   │                   #   C/C++/C#, Swift, Dart, Scala, Zig, OCaml, Clojure, F#, Elm,
│   │   │                   #   CUDA, COBOL, Verilog, GLSL, Svelte, MATLAB, Lean, Wolfram, …
│   │   └── integration/    # 85 plugins organized by category:
│   │       ├── framework/  #   Laravel, Django, Rails, Spring, NestJS, Express, FastAPI,
│   │       │               #   Flask, Hono, Fastify, Nuxt, Next.js
│   │       ├── orm/        #   Prisma, TypeORM, Sequelize, Mongoose, SQLAlchemy, Drizzle
│   │       ├── view/       #   React, Vue, React Native, Blade, Inertia, shadcn, MUI,
│   │       │               #   Ant Design, Headless UI, Nuxt UI
│   │       ├── api/        #   GraphQL, tRPC, DRF
│   │       ├── validation/ #   Zod, Pydantic
│   │       ├── state/      #   Zustand
│   │       ├── realtime/   #   Socket.io
│   │       ├── testing/    #   Playwright, Cypress, Jest, Vitest, Mocha
│   │       └── tooling/    #   Celery, n8n, data-fetching
│   ├── resolvers/          # PSR-4, ES module, Python module resolution
│   ├── pipeline.ts         # Two-pass indexing engine
│   ├── watcher.ts          # File change watcher
│   └── monorepo.ts         # Monorepo workspace detection
├── memory/                 # Decision memory (cross-session knowledge graph)
│   ├── decision-store.ts   #   SQLite store: decisions + session chunks + FTS5
│   ├── conversation-miner.ts # Pattern-based decision extraction from JSONL logs
│   ├── session-indexer.ts  #   Chunked session content indexer for search
│   ├── wake-up.ts          #   L0/L1/L2 wake-up context assembler
│   ├── enrichment.ts       #   Decision injection into code intelligence results
│   └── index.ts            #   Barrel export
├── analytics/              # Session analytics engine
│   ├── log-parser.ts       #   JSONL parser (Claude Code + Claw Code)
│   ├── analytics-store.ts  #   SQLite storage for parsed sessions
│   ├── sync.ts             #   Incremental session log sync
│   ├── session-analytics.ts #  Analytics query facade
│   ├── rules.ts            #   8 optimization rules (repeated reads, bash-grep, etc.)
│   ├── real-savings.ts     #   Real savings analysis (Read vs get_symbol)
│   ├── benchmark.ts        #   Synthetic benchmark (5 scenarios)
│   ├── tech-detector.ts    #   Manifest parser + coverage assessment
│   └── known-packages.ts   #   Catalog of ~200 known packages
├── tools/                  # 182 MCP tool implementations
├── scoring/                # PageRank, BM25, hybrid scoring, structured assembly
├── plugin-api/             # Plugin registry, loader, executor, test harness
├── init/                   # Setup & detection (Claude Code, Claw Code, Cursor, Windsurf, Continue)
├── utils/                  # Env parser, hasher, security, source reader, token counter
├── server.ts               # MCP server factory
├── config.ts               # Cosmiconfig + Zod validation
├── errors.ts               # Error types (neverthrow)
├── logger.ts               # Pino logger setup
├── cli.ts                  # Commander CLI (serve, serve-http, index, subproject, analytics)
├── cli-analytics.ts        # Analytics CLI subcommands (sync, report, optimize, benchmark, coverage, savings, trends)
└── cli-subproject.ts       # Subproject CLI subcommands (add --project, list --project, etc.)

Last updated: September 11, 2026