Benchmarks: AI agents & graph engines.
We run two distinct types of benchmarks:
1. End-to-End AI Agent Workflows (measuring how much time, tokens, and missed dependencies a graph saves an LLM on real tasks).
2. Graph Engine Head-to-Head (measuring cold scan speed, RAM, query latency, and extraction recall without an LLM).
1. AI agents in real tasks (time, tokens & completeness).
Identical model (Gemini 3.8 Flash), identical prompt, same pinned repository commit. Measures how fast an autonomous agent finishes a multi-hop architecture audit, how many tokens it burns, and whether it finds every affected service and table.
| Case Study | Engineering Task & Codebase | Compared Tools | Speed, Tokens & Completeness | Report |
|---|---|---|---|---|
| Case Study 01 twentyhq/twenty |
Find all DI service callers, class inheritance chains, BullMQ queue producers/consumers, and API-to-database write paths across 46 TypeScript packages (24,527 files, 2.49M LOC). | CodeExplorervs CBM |
~6.0× faster (1m 33s vs 9m 15s) ~7.0× fewer tokens (~242K vs ~1.7M) 100% vs 3.4% native graph recall |
Read Case 01 |
| Case Study 02 324 Microservices |
Full blast-radius audit across shared ORM tables, synchronous cross-service HTTP/RPC fan-in, and RabbitMQ/PubSub queues (13,782 files, TS + Go + SQL, zero schema hints). | CodeExplorervs ripgrep + Python |
~7.0× faster (4m 16s vs 29m 52s) ~25× fewer tokens (~686K vs ~17.2M) 100% vs ~82% (rg missed 9 services) |
Read Case 02 |
| Case Study 05 OrchardCore |
4-part architectural governance, AppSec AuthorizeAsync audit, YesSql.ISession layer violations, PublishedAsync reactive pipeline, and IContentManager blast radius across 334 projects (7,837 files). |
CodeExplorervs ripgrep |
~2.5× faster (4m 25s vs 11m 04s, 92 vs 193 turns) ~4.1× fewer tokens (~3.9M vs ~16.1M, ~2.9× cheaper) 100% vs ~93% (331/331 vs 307/331 items) |
Read Case 05 |
| Case Study 06 argoproj/argo-cd |
Automated pre-commit architectural guard & 4-phase SHA-256 ratchet showcase: enforce 6 invariant rules across 18 subsystems (4,216 files), freeze legacy debt with line-shift immunity, block single-line PR leaks, and monotonically tighten baselines. | CodeExplorer(Native SIG) |
422 ms native build check 0 tokens ($0.00) · 4-phase SHA-256 ratchet 6 of 6 gates evaluated (2 PASS, 1 WARN, 3 FAIL) |
Read Showcase |
2. Graph engine head-to-head (no LLM in the loop).
Direct indexer and MCP server comparison on twenty (24,527 files) and immich (3,205 files) across 371 source-verified items: cold scan time, memory/disk footprint, query latency, and extraction recall.
| Case Study | What Was Measured | Compared Engines | Indexing Speed & Graph Recall | Report |
|---|---|---|---|---|
| Case Study 03 twenty + immich |
Cold indexing speed, RAM & SQLite DB size, warm JSON-RPC query latency, basic code recall (130 items: routes, HTTP, calls, inheritance), and architecture recall (241 items: queues, ORM tables, SQL lineage). | CodeExplorervs CBM |
Basic Code: 78% vs 38% (102/130 vs 49/130) DB & Queues: ~98% vs 0% (235/241) Scan: ce faster on immich; CBM faster on twenty |
Read Case 03 |
| Case Study 04 twenty + immich |
Monorepo indexing scalability from 461k LOC (immich) to 2.49M LOC (twenty), cold MCP server startup time, and graph recall across all 371 items. |
CodeExplorervs Graphify |
~6.1× faster scan on immich (~18s vs ~108s)>18× faster on twenty (~147s vs DNF >45m)Basic Code: 78% vs 18% · DB & Queues: ~98% |
Read Case 04 |
1. CodeExplorer (
ce): v1.75.0 to v1.80.2 (.NET 10 & .NET 11, SQLite + OpenCypher, 36 MCP tools)2. codebase-memory-mcp (
CBM): v0.12.1 (win32-x64 Go + SQLite binary, 17 MCP tools)3. Graphify (
graphifyy): v0.9.104 (Python 3.10, Tree-sitter + NetworkX --code-only, 10 MCP tools)4. Lexical CLI Baseline:
ripgrep (rg v14.1.1) + Python 3.10 (Case 02) + view_file5. LLM Model (Track 1):
Gemini 3.8 Flash (symmetric system prompts across paired runs)6. Pinned Target Commits:
twentyhq/twenty @ b9ef7107 (Case 01), twentyhq/twenty @ 12e8fae4 & immich-app/immich @ 2153d22f (Cases 03 & 04), OrchardCMS/OrchardCore @ 05ad2d9a (Case 05)