Case Study 03: CodeExplorer vs codebase-memory-mcp (CBM).
Head-to-head indexer and MCP server comparison on twenty and immich: how accurately each engine extracts routes, function calls, inheritance, queues, and database tables across 371 source-verified items.
Basic Code & Call Graph
Across 130 basic items supported by both tools (routes, HTTP calls, DI method calls, class inheritance): ce scored 102/130 vs CBM 49/130 (7 wins, 1 tie, 0 losses).
102 / 130 vs 49 / 130
Queues, DB Tables & SQL
Across 241 message queue, ORM table, SQL read/write lineage, and sub-project items: ce matched 235/241. CBM does not index these.
235 / 241 verified items
OpenCypher Latency
Single-shot warm OpenCypher round-trip in ce (4.3 to 4.7 ms) vs 38.1 to 71.2 ms in CBM over persistent JSON-RPC stdio.
~8.9× to 15× faster Cypher query
Where CBM Won
CBM indexed twenty ~1.5× faster with ~2.2× lower peak RAM (2.1 GB vs 4.7 GB), while ce was faster on immich with a 5.1× smaller DB.
CBM faster on 24.5k-file cold scan
1. CodeExplorer (
ce): v1.75.0 (.NET 10, SQLite + OpenCypher, 36 MCP tools)2. codebase-memory-mcp (
CBM): v0.12.1 (win32-x64 Go + SQLite binary, 17 MCP tools)3. Target Repositories:
twentyhq/twenty (commit 12e8fae4, 24,527 files, 2.49M LOC) & immich-app/immich (commit 2153d22f, 3,205 files, 461k LOC)1. Graph completeness: basic code vs full architecture.
CBM indexes files, classes, routes, and function calls, whereas CodeExplorer also extracts message queues, ORM database tables, and SQL read/write queries. To keep the comparison fair, we separate the 371 verified items into Basic Code (130 items supported by both tools) and Architecture, DB & Queues (241 items supported only by CodeExplorer).
| What Was Tested | Repo | Sample Size | CodeExplorer |
CBM |
Why the Scores Differ |
|---|---|---|---|---|---|
Incoming HTTP & GraphQL Endpoints (C1) |
twenty |
20 | 100% (20/20) | 0% (0/20) | ce indexes NestJS GraphQL @Resolver operations and template-literal controller routes. |
Incoming HTTP & FastAPI Endpoints (C1) |
immich |
20 | 100% (20/20) | 90% (18/20) | ce resolves TypeScript enum route prefixes (@Controller(RouteKey.Asset)). |
Outbound HTTP Client Calls (C2) |
twenty |
12 | 50% (6/12) | 25% (3/12) | ce extracts fetch, axios, and wrapper HTTP client calls. |
Outbound HTTP Client Calls (C2) |
immich |
6 | 83% (5/6) | 0% (0/6) | ce resolves variable-URL and template-literal HTTP requests. |
Function & DI Service Call Graph (C6) |
twenty |
12 | 100% (12/12) | 75% (9/12) | CBM misses calls on constructor-injected TypeScript service fields across files. |
Function & DI Service Call Graph (C6) |
immich |
20 | 100% (20/20) | 60% (12/20) | ce links NestJS constructor DI properties to target class methods. |
Class Inheritance & implements (C7) |
twenty |
20 | 80% (16/20) | 20% (4/20) | ce links cross-package base classes and generic TypeScript interfaces. |
Class Inheritance & Mixins (C7) |
immich |
20 | 15% (3/20) | 15% (3/20) | Tie at 15%: 17 of 20 DTOs use functional mixin inheritance (extends createZodDto(...)). |
| Group A Total: Basic Code & Call Graph | Both | 130 | 78% (102/130) | 38% (49/130) | On immich alone (66 GT): ce 73% (48/66) vs CBM 50% (33/66). |
| Group B Total: Queues, DB Tables & SQL Lineage | Both | 241 | 97.5% (235/241) | Not modeled (0/241) | BullMQ producers/consumers, TypeORM/Kysely tables & columns, SQL read/write lineage, sub-projects. |
2. Cold indexing speed, disk size, and memory.
CodeExplorer was faster and produced a 5.1× smaller SQLite database on immich (3,205 files). On twenty (24,527 files), CBM completed cold indexing ~1.5× faster with lower peak RAM because it extracts only basic AST and call edges, whereas ce also runs 250 framework, ORM, and queue analyzers.
| Metric | Repo | CodeExplorer |
CBM |
Winner & Notes |
|---|---|---|---|---|
| Cold Index Wall Time | immich (3.2k files) |
17.60 s | 20.60 s | ce (1.17× faster) |
| Cold Index Wall Time | twenty (24.5k files) |
147.33 s (Layer 1 to 5 pipeline) | 96.90 s (Layer 1 to 3 pipeline) | CBM (1.52× faster) |
| SQLite Database Size on Disk | immich / twenty |
55.1 MB / 784.0 MB | 282.0 MB / 842.5 MB | ce (5.1× smaller on immich; 7% smaller on twenty) |
| Peak Working Set (RAM) | immich / twenty |
780.5 MB / 4,660.6 MB | 595.0 MB / 2,076.0 MB | CBM (1.3× to 2.2× lower peak RAM) |
| Parse Warnings | immich / twenty |
0 warnings / 0 warnings | 4 partial / 209 partial + 1 unusable | ce (0 parse failures) |
3. MCP server latency & tool trade-offs.
Single-shot JSON-RPC round-trip measurements over persistent stdio sessions (results/mcp-benchmark.json).
| MCP Operation | CodeExplorer (immich / twenty) |
CBM (immich / twenty) |
Winner & Notes |
|---|---|---|---|
Architecture Overview (get_architecture_map / get_architecture) |
267.7 ms (4.4 KB) / 371.7 ms (11.0 KB) | 117.5 ms (3.4 KB) / 286.9 ms (6.1 KB) | CBM faster; ce lists all 21 / 46 sub-projects and DBs. |
| Graph Schema / Taxonomy (1st / Cached) | 22.3 ms (1.9 ms cached) / 23.2 ms (2.7 ms cached) | 35.6 ms (40.9 ms 2nd) / 72.0 ms (75.3 ms 2nd) | ce 1.6× to 3.1× faster (21× to 28× cached) with full property schema (51.5 to 54.7 KB vs 1.0 KB). |
Read-Only Cypher Query (LIMIT 10) |
4.3 ms (1.0 KB) / 4.7 ms (1.1 KB) | 38.1 ms (0.3 KB) / 71.2 ms (0.3 KB) | ce 8.9× to 15× faster on single-shot Cypher execution. |
Inbound Callers by Method Name (validate) |
76.5 ms (1.5 KB) / 530.1 ms (2.5 KB) | 7.5 ms (18.6 KB ambiguous) / 6.5 ms (45.6 KB ambiguous) | ce resolves callers in 1 turn; CBM returns 62 / 67 ambiguous candidates requiring a 2nd call. |
Symbol Lookup by Name (AuthService) |
80.8 ms (0.4 KB) / 367.3 ms (1.1 KB) | 55.6 ms (1.1 KB) / 97.5 ms (0.8 KB) | CBM 1.5× to 3.8× faster on symbol lookup by name. |
File Outline (auth.service.ts / auth.resolver.ts) |
13.4 ms (0.4 KB) / 92.7 ms (0.9 KB) | 17.3 ms (4.8 KB) / 40.5 ms (7.8 KB) | ce 1.3× faster on immich; CBM 2.3× faster on twenty (ce payload 8× to 12× smaller). |
| Unique Capabilities | VS Code extension, 32+ CI gates (ce gate), data lineage, test impact |
Snapshot diffing (compare_graphs), OpenTelemetry overlay (ingest_traces), ADRs (manage_adr) |
Complementary tool surfaces beyond core graph queries. |
4. Reproducibility: tested queries & ground-truth specification.
Click any drawer below to expand verbatim queriesGroup A Queries (C1, C2, C6, C7) scripts/evaluate_ce_vs_cbm.py: Exact OpenCypher Queries Executed on Both Tools (130 Basic Code Items)
# C1: Incoming HTTP / GraphQL / FastAPI Endpoints # CodeExplorer (ce): MATCH (e:Endpoint)-[:EXPOSED_BY]->(f:Function) RETURN f.file_path, f.start_line, f.name, e.http_method, e.route_template # codebase-memory-mcp (CBM): MATCH (m)-[:HANDLES]->(r:Route) RETURN m.file_path, m.start_line, m.name, r.name # ------------------------------------------------------------------------------ # C2: Outbound Inter-Service & External HTTP Calls # CodeExplorer (ce): MATCH (ext:ExternalService)-[r:CALLED_BY]->(f:Function) WHERE COALESCE(ext.protocol, 'http') <> 'graphql' RETURN f.file_path, COALESCE(r.line, f.start_line), f.name, COALESCE(ext.url, ext.name); MATCH (f:Function)-[r:CALLS_ENDPOINT]->(ep:Endpoint) WHERE COALESCE(ep.protocol, 'REST') <> 'GraphQL' RETURN f.file_path, COALESCE(r.line, f.start_line), f.name, COALESCE(ep.route_template, ep.name); # codebase-memory-mcp (CBM): MATCH (a)-[:HTTP_CALLS]->(b) RETURN a.file_path, a.start_line, a.name, b.name # ------------------------------------------------------------------------------ # C6: Call Graph (Direct & DI Service Method Callers) # CodeExplorer (ce): MATCH (caller:Function)-[:CALLS]->(callee:Function) WHERE callee.name IN [<target_methods>] OPTIONAL MATCH (enclosing:Function)-[:DECLARES]->(caller) RETURN caller.file_path, COALESCE(enclosing.start_line, caller.start_line), COALESCE(enclosing.name, caller.name), callee.name # codebase-memory-mcp (CBM): MATCH (caller)-[:CALLS]->(callee) WHERE callee.name = '<target_method>' RETURN caller.file_path, caller.start_line, caller.name, callee.name # ------------------------------------------------------------------------------ # C7: Class Inheritance & Interface Implementation # CodeExplorer (ce): MATCH (c:Type)-[r:INHERITS_FROM|IMPLEMENTS]->(p) RETURN c.file_path, c.start_line, c.name, type(r), p.name # codebase-memory-mcp (CBM): MATCH (c)-[r:INHERITS|IMPLEMENTS]->(p) RETURN c.file_path, c.start_line, c.name, type(r), p.name
Group B Queries (C3 to C5, C8 to C11) scripts/evaluate_ce_vs_cbm.py: Queue, ORM Table, Data Lineage & 6 Architectural Queries (241 Items)
# C3 & C8: Message Queue Producers & Consumers (BullMQ / @OnJob) MATCH (t:Topic)-[:PUBLISHED_BY]->(f:Function) RETURN f.file_path, f.start_line, f.name, t.name; MATCH (t:Topic)-[:SUBSCRIBED_BY]->(m:Function) RETURN m.file_path, m.start_line, m.name, t.name; # C4 & C10: Database Tables, Entities & Schema Members MATCH (t:Table) OPTIONAL MATCH (t)-[:IMPLEMENTED_BY]->(ent:Type) RETURN COALESCE(ent.file_path, t.path), ent.start_line, COALESCE(ent.name, t.name), t.name; MATCH (t:Table)-[:IMPLEMENTED_BY]->(ent:Type)-[:HAS_MEMBER]->(m:Member) RETURN ent.file_path, m.start_line, m.name, t.name; # C5: Code-to-Table Read/Write Lineage (TypeORM & Kysely) MATCH (q:Query)-[r:WRITES_TO|READS_FROM]->(t:Table) OPTIONAL MATCH (q)-[:ACCESSED_BY]->(f:Function) RETURN COALESCE(f.file_path, q.file_path), q.start_line, COALESCE(f.name, q.name), type(r), t.name; # C9: Monorepo Services & Sub-Projects MATCH (p:Project) RETURN p.path, labels(p), p.name; # C11: 6 End-to-End Architectural Queries (Single-Query Expressibility) # Q1 — Topics with producers but no consumers (or vice versa): MATCH (t:Topic) WHERE NOT (t)-[:SUBSCRIBED_BY]->() OR NOT (t)-[:PUBLISHED_BY]->() RETURN t.name # Q2 — Pairs of projects sharing a database: MATCH (p1:Project)-[:USES_DB]->(d:Database)<-[:USES_DB]-(p2:Project) WHERE p1.name < p2.name RETURN p1.name, p2.name, d.name # Q3 — Top written database tables: MATCH (q:Query)-[:WRITES_TO]->(t:Table) RETURN t.name, count(q) AS writes ORDER BY writes DESC LIMIT 10 # Q4 — Endpoints transitively writing to a database table within 1..4 call hops: MATCH (e:Endpoint)-[:EXPOSED_BY]->(f:Function)-[:CALLS*1..4]->(w:Function)<-[:DECLARED_IN]-(q:Query)-[:WRITES_TO]->(t:Table) RETURN e.name, t.name LIMIT 20 # Q5 — Per-project endpoint and published topic counts: MATCH (p:Project) OPTIONAL MATCH (p)-[:EXPOSES]->(e:Endpoint) OPTIONAL MATCH (p)-[:PUBLISHES_TO]->(tp:Topic) RETURN p.name, count(DISTINCT e), count(DISTINCT tp) # Q6 — End-to-end chain: Endpoint -> Call Chain (0..3 hops) -> Queue Publish -> Queue Subscriber: MATCH (e:Endpoint)-[:EXPOSED_BY]->(f:Function)-[:CALLS*0..3]->(pub:Function)<-[:PUBLISHED_BY]-(t:Topic)-[:SUBSCRIBED_BY]->(sub) RETURN e.name, t.name, sub.name LIMIT 20
Ground-Truth Spec groundtruth/patterns.md: Deterministic Ground-Truth Sampling Rules (seed = 42, 371 Items)
All patterns were verified against real source files in `twentyhq/twenty` (`12e8fae471865e9dab60c0b0fd59f318505c804e`) and `immich-app/immich` (`2153d22f34349de690cd98890ee4c635808da638`). Test files (`*.spec.ts`, `*.test.ts`, `__tests__`, `test/`, `e2e/`, `fixtures/`, `test_*.py`) are excluded across all categories. Stratified random sampling uses `random.Random(42)`. - C1 (Incoming HTTP / GraphQL / FastAPI Endpoints): NestJS `@Controller` + `@(Get|Post|Put|Patch|Delete)`, NestJS GraphQL `@Resolver` + `@(Query|Mutation|Subscription)`, and Python FastAPI `@(app|router).(get|post|put|delete|patch)`. - C2 (Outbound Inter-Service / External HTTP Calls): `fetch(...)`, `axios.(get|post|put|delete|patch|request)`, `this.httpService.(get|post|...)` (excluding IMAP `imapClient.fetch`). - C3 & C8 (Message Queues & Async Jobs): `twenty` `enum MessageQueue` (27 queues), `@InjectMessageQueue` producers, `@Processor` + `@Process` consumers; `immich` `enum QueueName` (19 queues) & `enum JobName`, `jobRepository.queue` producers, `@OnJob` consumers. - C4 & C10 (Database Tables & Schema): `twenty` TypeORM `@Entity` + `@Column`/`@ManyToOne`/`@OneToMany`; `immich` `@Table` + `@Column`/`@ForeignKeyColumn`. - C5 (Code-to-Table Lineage): `twenty` `@InjectRepository` / `@InjectWorkspaceScopedRepository` read (`find*`, `count`, `createQueryBuilder`) & write (`save`, `insert`, `update`, `delete`, `softDelete`, `upsert`); `immich` Kysely `selectFrom` (read) & `insertInto`/`updateTable`/`deleteFrom` (write). - C6 (Call Graph): Deterministic sample of 10 service methods across `*.service.ts`, tracing non-test callers invoking `this.<serviceField>.<methodName>(` or `this.<methodName>(`. - C7 (Inheritance & Implementation): Top 5 base classes/interfaces with ≥3 non-test classes matching `export (abstract )?class <Child> (extends|implements) <Parent>`.