Mintlify has eliminated traditional RAG (Retrieval-Augmented Generation) from their AI documentation assistant, replacing it with ChromaFs, a virtual filesystem that delivers 100-millisecond boot times compared to 46 seconds for sandbox approaches. The solution intercepts Unix commands and translates them into database queries, achieving a 460x performance improvement while reducing marginal costs to nearly zero.
Traditional RAG Failed to Handle Documentation Exploration Patterns
Mintlify identified three critical weaknesses in RAG-based approaches: inability to retrieve answers spanning multiple pages, failure to surface exact syntax not in top-K results, and poor support for browsing-based exploration patterns. Traditional vector search excelled at keyword matching but struggled when users needed to navigate documentation hierarchies or explore related concepts across multiple pages.
ChromaFs Creates Filesystem Illusion Over Database Queries
Instead of deploying actual sandboxes, Mintlify built ChromaFs as a virtual filesystem layer that intercepts Unix commands (grep, cat, ls, find, cd) and translates them into Chroma database queries. Built on just-bash, a TypeScript reimplementation of bash by Vercel Labs, ChromaFs implements a pluggable IFileSystem interface that handles parsing and flag logic while storing the complete file tree as compressed JSON in the Chroma collection.
Key technical optimizations include:
- Bootstrapping: Directory tree loads into in-memory Set/Map structures, enabling instant ls, cd, and find operations without network calls
- Access Control: Path metadata includes isPublic and groups fields, with the system pruning unauthorized slugs before initialization
- Chunk Reassembly: When agents request files, ChromaFs fetches all relevant chunks, sorts by index, and joins them into complete pages
- Grep Optimization: Coarse filtering via Chroma queries, bulk-prefetch of matching chunks into Redis, and fine-grained in-memory regex matching
Performance Metrics Show Dramatic Infrastructure Savings
The performance comparison reveals substantial improvements across all metrics. Boot time dropped from 46 seconds (P90) for sandboxes to 100 milliseconds for ChromaFs. Marginal cost per conversation fell from $0.0137 to effectively zero, as ChromaFs reuses existing database infrastructure. At Mintlify's scale of 850,000 monthly conversations, sandboxes would cost approximately $70,000 annually compared to negligible per-conversation compute costs with ChromaFs.
The approach enables AI agents to explore documentation using familiar Unix patterns while maintaining security boundaries through metadata-based access control. The virtual filesystem abstraction provides agents with natural navigation capabilities without the overhead of actual file system operations or containerized environments.
Key Takeaways
- ChromaFs delivers 460x faster boot times (100ms vs 46 seconds) by replacing sandboxes with virtual filesystem queries
- Mintlify saves approximately $70,000 annually by eliminating $0.0137 per-conversation marginal costs across 850,000 monthly conversations
- The virtual filesystem intercepts Unix commands (grep, cat, ls, find, cd) and translates them to database queries without deploying actual sandboxes
- Access control is enforced through path metadata with isPublic and groups fields, pruning unauthorized content before agent initialization
- ChromaFs enables multi-page answer retrieval and documentation exploration patterns that traditional RAG approaches cannot handle