File-Based Orchestration for AI-Assisted Knowledge Production Preprint
Abstract
AI-assisted knowledge production at book length requires coordination across hundreds of disjoint large language model (LLM) sessions, yet current systems either assume persistent runtime state or address only single-session interactions. Multi-agent frameworks assign roles and coordinate agents within a single execution context. Commercial AI writing tools assist with generation but provide no orchestration, quality gating, or provenance infrastructure. Workflow engines manage compute pipelines but not knowledge artifacts requiring editorial judgment.
We describe a file-based orchestration architecture that enables context-window-spanning AI-assisted knowledge production at book length, combining quality gates, lane separation, promotion contracts, and deterministic session rehydration into an integrated system deployed over 233 production runs on a single book-length project. The architecture uses human-readable text files (markdown, YAML, TSV) as its sole state surface. Each new LLM session reads persistent state files to reconstruct working context — a process we term baton rehydration — eliminating the need for persistent databases or API-level memory. Seven quality gates (G0–G6) govern candidate progression through the publication pipeline. Eight defined roles with an explicit management-labor split preserve human authority as an architectural property. Six parallel orchestration threads operate with formal promotion boundaries governing cross-lane artifact movement.
We report descriptive metrics from a single 12-month deployment that produced a 57,969-word manuscript through two major structural revisions, supported by 42 version snapshots and 15 feedback cycles. A bounded case study of one structural transition — from a 10-chapter linear manuscript to a 16-chapter recursive architecture — demonstrates the system's capacity for governed architectural change. This is a single-deployment study; we do not present controlled comparisons or test generalizability.
Claims
- A file-based state surface is an effective architectural response to the ephemeral context window problem
- Software engineering patterns (quality gates, lane separation, promotion contracts) adapt to govern knowledge production
- Human authority can be embedded as an architectural property rather than a usage convention
- Successfully deployed over 233 runs producing a 57,969-word manuscript
Non-Claims
- Does not claim generalizability beyond this single deployment
- Does not claim superiority over alternative approaches
- Does not claim novelty of individual components — the contribution is the specific integration