Meta Builds an AI Agent That Cuts Compliance Review from Days to Minutes
Meta engineers built a domain expert AI that separates knowledge from reasoning and learns from expert corrections without any model retraining.
- Meta built an internal AI agent that acts as a secondary domain expert for compliance work, cutting assessments from days to minutes.
- Architecture separates knowledge (markdown files with YAML metadata) from reasoning (composable procedural recipes) for clean failure attribution.
- Roughly 200 files organized as positions, taxonomies, routing indexes, and gateways form a bidirectional dependency graph.
- Progressive disclosure via recipe-driven stages cut tokens per turn by around 80% versus a flat instruction file.
- Self-improvement loop diagnoses expert corrections, compiles minimal edits, and validates with blind replay plus regression tests, no retraining.
- Full writeup from Engineering at Meta details the architecture and results.
Meta’s engineering team has described an internal AI agent that captures the institutional knowledge and decision procedures used by senior compliance specialists. According to the engineering post, the system combines structured source files, explicit reasoning procedures, deterministic routing, and a feedback pipeline that converts accepted expert corrections into regression-tested updates. The model’s weights remain unchanged.
The agent addresses a recurring problem inside large organizations: similar compliance questions appear across hundreds of product reviews, assessments require days of research, and different reviewers may reach inconsistent conclusions. General-purpose language models lack the organization’s prior decisions, terminology, risk thresholds, and treatment of edge cases. Meta’s design makes that context inspectable and version-controlled.
Give knowledge an address
The architecture separates domain knowledge from reasoning procedures. Its knowledge base contains roughly 200 Markdown files organized as a curated directory tree. YAML frontmatter gives each file machine-readable metadata, including triggers, dependencies, and consumers.
- Position files record authoritative organizational decisions, constraints, and routing implications.
- Taxonomy and vocabulary files define shared terms for experts, developers, and the agent.
- Routing indexes and gateway files determine which material applies to a request.
Standard retrieval-augmented generation often selects passages through semantic similarity. Meta’s explicit routing rules reduce ambiguity by declaring when a file should be loaded and how it relates to other material.
Reasoning lives in composable procedures that Meta calls recipes. Each recipe specifies the analysis steps, the knowledge required at each stage, and the conditions for a complete answer. Developers can add an organizational position by creating a knowledge file and updating its routing index. They can correct a methodological flaw by editing a recipe. This separation also narrows debugging: failures can usually be traced to missing knowledge, incorrect routing, or a faulty procedure.
Progressive disclosure keeps the working context small by loading only the files required for the current stage. Meta reports that moving from one flat instruction file to staged recipes reduced token consumption per turn by about 80%. The design draws on ideas from Andrej Karpathy’s LLM Wiki and Google’s Open Knowledge Format, which encourage teams to extract and structure knowledge before inference.
Corrections become tested patches
Each accepted expert correction passes through a four-stage maintenance pipeline that resembles compilation:
- Diagnose: Determine whether the correct answer was reachable from the available source material. Reachable answers indicate a reasoning or recipe defect; unreachable answers indicate a knowledge gap. Conflicting expert judgments are escalated for human resolution.
- Compile: Generate the smallest necessary file changes. Parallel sub-agents inspect cross-references, conflicts, duplication, and token budgets.
- Validate: Replay the failed scenario without exposing the expected correction, then run the broader domain regression suite.
- Land: Submit the change as a reviewed pull request and add the original failure to the regression suite.
Two safeguards check proposed updates before deployment. An adversarial review agent starts with a fresh context and sees only the proposed diffs, limiting its exposure to the assumptions that produced them. A deterministic linter catches dangling references, identifier collisions, and dependency cycles.
Early results, with boundaries
After six weeks of development, Meta reported substantial gains across review speed, context use, and maintenance:
| Measure | Reported result |
|---|---|
| Tokens consumed per turn | Reduced by about 80% after adopting staged recipes and progressive disclosure |
| Assessment time | Reduced from days to minutes |
| Expert usefulness ratings | Outputs were rated useful in almost all reviews |
| Maintenance cadence | Validated edits replaced work that had previously taken full engineering sprints |
| Observed regressions | None across the reported improvement cycles |
These figures are Meta’s internal results from an early deployment. The zero-regression claim applies to the company’s reported evaluation suite and improvement cycles, so teams implementing the pattern would need benchmarks that reflect their own policies, failure modes, and risk thresholds.
What developers would need to build
Teams adopting this architecture would need more than a document repository. The operational system includes several connected components:
- A version-controlled knowledge tree with stable identifiers and ownership rules
- Metadata schemas for triggers, dependencies, consumers, and routing
- Recipes that define repeatable analysis procedures and completion criteria
- A deterministic router that selects relevant files
- A correction pipeline that proposes minimal edits
- Targeted replays and a representative regression suite
- Automated linting, adversarial review, and human approval
The text-based design keeps changes visible to domain experts. Reviewers can inspect a policy or procedural update as a small diff, while Git records its author, rationale, approval, and test history. Model retraining is unnecessary for these updates.
Best fit: auditable judgment
Meta designed the pattern for fields where decisions must remain consistent, reviewable, and grounded in institutional policy. Candidate domains include regulatory compliance, protocol adherence, financial risk assessment, security review, engineering standards, and procurement evaluation. They share high volumes of similar questions, specialist rules, and organization-specific edge cases.
The approach depends on experts who can express their knowledge as written positions and repeatable procedures. Human checkpoints remain central, so the system accelerates specialist review while preserving expert responsibility. Domains dominated by tacit physical skill, unresolved judgment, or rapidly changing ground truth may resist this structure.
Meta has not released the implementation as open source. Interested teams would need to build the routing, compilation, review, and evaluation pipeline themselves, using the published architecture as a reference.
Comments
