Table of Contents

The Problem Nobody Wants to Talk About
Part 1: How California Criminal Law Actually Works
Part 2: How a Case Actually Reaches a DA’s Desk
Part 3: The Base Model — SaulLM-7B-Instruct-v1
Part 4: deepagents — The Autonomous Tool-Calling Layer
Part 5: Self-Harness — Teaching Virtual DA to Improve Itself
Part 6: Putting It Together — The Virtual DA Application
Part 7: Limitations and What Virtual DA Is Not
Building This Yourself

Educational Disclaimer Virtual DA is a research and educational demonstration project. Nothing in this post or in the Virtual DA application constitutes legal advice, legal opinion, or a substitute for consultation with a licensed attorney. The system’s output has not been reviewed by a lawyer and must not be used to make real charging decisions, legal filings, or prosecutorial recommendations. Criminal law is complex, fact-specific, and jurisdiction-dependent. If you or someone you know is involved in a criminal matter, consult a qualified attorney immediately.

The Problem Nobody Wants to Talk About

Large language models are confidently wrong about the law.

Ask any general-purpose LLM to assess a robbery case under California Penal Code § 211. It will give you an answer — fluent, authoritative, formatted with section headers and bullet points. It will cite cases. It will reference sub-sections. Half of it will be fabricated.

This is not a temperature problem or a prompt engineering problem. It is a structural problem. LLMs are trained to predict plausible next tokens. Legal citations and statute text are exactly the kind of authoritative-sounding, specific, structured text that a language model will confidently hallucinate because that pattern is everywhere in its training data — legal blogs, court documents, law review articles — but the exact current text of a live statute is not reliably memorized, especially when it was amended last session.

We built Virtual DA to solve this for a narrow, well-defined scope: California criminal law. The approach is not to give the model better training data or to cross our fingers and hope. The approach is to make it structurally impossible for the model to invent statute text — by fetching the real text from official .gov sources before the model ever writes a word of analysis. Then we layered deepagents for autonomous legal reasoning and the self-harness loop so the system gets smarter from its own traces.

This post is a deep technical walkthrough of how we built it, what we learned, and what the architecture looks like under the hood.

The Virtual DA intake form — paste any incident report and click Analyze

The interface is deliberately austere: paste the incident report, click Analyze. Everything else is the pipeline’s job.

Part 1: How California Criminal Law Actually Works

Before we built anything, we needed to understand the domain deeply enough to know what we were building a system for. This section is that grounding — and it is more relevant to the architecture than it might seem.

The Classification Ladder

California criminal law divides offenses into three tiers, with meaningful consequences for how a DA actually approaches a case.

Infractions are the lowest tier. No jail time. A fine. Traffic violations, minor code violations, jaywalking. A DA is rarely involved in charging an infraction — that is handled administratively by citations. Infractions do not result in a criminal record in the traditional sense. They are not relevant to Virtual DA.

Misdemeanors are crimes punishable by up to one year in county jail, a fine, or both. California Penal Code § 17 defines a misdemeanor as any crime punishable by imprisonment in county jail not exceeding one year. Common examples: petty theft (PC § 484/488), simple assault (PC § 240), driving under the influence (Vehicle Code § 23152), trespassing (PC § 602). A misdemeanor filing by a DA triggers arraignment, a right to jury trial, and a criminal record. The charging decision — whether to file, plead down, or decline — is exactly the kind of work Virtual DA models.

Felonies are the most serious tier: crimes punishable by state prison time (more than one year) or death. California Penal Code § 17 again: a felony is any crime punishable by death or imprisonment in state prison. Robbery (PC § 211), assault with a deadly weapon (PC § 245), burglary (PC § 459), rape, murder. A felony filing carries consequences that reshape a person’s life: loss of voting rights during sentence, firearm prohibition, professional licensing bars.

The “Wobbler” — California’s Hybrid Category

Between misdemeanors and felonies lies a uniquely California concept: the wobbler. A wobbler is an offense that can be charged as either a felony or a misdemeanor at the DA’s discretion, based on the facts, the defendant’s record, and the interests of justice.

Classic wobblers: grand theft (PC § 487), assault with a deadly weapon (PC § 245), elder abuse (PC § 368), commercial burglary. A defendant charged with a felony wobbler can petition the court to reduce it to a misdemeanor at sentencing (PC § 17(b)) or after probation completion.

The wobbler creates a genuine analytical task — exactly the kind of judgment call where a DA must weigh actus reus, mens rea, victim impact, criminal history, and statutory language together. It is also why a mechanical lookup tool is not enough; you need reasoning over retrieved statute text.

Enhancement Allegations

Separate from the base charge, California allows the prosecution to allege enhancements that increase the sentence if proven. The most significant for street crime:

PC § 12022.5 / 12022.53 — personal use or discharge of a firearm (adds 3–25 years to life)
PC § 186.22 — gang enhancement (adds 2–15 years or triggers indeterminate terms)
PC § 667 / 1170.12 — prior serious or violent felony (Three Strikes law)

An enhancement is not a separate charge — it is alleged in conjunction with an underlying offense and requires separate proof. For our sample robbery case (a defendant who displayed a knife), the firearm enhancement is not triggered, but a DA must affirmatively rule it out. Virtual DA’s critique section explicitly addresses what enhancements are and are not supported by the evidence.

Felony Levels Within State Prison

California uses a determinate sentencing structure for most felonies, where the legislature specifies a low, middle, and high term. The judge selects among these based on aggravating and mitigating factors.

For robbery under PC § 211:

Simple robbery: 2, 3, or 5 years (low / mid / high)
Robbery in concert in an inhabited dwelling: 3, 6, or 9 years

The DA’s charging document (complaint or information) triggers which range applies. Overcharging — filing a count with an unreachable high term — can backfire with juries who see it as prosecutorial overreach. Undercharging leaves punishment on the table. This analytical tension is something Virtual DA explicitly surfaces in its evidentiary gap analysis.

Part 2: How a Case Actually Reaches a DA’s Desk

The criminal justice pipeline from incident to charging decision is worth understanding, because it defines what input Virtual DA processes and what output it is approximating.

1. The Incident and the Police Report

A crime is reported — by a victim, a witness, or through direct police observation. The responding officer writes an incident report: narrative description of what happened, who was present, what statements were made, what physical evidence was observed. This is the raw material.

The incident report is not evidence in the legal sense. It is a police officer’s written account. It contains hearsay. It contains the officer’s characterizations. A good defense attorney will attack it. But for a DA performing an initial charging assessment, it is the primary document.

Virtual DA’s input is a transcript or incident report. The sample case we use throughout development is a robbery at a convenience store: a defendant who selected merchandise, displayed a folding knife when confronted by the clerk, made an implicit threat (“Don’t make this a problem”), and left without paying. The victim’s statement, the defendant’s own post-Miranda statement, and the arresting officer’s observations are all in the input.

2. Evidence Assembly

Between arrest and charging, law enforcement assembles evidence: physical evidence (the recovered merchandise, the knife), surveillance footage, 911 recordings, written witness statements. A DA filing within 48 hours of arrest — the legal deadline for felony arraignment — often works from incomplete evidence. They make charging decisions under time pressure with what they have.

Evidence strength is a central concern of Virtual DA’s Phase 3 critique. The system explicitly calls out gaps: what evidence would strengthen the case, what the defense will argue, what the jury instruction will require the prosecution to prove.

3. The DA Desk Review — Charging and Filing

A deputy DA reviews the package. Their primary question: Is there sufficient evidence to support each element of each charged offense, such that a reasonable jury could find the defendant guilty beyond a reasonable doubt? This is the filing standard — not guilt, not probable cause, but a reasonable probability of conviction.

The output of a DA desk review is a charging decision:

File — proceed with the charge(s) as presented
File reduced — charge a lesser offense (e.g., attempted robbery instead of robbery; misdemeanor instead of felony wobbler)
Decline — insufficient evidence; REJECT
Investigate further — send back for additional investigation

These are the four verdicts Virtual DA outputs. The mapping is not random — it is derived from the actual analysis of elements, evidence, and evidentiary gaps that a deputy DA performs.

Part 3: The Base Model — SaulLM-7B-Instruct-v1

The inference backbone of Virtual DA is SaulLM-7B-Instruct-v1, a 7-billion-parameter language model specifically trained for legal reasoning. Understanding why we chose it — and what it is — matters for understanding the system’s capabilities and limits.

What SaulLM Is

SaulLM-7B was developed by researchers at CentraleSupélec and released in early 2024. It is built on the Mistral-7B architecture — a transformer with grouped-query attention, sliding window attention, and a byte-pair-encoded vocabulary. The “Saul” in SaulLM is a reference to the character Saul Goodman from Better Call Saul — a deliberate nod to its legal focus.

The model was trained in two stages:

Continued pre-training on a massive legal corpus: the Pile of Law dataset (~256 GB of legal text) plus court opinions, statutes, law review articles, bar exam materials, and legal contracts. This gives the model a legal vocabulary and fluency with legal reasoning patterns that general-purpose models lack.
Instruction fine-tuning to follow natural language legal instructions: analyze this contract, identify the elements of this offense, assess prosecutorial merit given these facts.

At 7B parameters it is small by frontier model standards — small enough to run comfortably on a single A100 80GB GPU at roughly 5–10 tokens/second with SGLang’s optimizations. This is what makes RunPod deployment practical at ~$1.49/hr.

Fluent, Authoritative, and Wrong

The honest answer is: we tried them. They are better writers. They produce more fluent legal prose. And they are wrong in ways that look exactly right — which is the most dangerous failure mode in law.

A frontier model will write you a paragraph about California robbery doctrine that reads like a Westlaw excerpt, cites People v. Williams with a plausible-sounding reporter citation, and gets the mental state requirement subtly backwards. The prose is impeccable. The law is wrong. A junior attorney reviewing it under time pressure will miss it. A defendant’s future depends on that paragraph. This is not a theoretical concern; it is the documented failure mode of GPT-4 in legal benchmarks, and it gets worse — not better — as models become more fluent.

SaulLM fails differently. Trained on actual legal corpus rather than the internet’s description of legal corpus, it hedges where it is uncertain. It says “depending on how the court interprets the use of force element” rather than authoritatively stating the wrong rule. Detectable uncertainty is better than confident error in any high-stakes domain. You can catch a hedge. You cannot catch a hallucination that reads as fact.

There is also a structural reason that has nothing to do with model quality: incident reports contain victim home addresses, witness names, defendant immigration status, juvenile priors. Every token you send to an external API is logged, stored, and subject to that company’s data handling policies, their breach risk, and their jurisdictional exposure. For any deployment that touches real case files, local inference is not a preference — it is the only defensible architecture.

Finally, GPT-4 and Claude give you output. SGLang gives you a runtime. With XGrammar running locally, we can clamp Phase 1 output to an exact JSON schema at the token logit level — not “please output JSON” as a prompt instruction, but a context-free grammar that makes schema-invalid tokens impossible to sample. That is a capability that does not exist in any commercial API, and it is what makes Phase 1 structurally reliable rather than statistically likely to be correct.

SGLang: The Inference Runtime

We run SaulLM through SGLang, a high-performance inference framework developed at UC Berkeley. Three SGLang features matter for Virtual DA:

XGrammar enforces structured output at the token sampling level. During Phase 1 fact extraction, we provide the model with a JSON schema (our ExtractedFacts Pydantic model). XGrammar converts this schema into a context-free grammar and masks logits at each decoding step to ensure only tokens that extend a valid partial JSON parse are allowed. The result: Phase 1 output is always valid JSON that conforms to the schema. No post-processing, no retry loops, no truncated output. This is structurally different from asking a model to “output JSON please” — it is tokenizer-level enforcement.

RadixAttention is a KV cache sharing mechanism. In Virtual DA, the incident transcript is processed in Phase 1 and a large portion of the KV cache (the transformer’s internal representation of the input) is identical between Phase 1 and Phase 3. RadixAttention identifies this shared prefix and reuses the cached computation. Phase 3 — which would otherwise be expensive because the full transcript must be re-attended over — becomes much cheaper. In practice, Phase 3 latency is dominated by generation, not prefill.

Chunked prefill (enabled with --chunked-prefill-size 8192) handles long context documents by processing the prefill in chunks, keeping memory usage predictable even for long incident reports.

Part 4: deepagents — The Autonomous Tool-Calling Layer

Phase 3 of Virtual DA uses deepagents, a Python framework for building autonomous tool-calling agents with LangChain-compatible LLMs. This is where the system goes beyond a simple prompt-and-response pattern.

What deepagents Is

deepagents provides an Agent abstraction that wraps a language model with a set of tools and a decision loop. The agent can:

Decide whether to call a tool or respond directly
Call the tool, observe the result
Incorporate the tool result into its context
Continue reasoning until it has what it needs to produce a final answer

This is the ReAct pattern (Reason + Act) at its core: the agent reasons about what it knows, acts by calling a tool, reasons again with the new information, and so on until it can respond.

Why Virtual DA’s Phase 3 Needs an Agent — Not Just a Prompt

After Phase 2, Virtual DA has the statute text for the primary charged offense. But the statute text for a charge like robbery (PC § 211) frequently cross-references other sections: PC § 212 (fear defined), PC § 212.5 (degrees), PC § 213 (punishment), PC § 215 (carjacking distinction). A complete prosecution assessment needs all of these.

A naive pipeline would pre-fetch all cross-referenced sections before Phase 3. But pre-fetching everything creates a different problem: the model’s context window fills with statute text it may not need, increasing latency and diluting the signal. deepagents solves this elegantly — statute retrieval is a tool that the model calls when it encounters a reference it actually needs to resolve:

Agent: I need to assess whether the defendant's conduct meets the "fear" element of
       robbery. PC § 211 requires that property be taken by force or fear. I should
       retrieve PC § 212 to understand how fear is legally defined.

[calls statute_search_tool("California Penal Code 212")]

Tool returns: "§ 212. The fear mentioned in § 211 may be either:
              1. The fear of an unlawful injury to the person or property
                 of the person robbed..."

Agent: The statutory definition of fear includes fear of unlawful injury to person.
       The victim's statement ("I feared for my safety") directly satisfies this.
       Proceeding with charge assessment...

This is not the model inventing a definition of fear — it is the model fetching the actual statutory definition and reasoning from it. The distinction is everything.

The Tool Interface in Practice

The statute search tool wraps the same retrieval stack used in Phase 2 (SerpAPI → DuckDuckGo fallback → BeautifulSoup parsing), exposed as a LangChain Tool:

statute_search_tool = Tool(
    name="statute_search",
    description=(
        "Search for the exact text of a California statute by section number "
        "or keyword. Returns the full text from official .gov sources. "
        "Use when you need a specific statute's definition or elements."
    ),
    func=search_statute,
)

The model decides when to call it, what query to use, and how to incorporate the result. Our portal-page detector — which filters out legislature navigation chrome — runs inside search_statute. The agent never sees garbage output, and it never has to reason about whether what it received is real statute text or a navigation menu.

Part 5: Self-Harness — Teaching Virtual DA to Improve Itself

This is the part of Virtual DA that makes it genuinely novel as a research system. Self-harness learning is our implementation of the methodology described in arXiv:2606.09498 — Self-Harness: Harnesses That Improve Themselves, by Zhang et al. at Shanghai AI Laboratory, published June 2026.

What the Paper Actually Says

Most writing about “self-improving AI” is vague about what is actually being improved. The paper is precise: it is not improving model weights, not fine-tuning, not updating training data. It is improving the harness — the non-parametric scaffolding that surrounds a fixed language model and governs how it operates. The harness includes system prompts, tool definitions, memory mechanisms, verification rules, runtime control logic, and failure-recovery procedures.

The paper’s central claim is that a fixed model, operating under its current harness, can identify its own failure patterns and propose targeted edits to the harness that governs its future behavior — without any human engineering effort and without a stronger external model guiding it.

This matters because different models fail differently. A harness that works well for GPT-4 may leave Qwen3.5 spinning in unproductive tool-use loops. The paper’s thesis is that model-specific harness design should not require human experts to study each model’s failure modes manually. The model can do this itself.

The paper validates Self-Harness on Terminal-Bench-2.0, a multi-turn agentic benchmark with 64 containerized terminal tasks. Results: held-out pass rates improve from 40.5% → 61.9% for MiniMax M2.5, 23.8% → 38.1% for Qwen3.5-35B-A3B, and 42.9% → 57.1% for GLM-5 — substantial gains without touching a single model weight. Crucially, the Qwen3.5 experiments were run locally on four H200 GPUs using SGLang — the same inference runtime we use for SaulLM in Virtual DA.

One more detail that is directly relevant: the paper’s initial harness is built on the DeepAgent SDK — the same deepagents framework that powers Phase 3 of Virtual DA. This is not a coincidence. The paper explicitly studies how Self-Harness can evolve a DeepAgents-based harness; we apply that methodology to evolve a DeepAgents-based legal critique agent.

The Three-Stage Loop

Think of it as a feedback flywheel. Each turn of the wheel, the agent gets a little sharper — not by updating its weights, but by refining the instructions it operates under. Here is how one revolution works:

Stage 1: WEAKNESS MINING
  Run agent on real tasks → Cluster failure traces into patterns

Stage 2: HARNESS PROPOSAL
  Same model proposes K targeted edits to its own instructions

Stage 3: PROPOSAL VALIDATION
  Regression-test each candidate
  ✅ passes both splits → Merge into harness v+1
  ❌ regresses any split → Log and discard

The flywheel only advances when a proposal survives the regression gate. No vibes, no self-grading. Evidence in, tested harness out.

Stage 1 — Weakness Mining: Reading Your Own Failures

The loop starts by simply running the agent and recording what happens — every message, every tool call, every intermediate result, and whether the task ultimately succeeded or failed. This part sounds easy. The insight is in what you do with the failures next.

A naive system would see ten failures and try to patch ten things. But ten failures might be one underlying problem wearing ten masks. The agent timed out on task A, produced a missing artifact on task B, gave a wrong answer on task C — all three might stem from the same root behavior: the agent keeps exploring when it should commit to producing an answer.

The paper solves this by building failure signatures — structured tuples that fingerprint each failed trace along three axes:

Axis	What it captures
Terminal cause	What the verifier ultimately rejected (timeout, missing file, wrong answer)
Agent behavior	What the agent did that caused that rejection
Reusable mechanism	The abstract behavioral pattern — the explorable archetype

Two failures with matching signatures get grouped together. The output isn’t a list of ten broken things — it’s a ranked evidence bundle of two or three recurring mechanisms, ordered by how fixable they are. The proposer receives causes, not symptoms.

Stage 2 — Harness Proposal: The Model Edits Its Own Instructions

Here is the twist that makes Self-Harness genuinely interesting: the model that proposes harness improvements is the same fixed model that failed. No stronger external oracle, no human annotator. The same SaulLM-7B that produced the bad output is handed its own failure patterns and asked: what instruction, if added to your operating harness, would have prevented this?

This is counterintuitive. Why would a model that failed know how to fix itself? Because failure diagnosis and failure causation are different cognitive tasks. A model that got lost in an exploration loop might not know it is stuck in the moment — but presented with a structured analysis of five traces where it got stuck the same way, it can often articulate a rule that would have broken the loop.

The proposal stage generates K candidate edits — purposely diverse, purposely minimal:

Diversity enforced  →  across candidates  (different target, different surface, different hypothesis)
Minimality enforced →  within each candidate  (touch only what you need to fix; don't rewrite the world)

Each proposal ships with an audit record: which failure pattern it targets, which harness surface changes, what behavioral effect is expected, what could go wrong. No black boxes. Every proposed edit is a diff you can read.

Stage 3 — Proposal Validation: Earning the Merge

A candidate harness edit does not get accepted because the model is confident about it. It earns the merge by surviving an empirical gate.

Each candidate is tested against two splits:

Held-in split  →  the failures that motivated this proposal  (did it actually fix the problem?)
Held-out split →  tasks the proposer never saw              (did it break anything else?)

The acceptance rule has no exceptions: the candidate must improve at least one split without degrading the other. A proposal that fixes three held-in cases but quietly breaks two held-out cases gets rejected. Even if the net gain is positive. Even if the model argues it is worth the trade-off.

This strictness is the point. Soft acceptance gates let the model optimize for its own rubric — which it will inevitably game. A hard regression test cannot be sweet-talked.

✅ MERGE  →  ΔHeld-in ≥ 0  AND  ΔHeld-out ≥ 0  AND  max(Δin, Δout) ﹥ 0
❌ DISCARD →  anything else  (logged, never applied)

Accepted edits are merged into the next harness version and the loop repeats. Rejected edits are logged for forensics but do not touch the active harness. The entire lineage is auditable: harness v0, v1, v2 — what changed each time, why, and what the test results were.

How Virtual DA Adapts This

The paper studies Terminal-Bench-2.0 terminal tasks. Virtual DA applies the same loop to a legal AI pipeline. The mapping is direct:

The harness in Virtual DA is the SKILL.md files. Each skill directory — skills/FactExtractor/, skills/StatuteSearch/, skills/LegalCritique/ — contains a SKILL.md that describes how that phase should behave: what tools to use, what output format to produce, what validation steps to perform, what failure modes to avoid. These are the non-parametric instructions that surround SaulLM. Improving the harness means improving these SKILL.md files.

Weakness Mining from legal pipeline traces. Every call to VirtualDA.analyze() appends a JSON trace to traces/. The trace records the full pipeline execution: extracted facts, statute retrieval queries and results, deepagents tool calls and reasoning, final critique. The harness reads recent traces and identifies failure patterns across them — not individual failures, but recurring mechanisms across multiple runs.

Harness Proposal by SaulLM (or DeepSeek in dev mode). In production on RunPod, SaulLM-7B acts as its own proposer — the same model that runs the pipeline reads its own failure traces and proposes SKILL.md updates. In local development with INFERENCE_MODE=deepseek, the DeepSeek API fills this role. This is a deliberate design choice: in dev mode, you want faster, cheaper proposal generation without requiring a GPU pod. The harness being improved — the SKILL.md files — is model-agnostic. A DeepSeek-proposed improvement to StatuteSearch SKILL.md is valid regardless of which model runs inference in production.

Proposal Validation via pytest. Instead of pass/fail on Terminal-Bench tasks, Virtual DA runs its 11-test pytest suite as the regression gate. This is our equivalent of the held-in + held-out split. The suite tests the full pipeline: schema validity for Phase 1, retrieval correctness for Phase 2, pipeline completion for Phase 3. A proposed SKILL.md change is accepted only if all 11 tests pass. A change that makes the critique more fluent but breaks statute retrieval is automatically rejected and the previous SKILL.md is restored from backup.

def _validate_proposal(self, skill_name: str, new_skill_md: str) -> bool:
    skill_path = self.repo_root / "skills" / skill_name / "SKILL.md"
    backup = skill_path.read_text()
    skill_path.write_text(new_skill_md)

    proc = subprocess.run(
        [sys.executable, "-m", "pytest", "tests/", "-x", "-q", "--tb=short"],
        capture_output=True, text=True, timeout=120,
        cwd=str(self.repo_root)
    )
    passed = proc.returncode == 0
    if not passed:
        skill_path.write_text(backup)
    return passed

Why pytest instead of a model-graded rubric? The paper uses deterministic verifiers for exactly this reason: a model evaluating its own proposals will construct proposals that score well on its own rubric, which is a soft target. A pytest suite that tests pipeline behavior is a hard target — it cannot be gamed by clever prompt engineering. This is the same logic the paper applies: “the acceptance rule is strict” because soft acceptance gates produce overfit changes that hurt held-out performance.

A Real Example: The Portal Page Bug, Discovered by the Loop

Early production traces showed a recurring failure mechanism: Phase 2 statute retrieval was returning California Legislature navigation chrome — “Quick Search, Bill Number, Bill Keyword, Sitemap” — instead of actual statute text. The deepagents loop in Phase 3 then attempted to reason about site navigation as if it were the California Penal Code.

The failure signature was consistent across multiple traces: same terminal cause (critique contained nonsense), same agent mechanism (statute text was navigation HTML, not statutory content), same causal status (Phase 2 retrieval returned portal content unchecked). The clustering identified this as a single failure pattern, not random noise.

The harness proposal targeted StatuteSearch SKILL.md — specifically, the instruction that said “fetch the page at the URL and extract the main content” with no validation step. The proposed update added: verify content is statute text before returning. Indicators of a portal page include: ‘Quick Search’, ‘Bill Keyword’, ‘Skip to Content’, ‘Sitemap’. If two or more are present, discard and try an alternative source.

This passed the pytest suite. It was applied. We then independently implemented the same logic as _is_portal_page() in code. The SKILL.md change governed agent-level behavior; the code change enforced it structurally. Both were discovered through trace mining. Neither was anticipated by the developers.

Part 6: Putting It Together — The Virtual DA Application

With all the pieces defined, the full application looks like this.

The Three-Phase Pipeline

Phase 1 — Structured Fact Extraction (SaulLM-7B + XGrammar)

The user submits an incident report. That is the only input Virtual DA ever receives — a plain text document, the same kind a responding officer writes. No checkboxes, no structured form fields, no pre-parsed metadata.

SaulLM reads the entire document and must extract six structured fields in a single pass: the actus reus (the physical acts committed), the mens rea (the mental state), the list of suspected charges, a chronological timeline, the witnesses, and an evidence summary. The catch: SaulLM is generating free text. Without enforcement, it would output whatever tokens it finds most probable — sometimes JSON, sometimes prose, sometimes a hallucinated charge name.

XGrammar prevents this entirely. Before decoding begins, we compile our ExtractedFacts Pydantic schema into a context-free grammar. At every decoding step, XGrammar masks any token that would produce an invalid partial JSON string. The model cannot emit a token that breaks the schema — not because we validate after the fact, but because those tokens are masked to zero probability during sampling.

The result is guaranteed-valid JSON every time:

{
  "actus_reus": "Defendant took a bottle of whiskey by displaying a folding knife and implicitly threatening the clerk",
  "mens_rea": "Defendant intended to permanently deprive the store of property; displayed knife to prevent interference",
  "charges": ["robbery", "petty theft with a prior conviction"],
  "timeline": ["9:10 PM: Defendant enters store", "9:10 PM: Defendant takes bottle without paying"],
  "witnesses": ["Victor Oduya (store clerk)", "Officer T. Nguyen (arresting officer)"],
  "evidence_summary": "Recovered bottle, arresting officer testimony, victim statement, post-Miranda defendant statement"
}

This schema is the spine of the entire pipeline. The charges array drives Phase 2 — every entry triggers a statute lookup. The actus_reus and mens_rea fields become the evidentiary facts Phase 3 must map to statutory elements. If Phase 1 misclassifies a charge, everything downstream carries that error. XGrammar makes Phase 1 structurally reliable so we can trust its output unconditionally.

Phase 1 complete — structured facts extracted from the incident report

Phase 1 output: actus reus, mens rea, suspected charges (amber pills), chronological timeline, and witnesses — all extracted from the raw incident report in a single model pass.

Phase 2 — Statute Retrieval (no LLM at all)

This is the phase most people underestimate. There is no language model here. Not even a lightweight one. Phase 2 is a deterministic retrieval pipeline, and that is intentional.

For each charge in extracted_facts.charges, the pipeline:

Constructs a targeted query: "California Penal Code robbery site:leginfo.legislature.ca.gov"
Sends to SerpAPI (DuckDuckGo as fallback if no key is configured)
Fetches the raw HTML of the top result
Parses with BeautifulSoup to extract the main content
Runs _is_portal_page() — the self-harness-discovered portal filter that rejects California Legislature navigation chrome
Stores the cleaned statute text, keyed by charge name

The text that enters Phase 3 came from leginfo.legislature.ca.gov — the California Legislature’s official publication system. It was not generated, paraphrased, or summarized. It is the current statutory text, fetched at runtime. This is the architectural guarantee: the model never writes statute text, it only reasons about statute text it retrieved from an authoritative source.

Phase 2 — statute text retrieved from official .gov sources for each charge

Phase 2 output: one card per charge, each showing the fetched statute text with its source URL. No LLM involved — this is deterministic web retrieval.

Phase 3 — DA Critique (deepagents + SaulLM-7B + RadixAttention)

The deepagents loop receives the full ExtractedFacts and all retrieved statute texts. The agent reasons, calls statute_search for any cross-referenced sections it needs, and produces:

Charge Assessment — element-by-element analysis against facts and statute
Evidentiary Gaps — what evidence would strengthen or is missing
Defense Analysis — what the defense will argue
Prosecution Recommendation — PROCEED / DECLINE / INVESTIGATE FURTHER

Streaming to the UI in Real Time

The backend exposes a Server-Sent Events endpoint at POST /analyze/stream. As each phase completes, an event fires to the frontend:

data: {"phase": "extracting_facts"}
data: {"phase": "facts_extracted",   "data": {...ExtractedFacts...}}
data: {"phase": "retrieving_statutes"}
data: {"phase": "statute_retrieved", "data": {"charge": "robbery", "text": "..."}}
data: {"phase": "running_critique"}
data: {"phase": "complete",          "data": {"da_critique": "...", "latency_ms": {...}}}

The Next.js frontend progressively reveals each phase as it arrives. The user sees facts appear, then statutes load one by one, then the full DA assessment streams in — the same sequence a deputy DA follows mentally when reviewing a case file.

Performance on the Sample Case

Running on a RunPod A100 80 GB pod (~$1.49/hr):

Phase	What Happens	Typical Time
Phase 1	SaulLM extracts structured facts from ~800-word transcript	12–18s
Phase 2	Statute retrieval for 2 charges	8–15s
Phase 3	deepagents critique with 1–2 tool calls	75–100s

Phase 3 dominates because SaulLM generates a ~600–800 word structured legal memo. RadixAttention reduces prefill cost but generation is the bottleneck at 7B parameters.

Complete analysis — all three phases finished, verdict banner and latency breakdown visible

End-to-end result: structured facts (Phase 1), statute cards (Phase 2), DA assessment with verdict banner and per-phase latency breakdown (Phase 3).

Part 7: Limitations and What Virtual DA Is Not

Building this system taught us where the approach works and where it does not.

What works well:

Statute retrieval is genuinely hallucination-free for well-known California Penal Code sections
XGrammar makes Phase 1 structurally reliable — 100% schema validity
SaulLM’s legal training produces noticeably better element-by-element charge assessment than general-purpose models of similar size
The self-harness loop discovered and fixed the portal page bug without human intervention

Where it struggles:

Obscure or recently amended statutes: SerpAPI may return a cached pre-amendment version
Multi-defendant cases: the current schema has one defendant; aiding-and-abetting analysis is limited
4th Amendment issues: the system has no mechanism to raise suppression of illegally obtained evidence
Sentencing enhancements: Prop 57, SB 81, and current Three Strikes (post-Prop 36, 2024) are modeled shallowly

What it is not:

It is not a legal information service for defendants or victims
It is not a tool for practicing lawyers
It is not evidence that AI can replace DAs
It is not a production system

Building This Yourself

Everything is open source at gitlab.com/sugix/virtual-da.

What you need to procure:

Item	Where	Cost
DeepSeek API key	platform.deepseek.com	Free tier available
SerpAPI key	serpapi.com	Free: 100 searches/month
RunPod account + A100 pod	runpod.io	~$1.49/hr
SSH key pair	`ssh-keygen -t ed25519` on your laptop	Free

The bootstrap script (runpod_setup.sh) handles everything on the pod — model download, SGLang startup, API server, frontend build — all in tmux sessions that survive SSH disconnects. The SSH tunnel brings port 8000 to your localhost for demo access.

Conclusion

Virtual DA is an existence proof for a narrow claim: you can build a legal analysis pipeline with zero statute hallucination if you enforce a strict separation between what the language model generates and what external sources provide. The model never writes a statute. It only analyzes statutes it fetched.

The self-harness loop turns the system into something that improves over time without human annotation — running virtual-da harness --validate mines accumulated traces, proposes SKILL.md improvements, validates them against a hard behavioral test suite, and applies only what passes. The portal-page fix was not written by a human; it was proposed by the model after observing its own failures.

deepagents makes Phase 3 genuinely useful: the agent resolves cross-referenced statute sections in real time, producing analysis grounded in complete statutory context rather than just the surface charge section.

None of this constitutes legal advice. All of it constitutes an interesting research direction.

Educational Disclaimer (repeated for clarity) Virtual DA is for research and educational purposes only. It is not legal advice. It is not a substitute for a licensed attorney. Nothing in this system should be used to make real prosecutorial, legal, or charging decisions. Criminal matters require qualified legal counsel. The authors are not lawyers.

Built with SaulLM-7B-Instruct-v1, SGLang, deepagents, FastAPI, and Next.js 14. Self-harness loop based on arXiv:2606.09498. California criminal law only.

Self-Harness Related Posts