Compliance auditing with AI orchestration

Process study · proposed method

Reuse the evidence. Preserve the questions.

Explore a review process that shares source extraction, divides independent evidence work and makes conflicts visible before specialist assessment.

Establish the baseline

Selected framework and scope
Approved policies and evidence
Control owners and review period

Evidence to retain

Reusable source and extraction index
Mappings with conflicts and rationale
Specialist review and open questions

Why work this way?

The process, and what it could improve.

Proposed method · no measured savings claim

Cost

Index evidence once, reuse the source trail.

Give reviewers a shared source index and scoped control assignments. Reuse extraction results when the same record supports several review questions.

What to compare: Extraction/model charges and reviewer effort per accepted evidence mapping, including corrections and stale-source refreshes.
The tradeoff: Reusing an extraction without its date and source can spread a single mistake across the package.

Compare the same scope and acceptance criteria. Include setup, coordination, failed attempts and human review alongside the successful model call.

Design the work

One problem. A coordinated agent team.

Proposed task plan

A model reasons. A harness runs its agent session and tool loop. Orchard coordinates the work, context and handoffs across them.

Orchard orchestratorScopes work · delegates · synthesizes evidence

CONTROL

Implementation lead

Example session

Model familyClaudeHarnessClaude Code

Map selected controls to documents, exports, owners and the assessment period.

Bounded task assigned

Counterproposal agent

Example session

Model familyGPTHarnessCodex CLI

Try to disprove each draft finding using stale evidence, conflicting records and missing coverage.

Opposing assumption assigned

Independent reviewer

Example session

Model familyGeminiHarnessGemini CLI

Inspect source links and distinguish supported observations from unresolved interpretations.

Evidence criteria assigned

Shared project contract

Rules · skills · hooks · MCP configuration

Adapted to each harness

01 / 04 · Dispatch

Split the work. Preserve the question.

One objective, distinct responsibilities

Turn an assessment scope into evidence-linked findings that a human assessor can challenge.

Read-only evidence access within the approved system boundary; no automated certification.

Implementation leadcontrol-evidence matrix + source index

Counterproposal agentcounter-evidence register + freshness checks

Independent reviewerreviewed findings + remediation-owner map

Illustrative execution planAssignment Evidence Review

Model families and pairings are illustrative. Assignments here are manual examples, not measured model comparisons or automatic cost optimization.

Trellis / shared knowledge

Assessment boundary

Selected controls, system scope, owners and review period.

Evidence history

Source metadata, freshness checks and reviewer dispositions.

BEAM / coordination

Bounded tasks, ownership notices, findings and handoffs between sessions. A queued message and a processed receipt are distinct states.

Recall → work → retain decisions

Tools / execution

Approved read-only exportsEvidence index + document toolsIssue tracker / review workflow

Access follows the configured harness, credentials and approved scope.

Inspect the example work orderJSON

An illustrative work-order format for this scenario. Changing an example session updates this document; no agents run from this page.

{
  "example": "compliance-auditing",
  "objective": "Turn an assessment scope into evidence-linked findings that a human assessor can challenge.",
  "boundary": "Read-only evidence access within the approved system boundary; no automated certification.",
  "sessions": [
    {
      "role": "proposer",
      "model_family": "Claude",
      "harness": "Claude Code",
      "task": "Map selected controls to documents, exports, owners and the assessment period.",
      "expected_artifact": "control-evidence matrix + source index"
    },
    {
      "role": "challenger",
      "model_family": "GPT",
      "harness": "Codex CLI",
      "task": "Try to disprove each draft finding using stale evidence, conflicting records and missing coverage.",
      "expected_artifact": "counter-evidence register + freshness checks"
    },
    {
      "role": "reviewer",
      "model_family": "Gemini",
      "harness": "Gemini CLI",
      "task": "Inspect source links and distinguish supported observations from unresolved interpretations.",
      "expected_artifact": "reviewed findings + remediation-owner map"
    }
  ],
  "project_contract": [
    "Rules",
    "Skills",
    "Hooks",
    "Tool configuration"
  ],
  "shared_context": [
    "Assessment boundary",
    "Evidence history"
  ],
  "evidence_required": [
    {
      "label": "Every finding has provenance",
      "evidence": "Follow the source link, collection time and exact assessment scope."
    },
    {
      "label": "Uncertainty stays visible",
      "evidence": "Separate supported observations, contradictions and requests for more evidence."
    },
    {
      "label": "A person owns disposition",
      "evidence": "Record reviewer decisions and remediation owners without inferring certification."
    }
  ],
  "release_owner_gate": "The designated assessor decides whether the evidence is sufficient and signs the assessment record."
}

The process principle

Make review easier to repeat and harder to misread.

Scope of this exampleThis supports audit preparation. It is not a certification, legal opinion, or compliance guarantee; qualified reviewers make that determination.

An audit-preparation scenario with sample evidence, not an assessment of an actual organization. NIST assessment reference

Compliance auditing

Different tools.
Shared project context.

Where the work changes.
How to judge the difference.

Outlook listener

Modernize an Android app

Modernize a COBOL system

Legacy OS to the cloud