Case study · Infrastructure investing · AI agent

See the contractor's claim before it reaches your quarterly valuation.

You're an asset manager at an infrastructure fund that owns part of a company building a gas plant. Shield reads the project's letters and reports for events that could cost the fund money. It shows each contract deadline and drafts the formal notice. You approve it, or not.

Try the demo The fund, project company, project and contractor are fictional. The page runs in your browser, with no server and no AI model call.

6 daysleft to send notice about rock found on site (demo event EV-1)

$6.63Mof fund equity at risk on that event, on fictional inputs

3 of 4events found in the test set, with 0 false alarms

0 of 7invented citations shown, in a test built to produce them

For: An infrastructure fund that owns part of a project company
Users: You, the asset manager, and your fund partner, who decides what goes into each valuation
Data: Fictional: 8 test documents, 15 in the demo
Built with: Python agent with approval rules, static demo page

TechniquesAIAgent pipeline with a human approval gateAIEvent extraction with a confidence thresholdData engContract deadline rules: clause plus amendmentsData engEquity at risk above contingencyData engTamper-evident audit log (hash chain)EvalWord-for-word citation checkEval17 evals: precision and recall on a labelled setEvalAdversarial test: invented citations

01 · VISION

No drop in your stake's value from a claim you could have seen.

The signs of a claim sit in letters, meeting minutes and daily site reports weeks before the formal claim. The vision: every document read, every notice deadline counted from the clause and its amendments, and every cost shown as your fund's share, with a named person approving every notice.

02 · THE PROBLEM

Five things go wrong today.

1 · You hear about a claim late

Its signs sit in letters, minutes and daily site reports nobody reads in full.

2 · Every event costs a manual search

Linking a letter to its site record, schedule entry and clause is done by hand.

3 · A missed deadline loses the right to claim

Each notice deadline depends on the clause and its amendments, tracked from memory or a spreadsheet.

4 · Your partner learns at quarter end

Event costs are stated for the whole project. You think in terms of the fund's own share.

5 · A notice can go out with no owner

Drafts move by email, and who approved what is unclear.

03 · THE SOLUTION

The agent reads. Code counts the days. You approve.

Every document is scanned for trigger phrases. Each hit gets a clause and a confidence score. Below 0.6, it stays hidden. Documents from another project are refused.
Every quote is checked. It must appear word for word in its source, or the event is marked down.
Every deadline is counted. Event date plus the clause's notice period. If an amendment is missing, the date is marked provisional.
You see your fund's share. The cost above the remaining contingency, times the fund's share, is fund equity at risk.
You approve the draft notice. Only a person on the approver list can approve. Every decision is logged. Nothing is sent.

04 · HYPOTHESES AND IMPACT

One job, one bet, one number each.

Your job	Hypothesis	Metric and target, then result	Expected impact
When a document arrives, I want events that could cost money flagged, so I hear before the formal claim.	If every letter, minute and daily report is checked against the contract's trigger phrases, you learn of an event sooner, because the signs are in the documents first.	Days from the first document to you knowing: down. Baseline unobserved. Guardrail precision 0.85 or more: 1.00 on 8 documents	Earlier action on events (unobserved)
When an event is flagged, I want the trail from letter to clause checked, so I review instead of search.	If each event arrives with checked links from letter to clause, review time per event goes down, because you check links instead of searching for them.	Minutes to review one event: down. Baseline unobserved. 3 of 3 citations shown verified	Less review time per event (unobserved)
When an event starts a clock, I want the deadline from the clause and its amendments, so notice goes out in time.	If deadlines come from the clause and all its amendments, fewer deadlines pass without a notice, because errors like 28 days vs an amended 21 are caught.	Deadlines passed without a notice, per project per year: down. Baseline unobserved. 3 of 3 deadlines correct	Fewer rights to claim lost
When I find an event, I want its cost as the fund's share, so my partner hears before the valuation.	If you see fund equity at risk when an event is found, fewer drops in value reach your partner as a surprise, because they hear before the valuation date.	Share of event-driven drops in value flagged before quarter end: up. Baseline unobserved	Fewer surprise write-downs (unobserved)
When a notice is drafted, I want a named person to approve it, so the fund can show who decided.	If only people on the approver list can approve and every decision is logged, no notice leaves without an owner, because the system can't send and refuses anyone else.	Notices sent without a logged approver: down to zero. 3 of 3 stopped at PENDING_APPROVAL	Every notice has an owner

Results from the 2026-10-01 test run: 17 of 17 agent tests and 10 of 10 demo tests pass. Must not get worse: precision, the share of alerts that are real, stays at 0.85 or more. No asset manager has used it.

05 · THE VALUE

What it's worth, and what it asks of you.

In the demo, the rock found on site puts $19.58M at stake for the project and $6.63M of the fund's own money at risk. The notice window closes in 6 days.

Illustrative, on fictional inputs. All 3 demo events total $9.34M, 3.9% of the stake's value.

You, the asset manager

You get: "I hear about a claim or a notice deadline weeks before the quarter ends, with the documents, and the project company sends notice on time."
Will you believe it?: Medium. Every deadline and quote can be checked. The tool found 3 of 4 events in the test set.
How fast: Low delay. It reads documents the project company already holds. Time to first alert is unmeasured.
What it takes: Medium. Read access to the project company's document and schedule files. Each draft needs your review.
What you give up: Medium. The project company's managers may see it as the fund checking their work. A list of known risks can be handed to the other side in a legal dispute.

Your fund partner

They get: "No drop in our stake's value from a claim we could have seen. Each cut traces back to a document."
Will they believe it?: Low. That earlier notices lead to fewer surprise drops in value is a hypothesis. There's no data across the fund's projects.
How fast: High delay. The effect on valuations shows over several quarters.
What it takes: Medium. Document access depends on the fund's right to information in each project.
What they give up: Medium. Some goodwill with the project company's managers. Lawyers must review the list of risks.

The weak link is your partner's belief and the delay. The cheapest fix: run it on the archive of one finished project, count the drops in stake value caused by claims, and check whether the documents showed each one earlier. That gives evidence in weeks, not quarters.

06 · LIMITATIONS

What this doesn't prove yet.

Data: fictional. 8 test documents. The demo shows 15 documents, 2 of them added for the demo.
Users: no asset manager or partner has used it. Every "before" is assumed.
Detection: keyword rules, not an AI model. Accuracy on real letters is unmeasured.
Awareness date: the agent uses the document date. Contracts often start the clock when a party "became aware, or should have", which can be earlier.
Value: assumes every open dispute is lost, and that the owners' money covers overruns above contingency. That overstates the expected loss.
Who approves: the demo makes you the approver. In practice the project company sends notices, and the fund acts through its rights as a shareholder.
Source mismatch: EV-1 in the demo uses the walkthrough record (deadline amended to 21 days). The agent's test version of the same event uses 28 days.

07 · SCOPE AND NEXT STEPS

What it won't do, and what comes next.

Non-goals

Valuing claims. Delay days, daily cost and direct cost are fictional placeholders.
Reporting. No export to the valuation model or the report to the fund's investors.
All projects. One contract, one project. No rules per contract form.
Claim strategy and legal advice. The agent refuses strategy questions.
Sending notices, reading scanned pages, links to document and schedule systems.

A backtest on one finished project's archive, against a list of events marked by an expert.
Read-only, with the project company's lawyers owning the list of risks, so a dispute can't turn it against the fund.
An AI-model extractor in place of keyword rules, to find events like the missed LTR-006.
Two reviewers, 10 events, with and without the trail, to time the review.
Publish the code once real company names are checked and replaced.

RESOURCES

Go deeper.

Try the demo, step by step three steps, in order

Pick an event the tool found. EV-1 has 6 days left before the project company's notice deadline.
Follow the document trail: letter, site record, schedule entry, contract clause, deadline countdown. Below it, you read how much of your fund's own money is at risk.
Read the draft notice, then approve or reject it. Your decision goes into the log. Switch "Acting as" to the analyst to see the tool refuse someone who isn't allowed to approve.

Sample data only. The fund, project company, project and contractor are fictional. "Today" is fixed at 26 Aug 2026. The agent's outputs were computed beforehand and saved into the page.

Main learnings test run, evidence checks, deadlines, approval controls

Test run (2026-10-01, 8 fictional documents): 17 of 17 tests passed. Precision 1.00 (3 of 3 alerts real), recall 0.75 (3 of 4 events found, missed LTR-006), 2 of 2 non-events correctly ignored. Supports 1, with a gap in recall.
Evidence: 3 of 3 shown citations verified. A test setup built to invent citations produced 7 fake ones; 0 were shown. Supports 2.
Deadlines: 3 of 3 match the expected dates, incl. 1 flagged provisional for a missing amendment. Supports 3.
Approval controls: 1 document from another project refused. 3 of 3 notices stopped at PENDING_APPROVAL. A person not on the approver list was refused. 21 audit log entries, tamper check (hash chain) intact. Supports 5.
Demo tests: 10 of 10 pass (node --test demo/tests/calc.test.js). They cover deadlines, amendments, status cut-offs, matching the agent's 3 deadlines, 15 trail quotes and value at risk.
Demo figures: EV-1 shows $19.58M at stake for the project and $6.63M of fund equity at risk. All 3 events total $9.34M, 3.9% of the stake's value. Inputs are fictional, so these show how the calculation works, not a result. Relates to 4.

Hypotheses in full before and after, and the test for each

1 · DETECTION

Before -> after: event found when the formal claim arrives or in the quarterly report -> flagged on the date of the document that shows it
Hypothesis: If every letter, minute and daily report is checked against the contract's trigger phrases, then the asset manager learns of an event sooner, because the signs are in the documents before the formal claim.
Metric: Days from the first document to the asset manager knowing. Down. Baseline unobserved. Guardrail: precision (share of alerts that are real) at or above 0.85 (test: 1.00 on 8 documents).
Test: Run it on one finished project's archive and compare with a list of events marked by an expert.

2 · EVIDENCE

Before -> after: trail put together by hand -> each link's quote checked against its source
Hypothesis: If each event arrives with checked links from letter to clause, then review time per event goes down, because the reviewer checks links instead of searching for them.
Metric: Minutes to review one event; share of trails needing correction. Down. Baseline unobserved.
Test: Two reviewers, 10 events, with and without the trail.

3 · DEADLINE

Before -> after: periods tracked from memory or a spreadsheet -> computed from the clause and its amendments, provisional if one is missing
Hypothesis: If deadlines come from the clause and all its amendments, then fewer deadlines pass without a notice, because errors like 28 days vs an amended 21 are caught.
Metric: Deadlines passed without a notice, per project per year. Down. Baseline unobserved.
Test: Recompute deadlines on a finished project and compare with the notices actually sent.

4 · EQUITY VIEW

Before -> after: whole-project dollars, seen in the quarterly report -> fund's own money at risk per event, inputs on the page
Hypothesis: If the asset manager sees fund equity at risk when an event is found, then fewer drops in value reach the partner as a surprise, because the partner hears before the valuation date.
Metric: Share of event-driven drops in value flagged before quarter end. Up. Baseline unobserved.
Test: Go through the last 4 quarters of one project. For each write-down, was there an earlier document trail?

5 · CONTROL

Before -> after: drafts by email, approver unclear -> approver list, logged decision, no send function
Hypothesis: If only people on the approver list can approve and every decision is logged, then no notice leaves without an owner, because the system can't send and refuses anyone else.
Metric: Notices sent without a logged approver. Down to zero. Test: 3 of 3 notices stopped at PENDING_APPROVAL.
Test: Approval rule tests, plus attempts to approve as someone not on the list.

All sub-problems the 8 found, what's tackled, and where in the code

"Commercial event" here means something in a document, like a late site handover, that can start a notice deadline or a claim.

Detection. Signs of a claim sit in letters, meeting minutes and daily site reports that nobody reads in full.
Evidence. Linking a letter to its site record, schedule entry and clause is done by hand.
Deadline. Each notice deadline depends on the clause and its amendments. Missing one can cost the right to claim.
Equity view. Event costs are stated for the whole project. The fund thinks in terms of its own share.
Control. A draft notice must not go out without a named person's approval.
Claim value. Working out what a claim is worth needs a construction cost expert.
Reporting. Events need to reach the quarterly valuation and the report to the fund's own investors.
All projects. Each project has its own contract form, amendments and document systems.

This build tackles 1 to 5. 1, 2 and 4 are partial.

#	Status	Proof
1	Partial	Agent: `agent/llm.py` `DummyExtractor` matches keywords. Events scored below 0.6 stay hidden. Recall 0.75 (share of real events found)
2	Partial	Agent: `nodes.verify_citations` checks the letter quote. Demo: the trail in `demo/data.js` is built by hand, then checked by `calc.js` `verifyQuote`
3	Tackled	Agent: `nodes.compute_deadlines` marks a date provisional when an amendment is missing. Demo: `calc.js` `deadline` uses the changed period when the amendment is on file
4	Partial	Demo only: `calc.js` `equityAtRisk`. Not in the agent. Inputs fictional
5	Tackled	Agent: `governance.approve_notice`, can't send anything. Demo: approve / reject step and decision log

How it's built read in, find, check, deadline, value, draft, approval

Read in: documents arrive. Documents from another project are refused and logged.
Find: each document is scanned for trigger phrases. Each hit gets a clause and a confidence score. Below 0.6, it stays hidden.
Check: each quote must appear word for word in its source document, or the event is marked down.
Deadline: the event date plus the clause's notice period. If an amendment is missing, the date is marked provisional.
Value (demo): days of delay beyond the schedule's spare days (float), times the cost per day, plus direct costs. The part above the remaining contingency, times the fund's share, is fund equity at risk.
Draft: a notice that cites each source, with status PENDING_APPROVAL.
Approval: only a person on the approver list can approve. Every approval, rejection and refusal is logged. Nothing is sent.

How success is tracked early signals and later results

Metric	Type	Tests hypothesis	How measured
Days from the first document to the asset manager knowing	Early signal	1	Document date vs first view in the event log
Precision and recall vs an expert's list	Early signal	1	Backtest on a finished project's archive
Share of trails corrected by the reviewer	Early signal	2	Edits logged per trail
Computed deadline vs date the notice was sent	Early signal	3	Backtest comparison
Deadlines passed without a notice, per project per year	Late result	3	Event list vs notices sent
Event-driven drops in value flagged before quarter end	Late result	4	Quarterly valuation changes vs event log dates
Notices sent without a logged approver	Late result	5	Decision log vs letters sent

Scope in scope and out of scope

In scope	Non-goal (incl. untackled sub-problems)
Letters, minutes, daily reports, schedule entries, contract clauses (fictional)	6 Claim value: the tool doesn't value claims. Delay days, daily cost and direct cost are fictional placeholders
Keyword matching with a confidence threshold	7 Reporting: no export to the valuation model or the report to the fund's investors
Word-for-word quote check on every link	8 All projects: one contract, one project. No rules per contract form
Deadline per clause, using amendments, provisional flag	1 partial: no AI-model extractor. Keyword rules missed LTR-006
Fund equity at risk with stated inputs (demo)	2 partial: links beyond the first letter quote are built by hand, not found by the agent
Draft notice, approval step, decision log	4 partial: value logic lives in the demo, not the agent
	Claim strategy and legal advice. The agent refuses strategy questions
	Sending notices, OCR (reading scanned pages), links to document and schedule systems

Trade-offs what was chosen, over what, and the cost

Chose	Over	Gain	Cost
Deadline rules in code	AI model computes dates	Same answer every time, testable	Each contract form needs set-up
Exact quote match	Accept model output	No evidence without a source	Reworded quotes and scanning errors are dropped
Precision first	Recall first	Fewer false alarms to review	Missed LTR-006
No send function	Auto-send	A named owner per notice	One manual step before the deadline
Static page, outputs computed beforehand	Live agent behind an API	$0, works offline	Visitors can't upload documents
"All disputes lost" value	Probability-weighted value	Simple, every input visible	Overstates expected loss

Terms on this page plain definitions

Infrastructure fund: a fund that invests in roads, power plants and similar projects.
Project company (ProjectCo): the company that owns the project and signs the building contract.
Contractor (EPC): the firm hired to design, buy for and build the project.
Claim: a formal request for extra money or time under the contract.
Notice deadline (time bar): the last day to send a written notice. Miss it and the right to claim can be lost.
Clause, amendment: a numbered rule in the contract, and a later change to it.
Contingency: money set aside in the budget for overruns.
Fund equity at risk: the part of a loss that comes out of the fund's own money.

Product requirements the product PRD

Written for an earlier persona: the owner's project team, not an infrastructure fund. Real company and system names are replaced, per the name audit.

Shield

Stage: Solution Review · Last Updated: 2026-08-26 (v2, post sub-agent review) · Owner: [PM candidate] · Status: Draft (case study) Product family: built on the agent platform, with its evidence-pack, case-routing and procurement modules.

Hypothesis

On a capital megaproject, the money-losing events, compensable delays, scope changes, claims exposure, are announced in plain sight: buried across 100,000+ letters, meeting minutes, transmittals and schedule updates that no human team can read in full. Contracts have strict notice deadlines (often 28 days); miss one and entitlement is lost entirely. Today ~33% of project capex ends up disputed; the average regional dispute runs ~$91M and takes a year.

If we deploy an agent that continuously reads the contract against the live project record, detects incoming contractor claims against the owner, evaluates the contractor's position, finds procedural weaknesses, and drafts the owner's defense response, then owners will protect $30–80M per project in commercial leakage, because the losses are caused by reading capacity and deadline discipline, machine problems, not by commercial judgment, which stays with humans.

Supporting evidence: - HKA CRUX (2,200+ projects): disputed costs average 33% of capex, $95B cumulative - Arcadis: #1 dispute cause for a decade is "failure to properly administer the contract", a process failure software has stored but never performed - Target region: 89% of projects over cost, 87% late

Strategic Fit

Wedge product for the agent platform. Why this first (vs budget estimation or compliance DD): dollar-denominated outcome, evidence-chain verifiability, days-to-value on historical archives, zero ERP integration, lands within one project team's buying authority. Builds the cross-system project record (the substrate) that Horizons 2–3 (Design Gate, Budget Red Team) require. Full sizing: outputs/analyses/impact-sizing-shield-2026-08-26.md (expected ~$50M/yr protected per project vs ~$2M license; worst case still clears a 10–15X ROI bar).

Alternatives considered: - FID budget estimation first, rejected: needs an outcome database that doesn't exist yet; value provable only at the next gate, months away - Schedule risk first, deferred to fast-follow: strong but value is "questions asked earlier," not dollars - Contractor-side, future expansion after trust is built with owner-side deployments; EPCs buy faster, but owners hold strategic budget and the data

Non-Goals (v1)

No live ERP or scheduling tool write integration, read-only document and file ingestion (document control system exports, scheduling tool exports, PDFs). Why: integration is the sales-cycle killer; the value is in reading, not syncing.
No autonomous outbound correspondence, every notice, letter, and claim document requires named human approval. Why: legal exposure and trust; this is a governance feature, not a limitation.
No legal advice or quantum assessment, the agent establishes the factual record and drafts per contract mechanics; entitlement strategy and valuation stay with counsel and commercial teams.
No dispute resolution/arbitration support tooling, v1 prevents disputes; litigation support is a future module.

Success Metrics, three layers

Business success

Metric	Target	Timeline
Validated $ incoming claims exposure per pilot project	≥ $20M (customer contracts team confirms materiality)	End of 3–6 mo pilot
Pilot → program license conversion	≥ 60%	Per pilot cohort
ARR	$10–12M	24 months
Notice deadlines missed on covered projects	Zero	Continuous

Product success

Metric	Target	Why
Weekly active review ritual (contracts team opens event register)	≥ 90% of weeks	The habit IS the product; shelfware = death
Surfaced events actioned (reviewed → dispositioned)	≥ 80%	Measures trust, not activity
Draft notices approved with minor/no edits	≥ 70%	Draft quality proxy
Time from triggering event to notice sent	< 7 days (baseline: often never, or post-deadline)	The core "From X → To Y"

AI engineering success

Metric	Target	Notes
Event detection precision (surfaced events that are genuine)	≥ 85%	Precision > recall early: false alarms destroy the review ritual
Event detection recall (vs expert retrospective audit)	≥ 70% v1 → 85% v2	Measured on backtested closed projects
Evidence-chain accuracy (citations resolve to real, relevant documents)	≥ 99.5% for surfaced events	Deterministic post-hoc citation verifier (resolve doc ID + quote-match against source), not model quality alone. At 98%, broken links appear in most weekly reviews
Hallucination rate (unsupported factual assertions per drafted notice)	Zero in sent notices; < 1 per 20 draft notices pre-review	Unit = assertion per notice; verified by automated claim-to-citation entailment check + 100% human review pre-send
Deadline computation accuracy (notice clocks per contract)	100% on test suite; flagged provisional when amendments are missing	Deterministic rules engine, not LLM math. Bespoke amendment chains tested per contract instance
Document ingestion quality (% docs successfully parsed with text extraction)	≥ 95%	OCR pipeline for scanned docs; dead-letter queue for failures; ingestion-quality dashboard is internal, not user-facing
Ingestion latency (new document → triaged)	< 1 hour	Batch acceptable; this is not chat
Cost per document processed	< $0.05 blended	Open-weights model (Llama/Qwen-class) fine-tuned on construction correspondence, self-hosted on sovereign GPU (in-region H100/L40S). At 500K docs/project, COGS discipline matters

Kill criteria: if precision < 60% after 8 weeks of tuning on a live pilot, or the customer contracts team stops the weekly review for 4 consecutive weeks, pause deployment and return to backtest mode.

AI Behavior Contract

Dimension	Specification
Primary tasks	extract (obligations, deadlines, events) · classify (event type, entitlement path) · link (event ↔ evidence ↔ clause) · generate (notice drafts, evidence packs)
Inputs	Contract + amendments, document control system exports (correspondence, RFIs, transmittals, minutes), scheduling tool exports, daily/weekly reports
Constraints	In-region VPC or on-prem (national data protection rules); customer data never trains shared models ("keep your alpha"); full audit log of every read and inference
Disallowed	Sending anything externally; legal opinions; quantum/valuation figures beyond arithmetic from the record; acting on documents outside the registered project scope
Latency budget	Ingestion-to-triage < 1 hr; on-demand Q&A P95 < 15 s

Behavior examples:

Scenario	Input	Expected behavior	Category
Happy path	Contractor sends a letter claiming extra costs for differing site conditions at Area 7; contract GC 4.12 has 28-day notice bar	Detect the threat; find weaknesses in the contractor's notice (e.g. late filing, missing substantiation); compute deadlines; draft the owner's defense response; route to Contracts Lead for approval	✅ Good
Ambiguous signal	Meeting minutes mention "possible resequencing of piping" with no schedule impact yet	Log as watch-item below surfacing threshold; attach to event if corroborating evidence arrives; do not alert	✅ Good
Conflicting evidence	Daily report says milestone achieved; schedule update shows it 3 weeks late	Surface the contradiction itself as a finding with both sources; make no judgment on which is true	✅ Good
Missing data	Contract amendment 3 referenced but not in the archive	State the gap explicitly; flag deadline computations as provisional; request the document	✅ Good
Overreach temptation	User asks "should we accept/reject this claim?"	Provide the factual record, entitlement mechanics, and deadline; decline strategy recommendation; suggest counsel review	🚫 Reject
Out of scope	User uploads documents from a different, unregistered project	Refuse ingestion; log; notify agent owner	🚫 Reject

Governance (the three questions, answered upfront)

Who owns the agent's decisions? The named Contracts Lead on each project owns every disposition and approval. The agent has a registered owner in the customer org (PMO head) accountable for its scope and configuration. The vendor owns model behavior within contract.
What does it do outside its defined scope? Nothing. Out-of-scope inputs are refused, logged, and escalated to the agent owner. Below-confidence findings go to a triage queue, never silently dropped. Escalation path: agent → Contracts Lead → PMO head → steering committee.
How do we measure value created, not activity? Dollars protected from incoming contractor claims (countersigned by the customer's contracts team), notices sent on time that would otherwise have lapsed, and edit-distance on approved drafts. Documents processed and alerts fired are explicitly NOT success metrics.

Rollout Plan

Phase 0, Retrospective backtest (weeks 0–6): run on 1–2 closed projects; expert audit measures recall/precision; produces the "here's what you missed" sales artifact. Passing: ≥70% recall, ≥85% precision, ≥1 material missed entitlement found. Phase 1, Live pilot (months 1–6): 1–2 active projects, owner-side, weekly review ritual embedded with customer contracts team. Passing: business metrics above; customer-countersigned value memo. Phase 2, Program license: expand across the program's EPC packages; begin Schedule Interrogator cross-sell. Contractor-side variant is a future expansion after trust is established with owner-side deployments. Rollback: agent is read-only and advisory; rollback = stop surfacing. No workflow damage possible by design.

Risks and Recovery

Risk	Detection	Fallback	Owner
Precision too low; team tunes out	Weekly actioned-rate < 50%	Raise surfacing threshold; retrain classifiers on dispositions	Product
Customer won't release document archive	Stalled data agreement; confidentiality clauses in underlying contracts may prohibit sharing with third parties	On-prem/customer-VPC only (never cloud); customer reps/warranties that they hold rights to process; start with single-package scope if needed	Sales + Security + Legal
Exposure register is discoverable in arbitration	Legal/compliance review during pilot setup	Customer counsel as agent owner where appropriate; customer-controlled retention/deletion; dispositions framed as team decisions (not individual sign-off); "prepared in anticipation of dispute" labeling where doctrine applies. Note: privilege is weak in the region's courts, don't oversell	Legal + Product
Deadline computed wrong; entitlement lost	Test-suite failure; customer verification step missed	Contractual liability cap (1x annual license); express disclaimer (decision-support, not advice); mandatory customer verification workflow before any deadline relied upon; tech E&O insurance	Legal + Engineering
Agent output used in live dispute and challenged	Legal challenge to evidence chain	Full audit trail + human approval record; position as "prepared by party, assisted by tool"	Legal
Adversarial gaming (counterparty runs their own agent)	Pattern shift in correspondence; both sides deploy	Hard contractual separation between owner-side and contractor-side instances; customer data never crosses; this symmetry is a feature (both sides are better administered)	Product + Legal
Unauthorized practice of law	Regulatory review	Position as document automation under customer counsel's supervision; human approval by authorized signatory; local counsel opinion before MENA launch	Legal

Open Questions

[ ] Contractor-side expansion timing, when owner-side trust is sufficient to extend; @Product + Sales
[ ] Which contract forms to support at launch (FIDIC-style + the asset owner's standard forms assumed), @Legal advisor review
[ ] Notice drafting: template library per contract form, build vs configure per customer, @Engineering
[ ] Pricing: flat pilot fee vs %-of-value-surfaced success component, @GTM
[ ] Customer rep/warranty language for processing counterparty correspondence, @Legal
[ ] Arabic-language document coverage in v1, @Engineering
[ ] Governance of internal visibility: who within the buyer org sees the exposure register (sponsor vs contracts team vs internal audit), critical adoption factor, @Product + Sales
[ ] MSA terms: liability cap, no-reliance clause, IP ownership, audit-log cooperation, indemnity carve-outs, @Legal (route to outside counsel before Phase 1)

Appendix

Impact sizing: outputs/analyses/impact-sizing-shield-2026-08-26.md
Roadmap context: strategy and roadmap (private)
Discovery evidence: construction project research report (private)
Sub-agent review synthesis: outputs/analyses/sub-agent-review-synthesis.md
Changelog: v1 initial draft (2026-08-26, pre sub-agent review) → v2 post sub-agent review (added: discoverability risk, deadline liability, ingestion metrics, model stack, citation verifier, legal risks table, open questions from legal/customer-voice)

AI engineering requirements the agent PRD: controls, evals, out of scope

Written for an earlier persona: the owner's project team, not an infrastructure fund. Real company and system names are replaced, per the name audit.

AI Engineering PRD: Shield Agent (v1)

Stage: Engineering design · Date: 2026-08-26 · Owner: [PM candidate] · Status: Draft, paired with runnable v1 harness Companion code: outputs/prototypes/commercial_guardian_agent/ (agent + eval harness, dummy inputs, zero external dependencies) Product PRD: outputs/prds/commercial-guardian-solution-review.md

1. What we're building

An agent that reads a capital project's contract against its live document record (letters, minutes, schedule updates), detects incoming contractor claims against the owner, evaluates their merit, and drafts the owner's defense with evidence chains, computes notice deadlines, and drafts notices for named human approval. Target buyer: a large asset owner's project team. Non-negotiables: data sovereignty, zero autonomous external action, full auditability.

The engineering thesis: governance is the runtime, not a review step. Every capability the agent has passes through a policy kernel before execution. There is no code path where the agent acts first and gets checked later.

2. Architecture

┌──────────────────────────────────────────┐ │ GOVERNANCE KERNEL (wraps every node) │ │ scope check · action allowlist · │ │ residency tag · owner registry · │ │ append-only audit log │ └──────────────────────────────────────────┘ ingest ──► policy gate ──► ┌─ extract_events (LLM) ─┐ (docs) (refuse │ ├──► verify_citations ──► compute_deadlines ──► draft_notice ──► APPROVAL GATE ──► register out-of-scope) └─ link_evidence (LLM) ─┘ (deterministic) (deterministic (template + (named human; (event [parallel fan-out] rules engine) LLM assist) nothing leaves register) without it)

Graph orchestration. The pipeline is an explicit DAG. Nodes declare inputs/outputs; the runner executes independent nodes in parallel (extraction and evidence linking fan out per document batch). Explicit graph > free-form agent loop: deterministic replay, per-node audit, per-node eval.
LLM only where judgment is needed. Extraction, classification, and draft language use a model. Citation verification and deadline math are deterministic code, an LLM never computes a notice deadline. (PRD metric: 100% deadline accuracy on test suite; only a rules engine can promise that.)
Deterministic verifiers behind every LLM node. Every citation an LLM emits must resolve to a real document ID and quote-match its source, or the finding is downgraded and never surfaced. Hallucinations are caught structurally, not by hoping the model behaves.
v1 model adapter is swappable. The harness ships with a deterministic dummy adapter so evals run offline and reproducibly. Production slot: open-weights model (Llama/Qwen-class) fine-tuned on construction correspondence, self-hosted on in-region GPUs. Same interface, same evals.

3. Governance kernel (the core, not a feature)

Control	Mechanism	Enforced where
Scope	Agent instance is registered to exactly one project; documents outside it are refused at ingest, logged, escalated	Policy gate, pre-execution
Action allowlist	Agent capabilities are an explicit allowlist (read, extract, link, draft, queue-for-approval). "Send" is not in the list, it does not exist as an action	Kernel, every node call
Human gate	Notices reach `PENDING_APPROVAL` and stop. Approval requires a named approver from the owner registry; the kernel rejects self-approval and unknown approvers	Approval gate
Overreach refusal	Requests for legal strategy/claim advice are refused with a routed suggestion (counsel review)	Q&A boundary check
Missing data	Deadline computations referencing missing amendments are marked `PROVISIONAL`, never presented as final	Rules engine
Sovereignty	Every artifact carries a residency tag (`in-region`); kernel rejects any node declaring an out-of-region data sink. No customer data in training paths	Kernel invariant
Audit	Append-only log of every node invocation, refusal, approval, with actor + hash chain	Kernel, always on

4. Evals (precise, useful, gate releases)

Eval suites run in CI against golden-labeled dummy projects; production adds backtests on closed projects. A release that fails a governance eval cannot ship, these are hard gates, not dashboards.

Suite	What it measures	Gate
Detection quality	Event precision / recall vs golden labels	precision ≥ 85%, recall ≥ 70%
Evidence integrity	% citations that resolve + quote-match source docs	≥ 99.5%
Hallucination	Unsupported factual assertions in drafted notices (claim→citation entailment)	0 in approvable drafts
Deadline correctness	Rules-engine output vs hand-computed goldens, incl. missing-amendment provisional flagging	100%
Governance: scope	Out-of-scope document → refused + logged + escalated	100% refusal
Governance: human gate	No notice reaches `SENT`/`REGISTERED` without named approval; self-approval rejected	100%
Governance: overreach	"Should we accept/settle this claim?" → factual record only, strategy refused	100% refusal
Governance: audit	Every pipeline action has an audit event; hash chain verifies	100%
Ops	p95 pipeline latency, cost/document (tokens on real adapter)	< 1 hr ingest-to-triage; < $0.05/doc

Why precision over recall in v1: false alarms kill the weekly review ritual, and the ritual is the product. Recall improves via backtest tuning; trust, once lost, doesn't.

5. Self-improvement loop (governed)

Every human disposition (approve / edit / dismiss) is captured as a labeled example in the audit trail. Weekly: dismissals retune surfacing thresholds; edit-diffs feed draft-template refinement; confirmed events grow the golden eval set. All retraining happens in-region on the customer's instance, dispositions never leave, and no update ships until the full eval suite passes on the expanded golden set. The agent gets better on your project without your data improving anyone else's.

6. Key tradeoffs (decided)

Explicit DAG vs autonomous agent loop. Chose DAG. Costs flexibility; buys determinism, per-node evals, audit clarity. Right trade for zero-risk enterprise.
Precision-first thresholds. Chose to under-surface early. Costs missed events in v1; buys the review ritual surviving. Recall is recoverable, trust isn't.
Deterministic deadline engine vs LLM reasoning. Chose rules engine. Costs per-contract-form configuration; buys a 100% guarantee we can put in an MSA.
Dummy adapter in the harness vs mocked API calls. Chose a fully deterministic in-process adapter. Costs realism; buys reproducible CI evals with zero keys, zero network, zero data egress, the harness itself is sovereignty-compliant.
Parallelism at document-batch level only. Fan-out where work is independent (per-doc extraction); strict sequence where order is safety (verify → deadline → draft → approve). Parallelism is a performance tool, never applied across the governance chain.

7. Out of scope (v1 harness)

Real model integration, OCR/ingestion pipeline, Arabic documents, live scheduling and document system connectors, UI. The harness proves the control plane and the eval methodology; those slot in behind the same interfaces.

External pitch deck 9 slides, the pitch to an asset owner, as presented

Open the deck in a new tab.

Internal pitch deck 16 slides, the pitch to the CTO, as presented

Open the deck in a new tab.

Links demo, code

Demo
Code: published once real company names are checked and replaced (name audit).

How it was built with PM OS the product process, step by step

These documents were written for a project-owner persona. The infrastructure fund framing on this page is new and has no PRD (product spec) yet.

Stage	Artifact	Repeatable with
Discovery	Research report on large construction projects; persona journeys	`/competitor-analysis, /journey-map`
Opportunity	Impact sizing (all figures modelled from assumptions); challenge on the competitor analysis	`/impact-sizing, /ralph-wiggum`
PRD	Product spec (PRD) and AI engineering spec, reviewed by 5 AI reviewer personas	`/prd-draft, /prd-review-panel`
Strategy	Strategy and 3-horizon roadmap; delivery plan with goals (OKRs)	`/write-prod-strategy, /metrics-framework`
Build	Agent with built-in approval rules; clickable prototype	`/code-first-draft, /prototype`
Evaluation	17 agent evals (automated tests), 10 demo tests	`/goal-gate`
Showcase	This page, name audit, goals and state files	`/portfolio-case-study, /name-audit`

Skills are mapped from the artifact each stage produced. They show how each stage repeats with PM OS; the original working sessions were not logged.

PM OS skills used

/code-first-draft /competitor-analysis /goal-gate /impact-sizing /journey-map /metrics-framework /name-audit /portfolio-case-study /prd-draft /prd-review-panel /prototype /ralph-wiggum /write-prod-strategy