Can we sign the room booking agreement today?

A helper can organize the evidence for an agreement without having authority to interpret or accept it.

The venue sends different terms and refers to another document that is not attached. The club checklist does not cover the new cancellation wording. The venue wants an answer today.

What should Leah do before the club agrees?

• Use the usual checklist summary to keep moving
  Why this choice?

  The missing document could change the agreement. A familiar summary cannot fill that gap; request the attachment and flag the unfamiliar terms before anyone commits.

• Get the missing document and ask the authorized organizer
  Why this choice?

  Hold acceptance while the organizer reviews the complete terms and gets qualified advice if needed. The helper can list unanswered questions, but should not guess what the missing document says.

Use summaries to support an accountable decision, never to replace missing terms or qualified judgment.

What would you want to see in the original agreement before trusting a summary?

Clause extraction helps reviewers find text and omissions. Counsel decides legal interpretation, and authorized humans approve binding terms. This is a workflow example, not legal advice.

Use AI to prepare a contract comparison, then recognize when changed terms require specialist judgment.

Legal operations and contract review · 8 minutes

The problem

Synthetic practice scenario: all quantities and evaluation outcomes are illustrative, not reported research. Fictional outcomes are not guarantees. A legal operations team prepares supplier agreements for counsel using an approved clause playbook; the assistant cannot sign, negotiate or give legal advice.

Your initial decision

How should the assistant help reviewers compare familiar agreement terms?

• The fictional workload has 24 agreements based on one familiar template and a current counsel-approved playbook.
• Reviewers need exact clause locations and omissions, including referenced schedules, not a general risk score.
• Contract text is untrusted evidence; instructions embedded in an attachment cannot change the review rules.
• Counsel decides legal interpretation and exceptions; an authorized human approves binding terms and signature.

Compare the alternatives

• Have counsel perform the entire first pass

  Counsel can examine language directly with no extraction dependency, but routine clause finding consumes time needed for substantive judgment.

• Show only a literal template comparison

  A text diff reliably exposes changed wording, but cannot reliably locate relocated clauses or resolve cross-references across attached schedules.

• Prepare a source-linked clause review packet

  Extraction maps clauses and omissions to the approved playbook while counsel checks source spans and decides their significance.

Mechanism, trade-offs, and evidence

• The useful delegation is document navigation and comparison, not a legal conclusion about enforceability or acceptable liability.
• Evaluate omissions as well as extracted text: an absent schedule can matter more than a fluent summary of the main agreement.
• Keep the original documents beside the packet so counsel can challenge the assistant rather than merely approve its wording.

Reconsider after a change

A new supplier proposes an unfamiliar jurisdiction and incorporates an unavailable schedule that may change liability; the approved playbook does not cover this combination.

What should happen to this agreement before anyone promises acceptance?

• In the illustrative review, 2 referenced schedules are missing and the governing-law clause is outside the tested template family.
• No local evaluation establishes that the assistant can identify the effect of this combination.
• Counsel must obtain the schedules and decide whether specialist advice or a playbook update is needed.
• Commercial staff may request documents but cannot treat an assistant summary as approval to sign or accept terms.

Compare the alternatives

• Route the exception to counsel with missing evidence

  Specialist review delays this agreement while missing schedules are obtained, but places an unfamiliar interpretation with its authorized owner.

• Apply the familiar template diff to the exception

  A familiar comparison offers a quick list of changed words, but its baseline cannot explain unseen schedules or a different legal context.

• Keep extraction only as an incomplete evidence aid

  Linked passages can still reduce document navigation effort, but the packet must visibly mark missing schedules and exclude out-of-playbook recommendations.

Mechanism, trade-offs, and evidence

• The missing information changes the task from familiar comparison to unresolved legal interpretation; better prose cannot fill that gap.
• Pause the consequential commitment, not every administrative action: staff can still request documents and organize cited evidence.
• This is a review-workflow exercise, not legal advice; the policy owner and counsel determine rights, obligations and approval requirements.

Related technical concepts

Follow-up lessons retain their existing sign-in and plan requirements.

• hallucination grounding
• human in the loop patterns
• agent uncertainty calibration

Technical sources

• AI Risk Management Framework — NIST

  Organizes voluntary risk management around governing, mapping, measuring and managing; it is neither certification nor a return-on-investment study.

• Demystifying evals for AI agents — Anthropic Engineering

  Explains tasks, graders, repeated trials and outcome evaluation for agents; use task-specific tests rather than importing reported scores.

Explore the technical decision