The caterers sent their prices. Are we comparing the same thing?

Agree what matters before comparing offers. Keep must-have requirements separate from attractive extras.

For the next event, offers arrive as emails, menus and attachments. One price is per tray and another is per person. A tried helper can find the amounts and show the exact source for each.

Where would that helper be useful now?

• Prepare comparable figures with source links
  Why this choice?

  That addresses the new searching work. Check servings, delivery charges and unclear dietary claims before comparing totals; keep the organizer in charge of the booking.

• Sort the quoted prices from lowest to highest
  Why this choice?

  A tray price and a price per person are not directly comparable. First establish servings and included charges, using the helper to find evidence rather than guess missing quantities.

Improve the evidence going into a decision before adding more opinions about the winner.

What hidden difference could make the apparently cheapest offer cost more?

AI can locate bid fields and their sources. Fixed calculations compare consistent units, while procurement owners verify important claims and approve supplier awards.

Choose a fair comparison method, then adapt when bids stop describing the same thing in the same format.

Procurement and supplier evaluation · 8 minutes

The problem

Synthetic practice scenario: all quantities and evaluation outcomes are illustrative, not reported research. Fictional outcomes are not guarantees. A procurement team compares packaging suppliers against published requirements and needs an accountable recommendation before an authorized buyer awards a contract.

Your initial decision

How should the team compare complete bids with agreed criteria?

• There are 6 fictional bids in a common form with price, delivery range and verified product test results.
• The procurement owner has fixed required checks and comparison weights before opening bids.
• A supplier statement is not proof of certification or performance; conflicts of interest must be disclosed.
• An authorized human awards the contract; the tool cannot change criteria to favor a bidder or make a purchase commitment.

Compare the alternatives

• Use a checked comparison table and sensitivity analysis

  A transparent table computes agreed measures consistently, but reviewers must examine how close results change under plausible cost or delivery variation.

• Extract bid evidence into the agreed comparison

  Document extraction can link claims to supporting passages, but adds correction and unit-conversion work when the supplied fields are already complete.

• Ask independent AI reviewers for a ranked shortlist

  Several reviews may surface different concerns, but their consensus can obscure shared assumptions and is less reproducible than the agreed calculation.

Mechanism, trade-offs, and evidence

• Separate mandatory qualification from preferences so an attractive price cannot cancel a missing required check.
• Keep calculations deterministic and auditable; ask whether changing a plausible delivery assumption changes the recommendation.
• Human award authority matters, but useful automation can still calculate totals and highlight missing evidence without manual rechecking of every arithmetic step.

Reconsider after a change

A new tender accepts varied document formats; buyers now spend most of their review time finding delivery conditions and converting price units rather than comparing complete fields.

Which addition addresses the new procurement bottleneck?

• An independent illustrative test uses 40 unseen bid documents; 36 produce fully verified fields and 4 are correctly flagged for review.
• Every extracted price retains currency, unit, quantity basis and a source span; unverified claims remain visibly unverified.
• The procurement owner retains the existing criteria; deterministic calculations reject missing or incompatible units.
• A buyer reviews material uncertainties and approves the award; supplier attachments have no authority to instruct tools.

Compare the alternatives

• Keep staff entering all fields into the table

  Human entry avoids model extraction errors and keeps the familiar comparison, but repeats the measured document-navigation burden for every bid.

• Pilot cited extraction into the fixed comparison

  Verified field extraction targets the new bottleneck while preserving deterministic scoring, but reviewers still resolve ambiguity and check unsupported claims.

• Expand AI panel ranking across all tender documents

  A panel can digest long proposals and suggest concerns, but combining rankings still hides whether prices and delivery terms are comparable.

Mechanism, trade-offs, and evidence

• The calculation did not become harder; evidence preparation did, so add assistance at that boundary rather than replacing the award process.
• The test set is illustrative and separate from development examples; finite success justifies a monitored trial, not guaranteed supplier quality.
• Preserve provenance from source span to normalized field to final comparison so a reviewer can challenge any recommendation.

Related technical concepts

Follow-up lessons retain their existing sign-in and plan requirements.

• verified analytical agents
• ai ready data
• human agent delegation

Technical sources

• Building effective agents — Anthropic Engineering

  Distinguishes fixed workflows from model-directed agents and recommends testing simpler solutions first; its tooling examples are historical.

• Demystifying evals for AI agents — Anthropic Engineering

  Explains tasks, graders, repeated trials and outcome evaluation for agents; use task-specific tests rather than importing reported scores.

• AI Risk Management Framework — NIST

  Organizes voluntary risk management around governing, mapping, measuring and managing; it is neither certification nor a return-on-investment study.

Explore the technical decision