192 / 192 catalog scenarios passed. They check that plans preserve saved preferences, offer only eligible choices, explore one selected limit and keep the normal return available.
These are software contracts using fictional catalog data. They are not model accuracy, visual fidelity or shopper outcomes.
Model pilot
Each model run keeps its own results. The image reviewers are from the same model family. Agreement is descriptive; independent human review remains 0.
Run 1 — stopped
AI-authored implementation note. The second model returned an observation longer than the contract limit and included a verification disclaimer. The validator rejected it; its finding also mixed the requested print criterion with unrelated texture fidelity. All calls settled; USD 0.029854 charged, zero reserved. This was an unusable assessment, not uncertain billing.
Copilot tool contracts completed: 6 / 6. Image cases with both assessments completed: 2 / 6.
Same finding in 4 / 4 assessed attribute pairs. This is not an accuracy score. Pilot ledger cumulative cost: USD 0.029854; unresolved reservations: USD 0.000000.
agent-01 — completed
Expected route: CHOOSE_LIMIT. Returned route: CHOOSE_LIMIT. Planner: claude.
No top meets every confirmed limit. Which one limit would you be willing to revisit? You can also continue with a normal return.
Read confirmed limits
Checked current catalog eligibility
agent-02 — completed
Expected route: REVIEW_TRADEOFF. Returned route: REVIEW_TRADEOFF. Planner: claude.
Changing only your selected limit opens a current catalog possibility. Review the change, then confirm preferences separately. Your saved limits are unchanged.
Read confirmed limits
Checked current catalog eligibility
Explored your selected limit
agent-03 — completed
Expected route: CHOOSE_LIMIT. Returned route: CHOOSE_LIMIT. Planner: claude.
No top meets every confirmed limit. Changing only the selected limit did not unlock an option. Which one limit would you be willing to revisit? You can also continue with a normal return.
Read confirmed limits
Checked current catalog eligibility
Explored your selected limit
agent-04 — completed
Expected route: REVIEW_TRADEOFF. Returned route: REVIEW_TRADEOFF. Planner: claude.
Changing only your selected limit opens a current catalog possibility. Review the change, then confirm preferences separately. Your saved limits are unchanged.
Read confirmed limits
Checked current catalog eligibility
Explored your selected limit
agent-05 — completed
Expected route: COMPARE_OPTIONS. Returned route: COMPARE_OPTIONS. Planner: claude.
The current catalog has eligible choices. Compare them with the item bought; an image cannot establish physical fit or exact appearance.
Read confirmed limits
Read confirmed limits
Checked current catalog eligibility
Checked requested-change review evidence
agent-06 — completed
Expected route: COMPARE_OPTIONS. Returned route: COMPARE_OPTIONS. Planner: claude.
The current catalog has eligible choices. Compare them with the item bought; an image cannot establish physical fit or exact appearance.
Read confirmed limits
Checked current catalog eligibility
Checked requested-change review evidence
review-dev-CALM-04 — completed
known development recording
bought_catalog · exact model inputreplacement_catalog · exact model inputdisplayed_preview · exact model input
claude-sonnet-5-5
AI requested-change assessment; independent human review pending Inspect the image yourself. This assessment cannot establish physical fit, exact appearance or overall fidelity.
sleeves — supports: Both sleeves in the preview extend down to the wrists with no visible rolling or pushing up; the right cuff is near the hip pocket and the left reaches the wrist.
print — supports: The preview top is a plain heathered green with only subtle texture, much quieter than the bought product's dense multicolour floral print.
claude-haiku-5-5
AI requested-change assessment; independent human review pending Inspect the image yourself. This assessment cannot establish physical fit, exact appearance or overall fidelity.
sleeves — supports: The generated preview shows full-length sleeves extending to the wrists with no visible rolling or pushing up, consistent with the requested long sleeves.
print — supports: The preview garment shows a plain, solid-looking green knit with no visible floral print, which is quieter than the busy floral pattern of the bought product.
review-dev-CALM-07 — completed
known development recording
bought_catalog · exact model inputreplacement_catalog · exact model inputdisplayed_preview · exact model input
claude-sonnet-5-5
AI requested-change assessment; independent human review pending Inspect the image yourself. This assessment cannot establish physical fit, exact appearance or overall fidelity.
sleeves — supports: Both arms in the preview are covered by full-length ribbed sleeves reaching the wrists, with no visible pushing up or rolling.
print — supports: The preview shows a solid red ribbed knit with only vertical rib texture, far quieter than the busy multicolor floral print of the bought blouse.
claude-haiku-5-5
AI requested-change assessment; independent human review pending Inspect the image yourself. This assessment cannot establish physical fit, exact appearance or overall fidelity.
sleeves — supports: The displayed preview shows full-length sleeves extending to the wrist on the red ribbed top, with no visible rolling or pushing up, though the replacement catalog photo also shows long sleeves.
print — supports: The replacement preview shows a plain solid red ribbed knit with no visible print, which is quieter than the busy floral pattern of the bought product.
review-heldout-v1-02 — failed_or_uncertain
known frozen photo-study output; new model assessment
bought_catalog · exact model inputreplacement_catalog · exact model inputdisplayed_preview · exact model input
claude-sonnet-5-5
AI requested-change assessment; independent human review pending Inspect the image yourself. This assessment cannot establish physical fit, exact appearance or overall fidelity.
sleeves — supports: In the preview, both sleeves extend down to the wrists with cuffs visible, and they are not rolled or pushed up.
print — supports: The preview top is a plain heathered green with only subtle texture, much quieter than the bought product's dense multicolor floral print.
Uncompleted assessment
No AI requested-change assessment The assessment did not complete. No result is inferred and no automatic retry was made.
review-heldout-v1-03 — not_attempted
Not attempted.
review-heldout-v1-05 — not_attempted
Not attempted.
review-heldout-v1-06 — not_attempted
Not attempted.
Run 2
Copilot tool contracts completed: 6 / 6. Image cases with both assessments completed: 6 / 6.
Same finding in 11 / 12 assessed attribute pairs. This is not an accuracy score. Pilot ledger cumulative cost: USD 0.091508; unresolved reservations: USD 0.000000.
agent-01 — completed
Expected route: CHOOSE_LIMIT. Returned route: CHOOSE_LIMIT. Planner: claude.
No top meets every confirmed limit. Which one limit would you be willing to revisit? You can also continue with a normal return.
Read confirmed limits
Checked current catalog eligibility
agent-02 — completed
Expected route: REVIEW_TRADEOFF. Returned route: REVIEW_TRADEOFF. Planner: claude.
Changing only your selected limit opens a current catalog possibility. Review the change, then confirm preferences separately. Your saved limits are unchanged.
Read confirmed limits
Checked current catalog eligibility
Read confirmed limits
Explored your selected limit
agent-03 — completed
Expected route: CHOOSE_LIMIT. Returned route: CHOOSE_LIMIT. Planner: claude.
No top meets every confirmed limit. Changing only the selected limit did not unlock an option. Which one limit would you be willing to revisit? You can also continue with a normal return.
Read confirmed limits
Checked current catalog eligibility
Explored your selected limit
agent-04 — completed
Expected route: REVIEW_TRADEOFF. Returned route: REVIEW_TRADEOFF. Planner: claude.
Changing only your selected limit opens a current catalog possibility. Review the change, then confirm preferences separately. Your saved limits are unchanged.
Read confirmed limits
Checked current catalog eligibility
Explored your selected limit
agent-05 — completed
Expected route: COMPARE_OPTIONS. Returned route: COMPARE_OPTIONS. Planner: claude.
The current catalog has eligible choices. Compare them with the item bought; an image cannot establish physical fit or exact appearance.
Read confirmed limits
Checked current catalog eligibility
Checked requested-change review evidence
agent-06 — completed
Expected route: COMPARE_OPTIONS. Returned route: COMPARE_OPTIONS. Planner: claude.
The current catalog has eligible choices. Compare them with the item bought; an image cannot establish physical fit or exact appearance.
Read confirmed limits
Checked current catalog eligibility
Checked requested-change review evidence
review-dev-CALM-04 — completed
known development recording
bought_catalog · exact model inputreplacement_catalog · exact model inputdisplayed_preview · exact model input
claude-sonnet-5-5
AI requested-change assessment; independent human review pending Inspect the image yourself. This assessment cannot establish physical fit, exact appearance or overall fidelity.
sleeves — supports: Both sleeves extend down to the wrists in the preview, with cuffs near the hands and no rolling or pushing up.
print — supports: The preview shows a plain green top with only subtle heathered texture, much quieter than the bought product's dense multicolour floral pattern.
claude-haiku-5-5
AI requested-change assessment; independent human review pending Inspect the image yourself. This assessment cannot establish physical fit, exact appearance or overall fidelity.
sleeves — supports: The preview shows full-length sleeves extending to the wrists with no visible rolling or pushing up, matching the requested long sleeve coverage.
print — supports: The preview top is a solid green knit with only a subtle textured surface, far quieter than the dense multicolour floral print of the bought product.
review-dev-CALM-07 — completed
known development recording
bought_catalog · exact model inputreplacement_catalog · exact model inputdisplayed_preview · exact model input
claude-sonnet-5-5
AI requested-change assessment; independent human review pending Inspect the image yourself. This assessment cannot establish physical fit, exact appearance or overall fidelity.
sleeves — supports: Both sleeves extend down to the wrists with ribbed cuffs visible at the hands, so the preview shows long sleeves without any pushing up.
print — supports: The preview shows a plain red ribbed knit with only vertical rib lines, far quieter than the dense multicolour floral print on the bought blouse.
claude-haiku-5-5
AI requested-change assessment; independent human review pending Inspect the image yourself. This assessment cannot establish physical fit, exact appearance or overall fidelity.
sleeves — supports: Preview shows full-length sleeves reaching the wrists, with the cuffs visible at the wrist and no rolling or pushing up evident.
print — supports: Preview shows a solid red ribbed knit with no printed pattern, quieter than the busy floral print of the bought product.
review-heldout-v1-02 — completed
known frozen photo-study output; new model assessment
bought_catalog · exact model inputreplacement_catalog · exact model inputdisplayed_preview · exact model input
claude-sonnet-5-5
AI requested-change assessment; independent human review pending Inspect the image yourself. This assessment cannot establish physical fit, exact appearance or overall fidelity.
sleeves — supports: Both sleeves extend down to the wrists in the preview, with cuffs visible near the hands and no pushing up or rolling.
print — supports: The preview top is a plain muted green with only a faint heathered texture, far quieter than the busy multicolor floral on the bought blouse.
claude-haiku-5-5
AI requested-change assessment; independent human review pending Inspect the image yourself. This assessment cannot establish physical fit, exact appearance or overall fidelity.
sleeves — supports: The preview shows long sleeves extending to the wrists with cuffs visible at the hands; the sleeves are not rolled or pushed up.
print — supports: The preview garment shows a plain heathered green surface with no floral or busy pattern, quieter than the bought floral print.
review-heldout-v1-03 — completed
known frozen photo-study output; new model assessment
bought_catalog · exact model inputreplacement_catalog · exact model inputdisplayed_preview · exact model input
claude-sonnet-5-5
AI requested-change assessment; independent human review pending Inspect the image yourself. This assessment cannot establish physical fit, exact appearance or overall fidelity.
sleeves — supports: Both sleeves extend down to the wrists and hands with ribbed cuffs, and they are not pushed up or rolled, so long coverage is visible.
print — supports: The preview shows a plain red knit with only vertical ribbing, far quieter than the dense multicolour floral print on the bought blouse.
claude-haiku-5-5
AI requested-change assessment; independent human review pending Inspect the image yourself. This assessment cannot establish physical fit, exact appearance or overall fidelity.
sleeves — supports: The preview shows full-length sleeves extending to the wrists with ribbed cuffs, not rolled or pushed up.
print — supports: The preview garment is a solid red ribbed knit with no visible print, quieter than the busy floral pattern of the bought product.
review-heldout-v1-05 — completed
known frozen photo-study output; new model assessment
bought_catalog · exact model inputreplacement_catalog · exact model inputdisplayed_preview · exact model input
claude-sonnet-5-5
AI requested-change assessment; independent human review pending Inspect the image yourself. This assessment cannot establish physical fit, exact appearance or overall fidelity.
sleeves — concern: Sleeves bunch and push up at the forearms, ending above the wrists with the cuffs rolled, so full long coverage is not clearly shown.
print — supports: Preview shows a plain green knit with only faint texture, far quieter than the busy multicolor floral print on the bought blouse.
claude-haiku-5-5
AI requested-change assessment; independent human review pending Inspect the image yourself. This assessment cannot establish physical fit, exact appearance or overall fidelity.
sleeves — supports: The preview shows the green top with full-length sleeves reaching the wrists, with no visible rolling or pushing up of the cuffs.
print — supports: The preview top shows a plain heathered green surface with only subtle textural mottling, far quieter than the dense multicolour floral print of the bought product.
review-heldout-v1-06 — completed
known frozen photo-study output; new model assessment
bought_catalog · exact model inputreplacement_catalog · exact model inputdisplayed_preview · exact model input
claude-sonnet-5-5
AI requested-change assessment; independent human review pending Inspect the image yourself. This assessment cannot establish physical fit, exact appearance or overall fidelity.
sleeves — supports: Both sleeves extend down to the wrists with ribbed cuffs near the hands; no rolled or pushed-up sleeves are visible.
print — supports: The preview shows a plain solid red ribbed knit, far quieter than the dense multicolor floral print on the bought blouse.
claude-haiku-5-5
AI requested-change assessment; independent human review pending Inspect the image yourself. This assessment cannot establish physical fit, exact appearance or overall fidelity.
sleeves — supports: The preview shows the red ribbed top with full-length sleeves reaching the wrists, with cuffs visible at both hands.
print — supports: The replacement top is a solid red ribbed knit with no printed pattern, quieter than the busy floral print of the bought shirt.
Limits and reproducibility
Known public synthetic recordings, not a new unseen-product study.
Both reviewing models belong to the same family; agreement is descriptive, not independent human accuracy.
Deterministic contracts do not measure shopper benefit, return reduction or visual fidelity.
Failed and unattempted cases remain visible; repeated runs retain their own denominators.
A requested-change assessment says what a model sees in a garment image. It cannot establish physical fit, comfort, exact real-world appearance or overall fidelity. Disagreement between the two recorded reviews becomes an unclear finding in the demo.
The first command runs the offline contracts without a model call. A paid run requires a prepared registration, an explicit USD cap and approval. Publishing reads the retained reports and makes no API call.