This page discloses the assignments’ planted mechanisms.

ENCODED TRUTHBoth opening scorecards can be arithmetically correct. The divergence combines incompatible 1–5 and 1–10 survey scales, migration-era duplicate source tickets, and non-random survey response. There is no secret preferred aggregation.

WATCH FORRaw-score averaging, deleting every repeated-looking ticket, or generalizing respondent satisfaction to every customer.

ACCEPTABLE OUTCOME SPACEOne carefully defined headline, paired measures with different estimands, or a refusal to collapse them into one number can all be excellent.

DELIVERY FORMATBest as an individual first assignment. Let pairs compare definitions only after each learner has written an independent metric statement.

ENCODED TRUTHAcquired-system order numbers are tenant-local and some timestamps are shifted by about five hours. Canonical order IDs, captured transaction amounts, named clocks, and a frozen cutoff can reconcile the quarter; price history is not a sales ledger.

WATCH FORDeduplicating on an unnamespaced source ID, overwriting captured price, or joining several one-to-many tables before aggregation.

ACCEPTABLE OUTCOME SPACEA bounded certification with an exception population is stronger than either a false clean bill of health or an unqualified refusal to certify anything.

DELIVERY FORMATRun individually or in pairs. In pairs, require separate SQL and Python ownership followed by a joint reconciliation review.

ENCODED TRUTHTreatment genuinely increases session event depth, but the estate does not label whether the extra depth is useful or frustrating and does not encode a matching conversion effect. Session is the registered assignment unit.

WATCH FORWeighting deep sessions more heavily through event joins, redefining eligibility from the experiment name, or interpreting no significant conversion difference as equivalence.

ACCEPTABLE OUTCOME SPACERollout, guarded rollout, or retest can all be defensible when the intent-to-treat estimate, uncertainty, UX interpretation, and monitoring match the action.

DELIVERY FORMATUse pairs for statistical checking, but have each learner write an independent recommendation before comparing decisions.

ENCODED TRUTHA severe winter event, missing-not-at-random telemetry, a firmware measurement shift, and a scoped Great Lakes issue overlap. The release does not identify a clean firmware-only physical effect or justify a single hidden rollback answer.

WATCH FORTreating missing telemetry as healthy equipment, using delayed outcomes at the decision time, or mistaking a measurement shift for physical failure.

ACCEPTABLE OUTCOME SPACEGlobal rollback, scoped containment, or monitored continuation can each score highly when the learner names assumptions, counterevidence, thresholds, and exposure.

DELIVERY FORMATStrong small-team incident simulation: assign evidence lanes, then require one shared command decision and one dissent note.

ENCODED TRUTHConvenient final-state tables leak the future. The point-in-time target is appointment-window breach; storm exposure and window length contain real signal, while the decision population shifts during the storm. Risk is not treatment effect.

WATCH FORUsing realized arrival/completion facts, optimizing AUC without a capacity policy, or turning technician identity into an employee score.

ACCEPTABLE OUTCOME SPACEA simple rule, calibrated model, shadow-only queue, or refusal to automate is valid if it respects capacity, abstention, data sensitivity, and evidentiary limits.

DELIVERY FORMATUse pairs or teams of three for the build; require named ownership for mart, model audit, and operations policy. Rotate the reviewer role.

ENCODED TRUTHThe movement table has a technical-event grain: linked scanner replays are not new physical movements. Accepted receipts, not gross dock quantity, post to the ledger; lines may split or remain partial. Position snapshots are sparse, demand lives in ISSUE events, absent intermittent-event days are zeroes, seasonal families genuinely peak, and realized vendor lead times vary from 1–83 days.

WATCH FORCounting scanner replays, using gross receipts, treating sparse positions as a daily spine, dropping zero-demand days, validating randomly through time, or substituting promised lead time for realized uncertainty.

ACCEPTABLE OUTCOME SPACEDifferent forecast families and action mixes are valid. Grade honest backtests, coherent uncertainty, feasible actions, and decision stability rather than a single winning model.

DELIVERY FORMATDesigned for teams of three or four with explicit data, forecasting, optimization, and review roles. Require a common reproducibility contract.

ENCODED TRUTHORION-2 creates a real operating-regime change but no clean 12% causal productivity answer. Adoption has no untreated geographic control, route stops are not work-order grain, and joins across histories and parts multiply rows.

WATCH FORClaiming causal ROI from before/after averages, counting rescheduled work twice, or ranking technicians without addressing assignment policy.

ACCEPTABLE OUTCOME SPACERenew, renegotiate with measurement conditions, defer, or decline can all be credible. Lack of clean causal identification does not remove the obligation to make a bounded procurement recommendation.

DELIVERY FORMATBest for pairs or a board-style team. Assign one learner to defend attribution and another to audit it before a joint recommendation.

ENCODED TRUTHThe current text templates, subjects, categories, and teams are generated independently, while the available team label is final rather than adjudicated initial ownership. Honest clean-holdout routing signal may be insufficient; refusing launch and designing shadow labels is a successful outcome.

WATCH FORUsing later agent messages, accepting final assignment as ground truth, overlooking duplicate interactions, or forcing a classifier to win.

ACCEPTABLE OUTCOME SPACELimited suggestions, human-review routing, shadow-only collection, taxonomy repair, or principled refusal are likely stronger than live automation in this estate.

DELIVERY FORMATUse teams of three for corpus, model, and governance roles. Grade the quality of the boundary and shadow design as seriously as model performance.

ENCODED TRUTHThe near-perfect model is invalid: future outcomes leak into predictors, accounts cross random row splits, time is mixed, and governance/deployment evidence is absent. A post-audit AUC is not a grading target.

WATCH FORPrefix-only leakage removal, random row splitting, or treating offline discrimination as deployment approval.

ACCEPTABLE OUTCOME SPACERejecting the submitted model, allowing a tightly governed shadow rebuild, or authorizing a simpler allow-listed baseline can all be strong dispositions. Immediate campaign action is not supported.

DELIVERY FORMATExcellent individual final audit or paired red-team exercise. Have learners review one another’s allow-lists and deployment dispositions.