THE ANALYST / INSTRUCTOR NOTES
RETURN TO PLANNING DESK

SPOILER-SEPARATED / PUBLIC RESOURCE

This page discloses the assignments’ planted mechanisms.

Keep learners in the workbench. These notes establish what the data generator actually encodes, where refusal is valid, and what not to grade toward. They are review guidance—not answer keys.

01The Monday ScorecardCC-241202validated

ENCODED TRUTHBoth opening scorecards can be arithmetically correct. The divergence combines incompatible 1–5 and 1–10 survey scales, migration-era duplicate source tickets, and non-random survey response. There is no secret preferred aggregation.

WATCH FORRaw-score averaging, deleting every repeated-looking ticket, or generalizing respondent satisfaction to every customer.

02Rollback Before DawnOP-250320validated

ENCODED TRUTHA severe winter event, missing-not-at-random telemetry, a firmware measurement shift, and a scoped Great Lakes issue overlap. The release does not identify a clean firmware-only physical effect or justify a single hidden rollback answer.

WATCH FORTreating missing telemetry as healthy equipment, using delayed outcomes at the decision time, or mistaking a measurement shift for physical failure.

03The Orion RenewalPR-260119validated

ENCODED TRUTHORION-2 creates a real operating-regime change but no clean 12% causal productivity answer. Adoption has no untreated geographic control, route stops are not work-order grain, and joins across histories and parts multiply rows.

WATCH FORClaiming causal ROI from before/after averages, counting rescheduled work twice, or ranking technicians without addressing assignment policy.

04Too Good to ShipMR-260120validated

ENCODED TRUTHThe near-perfect model is invalid: future outcomes leak into predictors, accounts cross random row splits, time is mixed, and governance/deployment evidence is absent. A post-audit AUC is not a grading target.

WATCH FORPrefix-only leakage removal, random row splitting, or treating offline discrimination as deployment approval.

05The Quarter That MovedCM-240708validated

ENCODED TRUTHAcquired-system order numbers are tenant-local and some timestamps are shifted by about five hours. Canonical order IDs, captured transaction amounts, named clocks, and a frozen cutoff can reconcile the quarter; price history is not a sales ledger.

WATCH FORDeduplicating on an unnamespaced source ID, overwriting captured price, or joining several one-to-many tables before aggregation.

06The Navigation VoteGX-250505validated

ENCODED TRUTHTreatment genuinely increases session event depth, but the estate does not label whether the extra depth is useful or frustrating and does not encode a matching conversion effect. Session is the registered assignment unit.

WATCH FORWeighting deep sessions more heavily through event joins, redefining eligibility from the experiment name, or interpreting no significant conversion difference as equivalence.

07The 7:30 Capacity CallFO-250320validated

ENCODED TRUTHConvenient final-state tables leak the future. The point-in-time target is appointment-window breach; storm exposure and window length contain real signal, while the decision population shifts during the storm. Risk is not treatment effect.

WATCH FORUsing realized arrival/completion facts, optimizing AUC without a capacity policy, or turning technician identity into an employee score.

08The Queue Nobody OwnsNL-241203validated with a deliberate refusal path

ENCODED TRUTHThe current text templates, subjects, categories, and teams are generated independently, while the available team label is final rather than adjudicated initial ownership. Honest clean-holdout routing signal may be insufficient; refusing launch and designing shadow labels is a successful outcome.

WATCH FORUsing later agent messages, accepting final assignment as ground truth, overlooking duplicate interactions, or forcing a classifier to win.

09Forty-Eight Hours of StockSP-251201validated generator

ENCODED TRUTHThe movement table has a technical-event grain: linked scanner replays are not new physical movements. Accepted receipts, not gross dock quantity, post to the ledger; lines may split or remain partial. Position snapshots are sparse, demand lives in ISSUE events, absent intermittent-event days are zeroes, seasonal families genuinely peak, and realized vendor lead times vary from 1–83 days.

WATCH FORCounting scanner replays, using gross receipts, treating sparse positions as a daily spine, dropping zero-demand days, validating randomly through time, or substituting promised lead time for realized uncertainty.