The scenario
A feature workbench reconstructs data for historical predictions. Events, arrival times, and later revisions must be interpreted without importing future knowledge into earlier rows.
A failure to reason about
A training row includes a corrected value that was published after the historical prediction. Offline results now benefit from information that was unavailable live.
Illustrative context for the topic. The exercise’s full requirements arrive in the workspace.
Questions to bring to your review
- When did the event occur, and when did it become available?
- How should a later correction affect earlier predictions?
- What makes each historical example independent?
Skills you will practice
- Point-in-time joins
- Training-serving consistency
- Bitemporal data
- Revision and tombstone semantics
- Boundary testing
Read the existing code, reproduce the reported behavior, review proposed changes, and verify your final implementation. You can use AI assistance while staying responsible for the result.
Prepare with a worked example
- ML evaluation and data leakage: inspect the evidence
Work through a confusion matrix and a historical feature example to spot misleading model metrics and unavailable training data.
See how nrml evaluates an attempt, including the limits of automated evidence.