FREE LESSON · AI & machine learning · 1 OF 8
Machine Learning Problem Framing and Data Leakage
Start with the decision, not the model: Targets, data, baselines, and leakage
Machine learning estimates a useful mapping from evidence.
Define who uses the output, which action it changes, when features are available, and how success is measured. The label is a proxy for the real outcome and can encode historical bias or process quirks. A simple heuristic or linear baseline reveals whether complexity earns its cost. Leakage occurs when training sees information unavailable at decision time.
A precise prediction target includes population, timestamp, horizon, action, and consequence.
The baseline is a scientific control
For rare-event prediction, always guessing “no” can have high accuracy while providing no value. Compare against prevalence, a business rule, or a simple model using the metric and cost that matter. If a complex model barely improves decisions, its latency, maintenance, and failure surface may not be justified.
Complexity must buy measured decision value, not merely a more impressive architecture.
Sources and further reading
- CS229 Machine Learning course materials: Stanford University
- The Elements of Statistical Learning: Hastie, Tibshirani & Friedman · authors’ open edition