Data, model, evaluation

What actually determines the behavior of a learned system?

Modern capability joined large datasets, labeling labor, accelerator compute, model architecture, optimization, evaluation, and post-training. Failures exposed that aggregate benchmark performance can hide uneven behavior and unexamined data choices.

Learned behavior is produced by the complete data-objective-model-evaluation system, so no model result can be interpreted apart from its evidence pipeline.

Reconstruct the mechanism

1. Define a taxonomy and sample data through human and institutional choices
2. Fit a parameterized architecture against an objective using compute
3. Evaluate on held-out tasks, groups, and failure slices
4. Use feedback or policy to shape deployment behavior and monitor change

Train the same small classifier on two differently sampled datasets, report aggregate and subgroup metrics, write a dataset card, and state which conclusion the evidence does not support.

Evidence and uncertainty

This is provisional history involving living people and contested institutions. Teach documented mechanisms and lineages, add review dates, and avoid settled heroic verdicts.

• ImageNet — IEEE CVPR · primary paper
• Attention Is All You Need — NeurIPS
• Gender Shades — ACM FAccT program archive