FREE Research practice LESSON · Research practice
Design evidence that can disagree with you
Baselines, ablations, reproducibility, and uncertainty
A strong experiment creates a credible path to being wrong.
Write the hypothesis and analysis plan before inspecting the decisive result when possible. Choose competitive baselines, isolate mechanisms with ablations, control data and compute budgets, repeat stochastic runs, and report uncertainty. Preserve environment, code, configuration, data lineage, and deviations so another person can reproduce both success and failure.
Reproducibility is not clerical polish; it is part of the evidential chain.
Exploration and confirmation answer different questions.
Exploration discovers hypotheses and useful representations; confirmation estimates whether a pre-specified claim survives new evidence. Reusing the same data for both without accounting for selection makes chance discoveries look stable. Label phases honestly, keep a final holdout or external replication, and report the path that produced the hypothesis.
Transparency about iteration is stronger than pretending the first idea was inevitable.