Evaluation and reliability

Golden set

working definition

Definition

A curated collection of examples with trusted expected outcomes used for regression testing or evaluation.

Distinguish it from nearby terms

A golden set is trusted reference data; a held-out set is defined by separation from tuning, whether or not labels are perfect.

Check your understanding

Golden sets require maintenance when reality, policy, or product behavior changes.