Definition
Evaluation examples deliberately withheld from training, prompt tuning, workflow design, or agent feedback to reduce leakage and overfitting.
Distinguish it from nearby terms
Held-out describes isolation; hidden describes visibility; either can still have label errors or distribution mismatch.
Check your understanding
If an agent can inspect or iteratively optimize against the set, it is no longer a clean holdout.