---
title: 'Golden set'
description: 'A curated set of reference examples with reviewed expected outcomes used for evaluation or regression testing.'
canonical_url: 'https://darkfactory.dev/glossary/golden-set'
markdown_url: 'https://darkfactory.dev/glossary/golden-set.md'
collection: glossary
date_published: '2026-08-03T00:00:00-04:00'
date_modified: '2026-08-26T00:00:00-04:00'
---

# Golden set


## Definition

A golden set is a curated collection of reference inputs with reviewed expected outcomes, labels, or scoring rubrics. Teams use it to compare versions, reproduce important failures, and check that behavior the product already supports has not regressed. Google's machine-learning glossary uses the closely related term **golden dataset** for manually curated data that captures ground truth.

"Golden" means trusted enough to serve as a reference, not permanently correct. Labels can be ambiguous, rubrics can omit acceptable answers, and product policy can change. Useful sets preserve provenance, reviewer disagreements, edge-case coverage, version history, and the reason each example belongs.

Do not turn every production incident into an equally weighted test without deciding what behavior it represents. A small set that covers distinct risks and common workflows can be more informative than a large collection of near-duplicates.

## Distinguish it from nearby terms

- A held-out set is defined by separation from development and tuning. A golden set is defined by curation and trusted reference outcomes. A set can be both.
- A regression suite protects behavior the system already demonstrated. A golden set is often used for that purpose, but it may also support capability evaluation.
- Ground truth is the reference outcome for an example. The golden set is the collection and its supporting metadata.

## Check your understanding

Your team reviews the same golden set after every prompt change and edits the prompt until all cases pass. What additional set would you use to detect overfitting?

## Also called

gold set, golden dataset

## Related factory areas

- [Verification, evaluation & quality truth](https://darkfactory.dev/factory/verification)

## Evidence and further reading

- [Google Machine Learning Glossary](https://developers.google.com/machine-learning/glossary/)
- [Demystifying evals for AI agents](https://www.anthropic.com/engineering/demystifying-evals-for-ai-agents)