Definition
A dataset is an assembled collection of examples or records with an intended structure and use. It may contain raw observations, transformed features, labels, annotations, metadata, relationships, and defined splits. AI teams use datasets to train, tune, evaluate, monitor, retrieve for, or operate systems.
The same underlying records can form different datasets when selection rules, labels, preprocessing, or versions change. A filename or storage table is therefore not a complete dataset identity. Reproducible use needs a version, lineage, schema, inclusion and exclusion rules, transformations, and a record of how the data was split.
Documentation and evidence
Timnit Gebru and coauthors proposed "datasheets for datasets" in 2018, borrowing the idea from electronics component documentation. Their questions cover why a dataset was created, what it contains, how it was collected and processed, its recommended uses, distribution, maintenance, and legal or ethical concerns.
Documentation is evidence about provenance and intended use, not a guarantee of quality. Teams still need to test representativeness, label reliability, duplication, contamination, drift, and suitability for the decision at hand.
Distinguish it from nearby terms
- A training set is the portion used to fit model parameters. A dataset is not automatically a training set.
- A validation set guides model or threshold selection, while a test set estimates performance after those choices. Reusing either for training weakens the estimate.
- A corpus is a body of material, often text or speech, assembled for study or processing. It becomes a dataset when its records, structure, and intended use are specified.
- A database is an organized storage and query system. It can hold several datasets or continuously changing operational records.
Operational significance
Dataset failures propagate. Duplicated examples can inflate evaluation scores, leaked test records can erase independence, stale labels can encode old policy, and unclear rights can block deployment. Treat dataset versions and transformations as governed artifacts, not anonymous input to the model team.
Check your understanding
A team downloads a public collection and calls it "the training data." Before using it, ask who assembled it, which version was obtained, what rights apply, how records were filtered and labeled, and whether any evaluation examples or near-duplicates are present.