Skip to content
Outrich Data

Glossary

Held-out set

A held-out set is a portion of data deliberately kept separate from training so it can be used to test a model fairly. Because the model has not learned from these examples, its performance on them indicates how well it will handle new cases. Keeping the separation intact is essential.

The simplest held-out set is a random slice of a dataset set aside before training begins. More careful splits respect the structure of the data. If records from one customer appear in both training and test portions, the model may recognize the customer and appear better than it is. Splitting by customer, by time period or by project avoids this kind of leak.

In data licensing, held-out has a second meaning that matters to price. It describes data that has been kept away from all model training, not only one model's. A company dataset that was never published, never scraped and never licensed is held out with respect to the whole field. Buyers pay for that status and ask the seller to represent it. A seller can protect it by tracking where every copy of the data has gone and by considering whether to reserve part of a dataset for evaluation licensing when selling the rest for training. Once a held-out set has been used for training anywhere, the status is lost.

Buyers verify the claim where they can, so the history given in the data card should be accurate and complete.

Find out if your data qualifies.

Eleven questions, about five minutes, no call required. You get an answer on screen and a written follow-up.

esc