Skip to content
Outrich Data

Glossary

Data contamination

Data contamination occurs when material intended for testing a model has already appeared in its training data, or when a dataset sold as unseen is in fact present in existing models. Contaminated test results overstate ability, because the model may be recalling answers instead of working them out.

Contamination is hard to see and common. Benchmarks are published, discussed, copied into forums and collected by crawlers. A model trained on a later crawl may have seen the questions and the answers. The same applies to any company data that was ever reachable on the public internet: a help center, a public issue tracker, an open repository or a document shared by an open link.

Buyers check for contamination during diligence. They ask whether any part of the dataset was published or licensed before and how the source systems were secured. Some test directly by asking existing models to continue passages from a sample and seeing whether the output matches. For sellers, the practical steps are to separate material that was ever public from material that was not, to state the history accurately in the data card, and to keep a record of every party that has received a copy. Mixing machine-generated text into a dataset presented as human work is a related problem and should be disclosed in the same way.

A clear statement of history is worth more to a buyer than an optimistic one that later proves wrong.

Find out if your data qualifies.

Eleven questions, about five minutes, no call required. You get an answer on screen and a written follow-up.

esc