Skip to content
Outrich Data

Glossary

Training data

Training data is the collection of examples a machine learning model learns from. During training, the model adjusts its internal parameters to predict or reproduce patterns in those examples. The quality, variety and relevance of the training data largely determine what the model can do and where it fails.

Large language models are first trained on very large bodies of text to learn general language and knowledge. This stage is often called pre-training. They may then be trained further on narrower data for particular domains or tasks. Each stage uses training data, but the amounts and the standards differ. Pre-training tolerates noise because of scale. Later stages need cleaner, well-structured examples.

For companies with data, the relevant fact is what labs lack. The public web has been widely collected. Records of real work inside organizations have not, and they show processes that finished public documents do not. That is why operational data such as support threads, reviewed documents and workflow logs is licensed. A license for training should say which kinds of training are permitted, since pre-training, fine-tuning and evaluation are commonly listed separately. Sellers should also understand that training cannot be reversed. Files can be deleted when a license ends, but what a model learned from them remains in its parameters.

Buyers judge training data on four things: how unique it is, how much there is, how clean and well structured it is, and how clearly the right to use it is documented.

Find out if your data qualifies.

Eleven questions, about five minutes, no call required. You get an answer on screen and a written follow-up.

esc