Skip to content
Outrich Data

Acceptance criteria for a dataset delivery

Updated 2026-11-16 · 7 min read · By the Outrich Data team

Acceptance criteria are the tests a delivered dataset must pass before the buyer accepts it and escrowed funds release. Good criteria are measurable: each names a property, a method, a threshold and who runs the check. Common ones cover volume, schema, time span, de-identification, duplicates and file integrity through a SHA-256 checksum manifest.

Why acceptance criteria decide the closing

In a data sale, payment is tied to acceptance. The license describes the dataset, the seller delivers, and the buyer has a review period to confirm that what arrived is what was described. If funds are in escrow, acceptance is the event that releases them.

This makes the acceptance criteria the most practical clause in the agreement. A vague clause, such as data of satisfactory quality, gives the buyer a free option to reject and gives the seller no way to prove it delivered. A precise clause protects both sides. The seller knows exactly what to build. The buyer knows exactly what it can reject.

Criteria are written before signing and attached to the license as a schedule. They should be drafted by the people who will run them, with counsel checking that the schedule and the main agreement say the same thing.

What makes a check testable

A testable check can be run by a third person who was not part of the negotiation and will produce the same result each time. Four parts make that possible.

Subjective words are the usual cause of disputes. Words such as clean, complete, representative and high quality mean different things to a data engineer and a lawyer. Replace each with a number or a rule. If a property cannot be measured, it belongs in the warranties, not in the acceptance schedule.

  • The property: what is being measured, such as record count or the share of rows with a null in a required field
  • The method: the tool, query or script used, and the version if it matters
  • The threshold: the value that counts as a pass, with any tolerance stated
  • The decision maker: who runs the check and how the result is reported

Volume, schema and time span

Volume is the simplest check and the most often misdefined. State the unit. A count of files, rows, conversations, messages and tokens can all describe the same dataset with very different numbers. Pick the unit the price was based on and set a floor, with a tolerance for the records removed during de-identification and de-duplication.

Schema checks confirm the shape of the data. List the required fields, their types and which may be empty. For JSONL deliveries, every line should parse as one JSON object with the agreed keys. A schema check should also set a maximum rate of empty values in the fields that carry the substance, because a file can match the schema and still be hollow.

Time span checks confirm coverage. State the earliest and latest record dates and, where it matters, the minimum coverage in each period. A dataset sold as several years of history should not arrive with most of its records from the last few months.

De-identification and duplicates

A de-identification check has two halves. The first is a scan: an agreed pattern scanner is run across the full delivery for structured identifiers such as email addresses, phone numbers and account numbers, and the pass mark is set in advance. The second is a manual review of a random sample, because names and context clues in free text escape pattern matching.

State what happens when a single identifier is found. Many agreements treat any direct identifier found in the sample as a failure that triggers the cure period, since one find suggests others. The agreement should also say which identifier classes are in scope, so the check matches the method described in the data card.

Duplicate checks set a ceiling on repeated records. Define a duplicate precisely: an exact match on the full record, a match on a key field, or a match after whitespace and case are normalized. Exact duplicates inflate volume without adding value, and buyers discount them.

Sampling plans

Nobody reads a full dataset by hand, so manual checks rely on samples. A sampling plan says how the sample is drawn, how large it is and what result passes. Without a plan, each side samples the part of the data that supports its view.

A sound plan uses random selection with a recorded seed, so the same sample can be drawn again by either party. It stratifies where the data is uneven, taking records from each source system, each year and each record type. It fixes the sample size before delivery. And it states the pass rule in plain terms, such as no direct identifiers in the sample, or no more than an agreed number of records failing the content review.

A sample supports a statement about likelihood, not certainty. A clean sample lowers the chance of a widespread problem. It does not prove the full dataset is clean, which is why the seller's warranties continue after acceptance.

Checksum manifests

A checksum manifest proves that the files the buyer received are the files the seller sent. SHA-256 is a standard hash function. It reads a file and produces a fixed string of 64 hexadecimal characters. Change one byte of the file and the string changes completely. Two files with the same SHA-256 value are, for practical purposes, identical.

The seller builds a manifest before delivery: a list of every file with its name, size in bytes and SHA-256 value. The manifest travels separately from the data. After transfer, the buyer computes the hashes again and compares them. A match shows nothing was lost, cut short or altered on the way. A mismatch points to the exact file to send again.

The manifest also fixes the delivery in the record. It can be attached to the closing statement, so both sides can later show precisely which files the license covers. The free builder at /tools/delivery-manifest creates a manifest in the browser, and /tools/delivery-verifier checks received files against one. Neither uploads the files.

Example checks and how each is measured

The table shows how common criteria look once they are written to be run. The thresholds are left as agreed values because they depend on the dataset and the price. Outrich Data attaches a schedule in this form to its own purchases.

CheckExample criterionHow it is measured
File integrityEvery delivered file matches the manifestRecompute SHA-256 for each file and compare with the manifest
VolumeAt least the agreed number of conversationsCount records by the unit named in the license
SchemaEvery record has the required fields with the stated typesRun a validator against the agreed schema; zero parse errors
CompletenessEmpty values in key fields stay under the agreed rateProfile each required field and report its fill rate
Time spanRecords cover the agreed start and end datesTake the minimum and maximum of the record date field, with counts by year
De-identification scanNo structured identifiers of the listed classesRun the agreed pattern scanner across the full delivery
De-identification reviewNo direct identifiers in the sampleManual review of a seeded random sample of agreed size
DuplicatesExact duplicates stay under the agreed rateHash each normalized record and count repeats
SecretsNo credentials or keys in the filesRun a secrets scanner across code and text fields
FormatFiles are UTF-8 JSONL, one object per lineValidate encoding and parse every line

When a check fails

The schedule should say what follows a failure. The buyer sends a written rejection that names the failed check and includes the evidence. The seller then has a cure period to correct and redeliver, with a new manifest. The review period restarts for the corrected files.

Not every failure should carry the same weight. Parties often sort checks into those that block acceptance, such as leftover personal data, and those that adjust the price, such as a small shortfall in volume. Writing this down in advance keeps a minor defect from ending a deal both sides still want.

Common questions

Who runs the acceptance checks?
The agreement names the party. In a funds-only escrow the buyer usually runs them and reports the result. Where the dataset is also held in escrow, the escrow holder or an agreed reviewer can run them before the buyer receives the data.
What is a SHA-256 manifest?
A list of every file in a delivery with its size and SHA-256 hash, a 64-character fingerprint of the file's contents. The buyer recomputes the hashes after transfer. If they match, the files arrived complete and unaltered.
Should quality be an acceptance criterion?
Only the parts of quality that can be measured, such as fill rates, duplicate rates and parse errors. Broader statements about accuracy or fitness belong in the warranties, where they are handled by a different remedy.
How long should the review period be?
Long enough to run every check in the schedule, and short enough that the seller is not left waiting. The parties agree the length in advance and usually pair it with deemed acceptance if the buyer stays silent.

Find out if your data qualifies.

Eleven questions, about five minutes, no call required. You get an answer on screen and a written follow-up.

esc