What Is a Quality Gate in Data Collection?

Without a gate, quality is an adjective. A vendor can say the data is clean and a buyer has nothing to test the claim against. A stated gate fixes a measure, a threshold and a consequence, so the delivered files can be checked and the invoice can be tied to what passed. Gates also decide what you pay for: when billing follows validated hours, footage that fails a gate is not part of the count.

Firsthand states gates for sync (2.0 ms reject ceiling), calibration (RMS reprojection under 0.28 px and drift within tolerance), coverage against the spec, frame-level labels, and consent with anonymization. Failing footage goes to a reject log and is excluded from the validated-hour count. Any threshold the site does not give is Not stated and belongs in the spec of a custom or on-demand collection.

A quality gate is a checkpoint with a pass-or-fail rule. Data reaches it, is measured against a stated condition, and either moves on or is held back. In data collection the purpose is to decide, before delivery, which footage counts toward what a buyer receives and pays for.

A gate differs from a general promise of quality because it can be checked. Naming a measure, a threshold and a consequence turns “we do QA” into a condition a buyer can test against the delivered files.

The three terms describe one system from different sides. A quality gate is the check itself. A rejection criterion is the condition that fails it, which the site describes as the opposite framing of an acceptance criterion written into a capture spec. A validated hour is the unit that results: 60 minutes of footage that passed QA against the spec, as opposed to 60 minutes of raw recording.

The site bills on validated hours only.

The site states six families of check: cross-stream sync, calibration, coverage against the capture spec, frame-level labels, no missing or misaligned streams, and consent with anonymization. Two carry numbers, a 2.0 ms sync reject ceiling and an RMS reprojection figure under 0.28 px. The others are stated as rules without a threshold.

The table below marks each threshold as stated or Not stated.

Every stream is hardware-triggered onto one PTP clock domain rather than timestamped in software. Firsthand measures the residual timing offset between streams per episode, against a periodic strobe that is visible to the cameras and detectable in the IMU. An episode that breaches the 2.0 ms ceiling is rejected and logged instead of delivered.

The site also reports measured results, a median of 1.12 ms and a p99 of 1.94 ms. The 2.0 ms ceiling is the gate, the line an episode must stay under.

Each episode is bracketed by an opening calibration solve before capture and a closing solve after it, and both ship with the episode. The gate has two parts: RMS reprojection must stay under 0.28 px, and the measured drift between the two solves must be within tolerance. An episode that fails either goes to the reject log rather than shipping with unverified geometry.

OpenCV’s calibration tutorial explains the quantity: re-projection error gives a good estimation of how exact the found parameters are, and the closer it is to zero, the more accurate they are. The tutorial gives no pass or fail value, so 0.28 px is Firsthand’s own gate and not an OpenCV standard. The drift tolerance is Not stated.

Coverage is checked against the capture spec: its required skill variations and its failure-case coverage. Footage that does not meet the spec’s coverage requirements does not count toward the delivered total.

Labels are checked at the frame level. Hand joints are recorded in metres in the head-camera frame, which only means what it claims if calibration holds, so a frame-level label is checked against calibration and sync before it is trusted, rather than accepted on the strength of the annotation alone. Coverage minimums and label accuracy thresholds are Not stated.

Every contributor and operator signs a documented consent release, and that record ships with the data. Faces, license plates and screens in frame are detected and irreversibly blurred at ingest, and a human reviewer confirms the anonymization before anything ships.

These gates are checked independently of the technical signals. A session without a release, or footage where anonymization is not confirmed, does not clear the acceptance bar however clean the sync, calibration and labels are. This page describes the gate as the site states it and makes no legal claim.

Failing footage is logged in a reject log and excluded from the validated-hour count rather than delivered anyway. On custom collection engagements, rejected items are recaptured at the provider’s cost, and delivery is billed per validated hour, not per raw hour recorded.

Whether a failure is re-run depends on whether the scene can be repeated. A sync or calibration failure on a repeatable task is typically re-captured, because timing and geometric accuracy cannot be restored in post-processing. A consent or anonymization gap is excluded, not patched.

Several figures a buyer might ask for are not on the site: the calibration drift tolerance, minimum coverage per skill variation, label accuracy thresholds, the tolerance for a missing or misaligned stream, the share of episodes rejected, and any pass rate. This page lists each as Not stated rather than estimating it.

Where a figure matters to your model, put it in the spec so a delivered dataset can be checked against it.

Custom and on-demand collections are QA-checked against a written spec, so the spec is where gates are set. The site states a median of 48 hours from a signed spec to the first validated batch where coverage is already deep for the task and environment mix, and three to four weeks where coverage is thin.

The clock ends at a validated batch, one that has passed QA against the spec, not at raw footage.

Name the measure, the threshold and the consequence for each gate. A sync gate names the cross-stream offset and its ceiling, a calibration gate names the reprojection figure and the drift tolerance, and a coverage gate names the skill variations and failure cases required. Then say what happens to failing footage and how it is counted.

A spec that states a sync ceiling, a calibration threshold, the required variations and failure-case coverage, and the consent and anonymization steps gives you something to check a delivered dataset against. To start, request a custom dataset and list the gates you need.

What is a quality gate in data collection?

A stated pass-or-fail check that data must clear before it counts toward a delivered dataset. It names a measure, a threshold and what happens on failure.

Which quality gates does Firsthand state?

Cross-stream sync with a 2.0 ms reject ceiling, calibration with RMS reprojection under 0.28 px and drift within tolerance, coverage against the spec, frame-level labels, no missing or misaligned streams, and consent with anonymization.

What happens to footage that fails a gate?

It is logged in a reject log and excluded from the validated-hour count. On custom collection engagements, rejected items are recaptured at the provider’s cost.

Is a quality gate the same as a validated hour?

No. The gate is the check. A validated hour is 60 minutes of footage that passed QA against the spec, and it is what delivery is billed on.

Does Firsthand state a pass rate or a drift tolerance?

Not stated. The site gives no pass rate, no share of rejected episodes and no numeric calibration drift tolerance, so this page does not supply one.

Can I set my own quality gates in a custom collection?

Custom and on-demand collections are QA-checked against a written spec, so write each gate, its threshold and the consequence into it. Request a custom dataset and list the gates you need.