The challenge
The customer needed to train an automated takeoff and review system on a large library of mechanical CAD drawings. Existing annotation vendors were either too slow (offshore generalist labelers) or too narrow (single-discipline drafters). The bottleneck was finding annotators who could read multi-discipline drawings and tag fifteen technical fields consistently across 500 sheets.
Our approach
Domain-vetted annotation pod
We assembled a pod of annotators with mechanical and manufacturing engineering backgrounds, plus a senior reviewer with twenty years of drafting experience. Every annotator passed a calibration set before being assigned production work.
Two-pass review
- Pass 1 — primary annotator tags all 15 fields per drawing in our annotation tool.
- Pass 2 — independent reviewer re-tags a 30% audit sample, with disagreements escalated to the senior reviewer.
Disagreement rates were tracked per field so the customer could see where the schema needed sharper definitions.
Daily delivery cadence
Drawings were delivered in daily batches with a rolling QA report. The customer's ML team could re-train every week instead of waiting for a single big-bang delivery.
Outcome
- 500 CAD drawings annotated across 15 technical fields.
- 95%+ field-level accuracy validated against the customer's hold-out set.
- Full program delivered in 30 days, on schedule.
What made it work
The non-obvious lesson was that drawing reading skill mattered more than annotation tooling. Once the right people were on the pod, the throughput took care of itself.
