Annotation Data Engineer Job at 2T Consulting, Dallas, TX

  • 2T Consulting
  • Dallas, TX

Job Description

We are seeking a detail-oriented and analytical Annotation Data Engineer to support annotation quality, machine learning data operations, and dataset validation for AI/ML initiatives. The ideal candidate will have experience in annotation quality management, data quality engineering, and quality measurement systems, preferably within physical AI, robotics, computer vision, or foundation model development.

Key Responsibilities

  • Define, implement, and maintain annotation quality standards, guidelines, defect taxonomies, and acceptance criteria.
  • Develop and maintain quality metrics and reporting frameworks to measure accuracy, consistency, completeness, rework rates, throughput, and reviewer agreement.
  • Design and execute sampling strategies, annotation audits, escalation workflows, and root-cause analysis processes for large-scale annotation programs.
  • Evaluate generated captions, reasoning outputs, and foundation model training datasets against established quality benchmarks.
  • Analyze annotation quality trends, identify recurring defects, and coordinate corrective and preventive actions with cross-functional teams.
  • Validate dataset releases to ensure compliance with defined quality requirements and readiness for model training.
  • Maintain traceable quality records, audit documentation, issue logs, and continuous improvement plans.
  • Conduct calibration sessions and support reviewer alignment to improve annotation consistency and inter-annotator agreement.
  • Collaborate with annotation operations, data engineering, quality engineering, and AI/ML teams to optimize annotation workflows.
  • Leverage SQL, Python, and reporting tools to analyze quality data, automate measurements, and generate actionable insights.

Required Technical and Functional Skills

  • 2–5 years of relevant experience in annotation quality, machine learning data operations, quality engineering, or data quality management.
  • Experience supporting physical AI, robotics, computer vision, or machine learning programs.
  • Demonstrated experience developing quality metrics, KPIs, and performance measurement systems.
  • Hands-on experience working with large-scale annotation programs and dataset quality assurance processes.
  • Experience evaluating generated captions, reasoning outputs, or foundation model training datasets.
  • Strong understanding of annotation guidelines, sampling plans, acceptance criteria, defect classification, and audit methodologies.
  • Ability to perform quality trend analysis, identify root causes, and drive corrective actions.
  • Working knowledge of SQL and Python for data analysis, quality measurement, and reporting.
  • Experience creating quality dashboards, reports, and performance tracking mechanisms.
  • Strong analytical, problem-solving, documentation, and cross-functional communication skills.

Job Tags

Full time

Similar Jobs

Inova

Certified Registered Nurse Anesthetist (CRNA) Job at Inova

 ...Qualifications: \n \n \n Education: Mastersin Advanced Clinical Practice or graduate of a post masters certificate program for nurse practitioners, OR certificate from an education program that is accredited by the Council of Nurse Anesthesia Educational Programs/... 

Goodwin Personnel Consultants

Port Control Tower Operator Job at Goodwin Personnel Consultants

12 hour rotating Shifts (including nights, weekends, holidays) Must be able to obtain TWIC and pass all required background checks, physical, and drug screen Must have valid TX DL Monitoring and Coordinating Port traffic Documenting and logging arrivals, departures...

ALIGN Executive Search

Regional Nurse Consultant Job at ALIGN Executive Search

 ...Skilled Nursing / Long-Term Care | Greater Naperville, IL $130,000 - $140,000 + Excellent Benefits + Growth Opportunity Are you an experienced Regional Nurse Consultant or accomplished Director of Nursing looking to join a resident-focused organization... 

Hull Lift Truck, Inc.

Payroll & Accounting Specialist Job at Hull Lift Truck, Inc.

 ...Job Description Job Description Hull Lift Truck, Inc. is currently seeking a Payroll & Accounting Specialist at our Elkhart, IN corporate office.The Payroll & Accounting Specialist is responsible for the accurate and timely processing of multi-state payroll using... 

Healthforce

Travel Registered Nurse ER Job Job at Healthforce

 ...Job Description Job Description Overview Registered Nurse ER, Los Angeles, CA. Travel assignment starting 10/26/2026 for 13 weeks. Weekly pay...  ...guaranteed hours. Shift: 3x12-hour nights. Work setting: home healthcare hospice travel assignment. Key Highlights...