Proctor Quality and Learning Manager, Scaling Operations

About the role

The Proctor Quality and Learning Manager will own the end-to-end quality standards and learning systems for the Duolingo English Test proctor network. This role involves defining accuracy standards, managing calibration and adjudication programs, and developing training to ensure consistent decision-making as security threats and workflows evolve. You will partner with cross-functional teams to validate human and AI-assisted workflows and provide functional leadership to training specialists and adjudicators.

Responsibilities

• Own the DET proctoring quality standard by maintaining rubrics, decision rules, and guidance.
• Lead calibration and adjudication programs, including recurring certification cycles and resolving disputed decisions.
• Build and run an ongoing learning program, including monthly interventions and targeted training.
• Measure effectiveness using audits and performance trends to identify learning needs and evaluate training impact.
• Guide specialist execution by setting expectations, assigning deliverables, and overseeing onboarding.
• Maintain durable sources of truth such as calibration and training runbooks.
• Evolve quality standards in partnership with Psychometrics, Security, Operations, Product, and Engineering.
• Represent the proctoring craft in product experiments and workflow decisions.
• Establish benchmarks for AI-assisted workflows and validate automated capabilities.

Required skills

• Assessment development
• Quality assurance
• Calibration programs
• Adult-learning program design
• Analytical fluency
• Rubric design
• Human-technology workflow validation

Nice to have

• Language assessment
• Test security
• Remote proctoring
• Trust and safety
• Fraud prevention
• Inter-rater agreement
• Learning-management systems

Qualifications

• Experience in assessment development, scoring/rater quality, certification, or quality assurance
• Experience in adult-learning programs

About the Company

Duolingo's mission is to develop the best education in the world and make it universally available. The company focuses on educating users, experimenting with big ideas, making fact-based decisions, and finding innovative solutions to complex problems.