Product Operations Manager, Model Quality

About This Role

This Product Operations Manager role focuses on driving model quality across all Meta surfaces, joining the Product Operations Foundations team. The position involves building and operating AI-driven workflows and owning complex quality programs end-to-end, with a focus on managing and evaluating AI solutions and infrastructure to improve precision, prevent drift, and maintain real-time observability. The role is suited for someone energized by setting strategy for LLM models, determining investment areas for accuracy, advising leadership on risks, and defining roadmaps for the future of AI work.

Responsibilities

• Define the technical direction for model maintenance (retraining cadences, drift mitigation, performance recovery) and evolution (new capabilities, architecture improvements, multi-modal expansion)
• Translate cross-product performance patterns into investment recommendations for evaluation leads
• Provide cross-product context, define what good looks like at the model level, and inform evaluation methodology
• Ensure consistency and identify gaps across the portfolio while building institutional competence by surfacing performance patterns and proven methodologies
• Define what leadership needs to see, how model health should be measured and reported, and what thresholds trigger escalation
• Provide thought partnership to evals managers on narrative of model health, provide visibility into classification strategy and accuracy measurement process
• Work with evaluation managers to drive cross-app taxonomy alignment in alignment with cross-functional needs and advise on a strategy for the migration of LLM accuracy assessment to judges
• Own the consolidated view of all production model performance, identify systemic patterns and emerging risks, and ensure leadership can verify model health on demand
• Partner with AI Implementations, operational systems teams and the Metrics & Measurement team to build and maintain the infrastructure that surfaces this information
• Establish performance guardrails that evals captains implement
• Continuously scan industry developments and best practices to incorporate into org-wide approach
• Maintain a tight feedback loop with product and eng teams across apps to ensure alignment on production priorities and deployment risks
• Deploy deep SME expertise to diagnose, unblock and directly resolve technical bottlenecks to complex model quality problems (atrophy, accuracy regressions, performance plateaus) when evaluation leads encounter blockers they cannot resolve independently
• Drive alignment with cross-functional teams (quality and reliability partner teams) on tooling needs to support Product Operations classification strategy (ML classification tooling for initial-tier classification, user voice, breakdown graphs)
• Advocate for investment, flag risks, influence direction