Staff AI Product Analyst

About the role

The Staff AI Product Analyst will define AI-native product analytics for Angi, focusing on prioritized use cases, evaluation disciplines, and a trusted data ecosystem. This role involves designing evaluation frameworks for agentic AI, ensuring alignment on quality and usefulness, and bridging the gap between deep AI capabilities and practical business outcomes. As a senior individual contributor, you will influence the product analytics function by establishing system standards and driving technical investments that yield business value.

Responsibilities

• Partner with AI product, engineering, and domain experts to define north-star and guardrail metrics for flagship AI journeys.
• Align with data science partners on test design, interpretation, and causal framing for new AI products.
• Map agent interactions with metrics, documentation, and APIs to ensure data requirements are structured and testable.
• Design and maintain formal evaluation sets, golden datasets, rubrics, and regression checks for agent outputs.
• Systematically categorize failure modes and feed insights back into prompts, retrieval, and data modeling pipelines.
• Monitor live usage of agentic AI analytics workflows to maintain a prioritized engineering improvement backlog.
• Own internal documentation and playbooks regarding agent usage, human checkpoints, and communication of uncertainty.
• Deliver recurring measurement artifacts to leadership to prioritize roadmaps and reduce high-severity incidents.

Required skills

• Product Analytics
• Data Science
• Technical Product Management
• AI/LLM systems
• Python
• ML frameworks
• Data visualization
• Data governance
• AI evaluation frameworks

Nice to have

• Agentic analytics workflows
• Orchestration frameworks
• Multi-agent systems

Qualifications

• Bachelor's in computer science or a quantitative field
• 10+ years of experience in product analytics, data science, or technical product management
• 1+ years of direct experience working with AI/LLM systems

Benefits

• Performance bonus
• Equity package
• Medical insurance
• Dental insurance
• Vision insurance
• Flexible vacation policy
• Pet discount plans
• 401K retirement plan with company match

About the Company

Angi has powered the home services industry for over 30 years, connecting homeowners with skilled pros. Founded in 1995, the company operates 9 brands across 8 countries with employees worldwide.