About the Role

As a Senior Product Manager-GenAI Performance, you will own the end-to-end development of a high-quality evaluation and quality-check platform for AI and LLM-driven applications. This platform will ensure the reliability, scalability, and accuracy of AI solutions across Uber's support and engagement of AI products.

In this role, you will collaborate with engineering, data science, LLM labeling operations, and senior executives to define and execute strategies that ensure AI systems meet the highest quality and performance standards. You will be instrumental in enabling automated, scalable, and explainable AI products.

What You'll do:

• Product Development: Lead the creation and evolution of a high-quality AI performance evaluation and quality check platform. Develop and simulate conversations to evaluate AI models against real-world conditions, identify performance gaps, and iterate to improve accuracy and customer experience.
• Product Ownership: Own the end-to-end product lifecycle by understanding customer pain points, defining product requirements, managing development and testing, and successfully launching scalable evaluation tools.
• Evaluation Frameworks: Design automated and human-centric evaluation frameworks that adhere to industry best practices, ensuring robust quality and reliability across AI systems.
• Cross-functional leadership: Collaborate with engineering, data science, and LLM labeling operations to build and refine AI evaluation tools, fostering strong team alignment.
• Strategic Execution: Partner with executive leaders to determine and execute the product strategy, ensuring alignment with overall business and AI objectives.
• Data-Driven Decisions: Use data insights to continuously refine evaluation frameworks, prioritize product features, and optimize AI performance by addressing risks such as hallucinations or inaccuracies.
• Executive Communication: Present clear and compelling updates on product progress, vision, and impact to senior leaders, securing buy-in and alignment across the organization.

1. Experience: Atleast 5+ years of product management experience, with some experience building enterprise/ SaaS products.
2. Execution Ability: Demonstrated ability to lead cross-functional teams, prioritize complex roadmaps, and deliver impactful AI tools at scale.
3. Data-Driven Approach: Proficient in analyzing AI performance data, leveraging insights to inform product improvements, and addressing quality gaps like hallucinations.
4. Bachelor's degree in Computer Science, Engineering, or a related field. Advanced degrees in AI/ML or related areas are a plus.
5. Strong understanding of AI/ML technologies, including LLMs, fine-tuning, prompt engineering, dataset creation, regression testing, and performance metrics.
6. Experience working closely with machine learning engineers and data scientists to iterate and refine AI models.