As a Data Scientist, you will collaborate on a wide array of product and technical problems with a diverse set of cross-functional partners across Product, Engineering, Research, Data Engineering, Finance and others. You will use data and analysis to identify and solve product development's biggest challenges. You will influence product strategy and investment decisions with data, be focused on impact, and collaborate with other teams. By joining Meta, you will become part of a world-class analytics community dedicated to skill development and career growth in analytics and beyond.One of our company priorities is to lead in AI by developing market-leading models across modalities (text, photo, video, voice). A crucial ingredient for success is being able to understand how our models are performing - to benchmark against competitors, understand performance wins and gaps so we can adjust training/development, inform leadership investment decisions, support open-source rollouts, etc. As we scale and develop models with capabilities across modalities, there is a huge amount of work needed to improve our evaluations framework. This DS will lead that work with partners in Research, Infra, Policy to both operationalize and scale our existing model evaluations so they are easily reproducible, and to develop new frontier evaluations to help us measure new capabilities. You will be part of the AI Foundations pillar within the Gen AI product group. This team owns both developing foundational State of the Art Gen AI technology.Product leadership: You will use data to shape development, quantify new opportunities, identify upcoming challenges, and ensure the products we build bring value to people, businesses, and Meta. You will help your partner teams prioritize what to build, set goals, and understand their product's ecosystem.Analytics: You will guide teams using data and insights. You will focus on developing hypotheses and employ a diverse toolkit of rigorous analytical approaches, different methodologies, frameworks, and technical approaches to test them.Communication and influence: You won't simply present data, but tell data-driven stories. You will convince and influence your partners using clear insights and recommendations. You will build credibility through structure and clarity, and be a trusted strategic partner.
Want more jobs like this?
Get Data and Analytics jobs in Menlo Park, CA delivered to your inbox every week.
Data Scientist, GenAI Responsibilities:
- Leverage internal tools and external APIs to run existing evaluations on Llama models and key competitors to benchmark our performance against the competition.
- Work to operationalize and standardize the evaluation process to make it faster and more reliable. Consult with DE on data infra/dashboarding and building eval datasets
- Extract insights from evaluations and competitor comparisons to feed back into model training and development
- Work with research to develop new frontier evals, both to measure new modalities/capabilities (e.g., agentic models, speech generation, video generation) and to better assess core existing capabilities like coding, math, general knowledge
- Do foundational work to assess how we can measure AGI
- Bachelor's degree in Computer Science, Computer Engineering, relevant technical field, or equivalent practical experience.
- 7+ years of work experience leading analytics work in IC capacity, working collaboratively with Engineering and cross-functional partners, and guiding data-influenced product planning, prioritization and strategy development
- Experience working effectively with multiple stakeholders and cross-functional teams, including Engineering, PM/TPM, Analytics and Finance
- Experience framing and communication skills
- Intellectual curiosity and drive, decisiveness, with ambiguity and complexity
- Experience doing product analytics on consumer facing digital products, or working on AI Infra and Privacy related products.
- Experience working with researchers and infrastructure teams.
- Experience with Python.
- Experience with predictive modeling, machine learning, and experimentation/causal inference methods.
- Masters or Ph.D. Degree in a quantitative field
Meta builds technologies that help people connect, find communities, and grow businesses. When Facebook launched in 2004, it changed the way people connect. Apps like Messenger, Instagram and WhatsApp further empowered billions around the world. Now, Meta is moving beyond 2D screens toward immersive experiences like augmented and virtual reality to help build the next evolution in social technology. People who choose to build their careers by building with us at Meta help shape a future that will take us beyond what digital connection makes possible today-beyond the constraints of screens, the limits of distance, and even the rules of physics.
Meta is proud to be an Equal Employment Opportunity and Affirmative Action employer. We do not discriminate based upon race, religion, color, national origin, sex (including pregnancy, childbirth, or related medical conditions), sexual orientation, gender, gender identity, gender expression, transgender status, sexual stereotypes, age, status as a protected veteran, status as an individual with a disability, or other applicable legally protected characteristics. We also consider qualified applicants with criminal histories, consistent with applicable federal, state and local law. Meta participates in the E-Verify program in certain locations, as required by law. Please note that Meta may leverage artificial intelligence and machine learning technologies in connection with applications for employment.
Meta is committed to providing reasonable accommodations for candidates with disabilities in our recruiting process. If you need any assistance or accommodations due to a disability, please let us know at accommodations-ext@fb.com.