Microsoft AI (MAI) is seeking experienced Data Scientists to help build Copilot memory and personalization — AI that remembers, evolves, and grows with each user. You’ll work in a highly collaborative, fast-paced environment to develop systems that deepen memory with every interaction, personalize experiences to reflect each user’s unique style and goals, and make interactions feel more like working with a trusted partner than using a tool.
By applying to this U.S. Mountain View, CA you are required to be local to the San Francisco area and in office 3 days a week.
Responsibilities
In this role, you will:
- Develop and improve evaluation methodologies to assess model output quality, for both machine eval and human eval metrics and coverage.
- Design and implement scalable data pipelines to extract, transform, and structure product logs for evaluation use cases.
- Stay current on the latest in LLM research on evaluation and prompting.
- Drive actionable product insights, opportunity analyses, and metric tracking to guide product direction and success.
- Drive new ways of instrumentation and measurement approach to evaluate new feature performance through experimentation.
- Design and maintain core data pipelines and conduct hands-on analysis of large-scale telemetry data using advanced algorithms and tools.
- Articulate insights, storyboard with data and communicate to influence leadership and other key decision makers.
- Stay current on the latest in LLM research on evaluation and prompting.
- Enjoy working in a fast-paced, design-driven, product development cycle.
Required Qualifications
- Doctorate in Data Science, Mathematics, Statistics, Econometrics, Economics, Operations Research, Computer Science, or related field AND 1 year(s) data-science experience (e.g., managing structured and unstructured data, applying statistical techniques and reporting results)
- OR Master's Degree in Data Science, Mathematics, Statistics, Econometrics, Economics, Operations Research, Computer Science, or related field AND 3 years data-science experience (e.g., managing structured and unstructured data, applying statistical techniques and reporting results)
- OR Bachelor's Degree in Data Science, Mathematics, Statistics, Econometrics, Economics, Operations Research, Computer Science, or related field AND 5 years data-science experience (e.g., managing structured and unstructured data, applying statistical techniques and reporting results)
- OR equivalent experience.
- 1 year customer-facing, project-delivery experience, professional services, and/or consulting experience.
Preferred Qualifications
- Doctorate in Data Science, Mathematics, Statistics, Econometrics, Economics, Operations Research, Computer Science, or related field AND 5 year(s) data-science experience (e.g., managing structured and unstructured data, applying statistical techniques and reporting results)
- OR Master's Degree in Data Science, Mathematics, Statistics, Econometrics, Economics, Operations Research, Computer Science, or related field AND 7 years data-science experience (e.g., managing structured and unstructured data, applying statistical techniques and reporting results)
- OR Bachelor's Degree in Data Science, Mathematics, Statistics, Econometrics, Economics, Operations Research, Computer Science, or related field AND 10 years data-science experience (e.g., managing structured and unstructured data, applying statistical techniques and reporting results)
- OR equivalent experience.
- Experience in designing core data pipelines and conduct hands-on analysis of large-scale telemetry data using advanced algorithms and tools.
- Experience building or evaluating LLM applications in production.