The point where experts and best companies meet
Apple Human Centered AI - Data Scientist ASE Singapore 380901502
25.03.2025
Share
Description
Responsibilities include:- Create evaluation task design and guidelines; identify a relevant data annotation platform to run evaluations at scale.- Implement metrics to measure the effectiveness and accuracy of models to ensure they meet performance standards.- Monitor LLM performance in production environments through human evaluations, identifying trends, and raising alerts when quality degradation occurs. - Perform detailed failure analysis to understand model weaknesses and identify areas for improvement, offering actionable insights to engineers- Maintain high standards for data quality and continuously enhance processes based on both quantitative and qualitative feedback
Minimum Qualifications
- Experience with machine learning concepts, including model evaluation metrics, and data analysis. Proven data analysis expertise using SQL, Python, and Tableau to deliver actionable insights
- Experience with Large Language Models and evaluation techniques
- Fluency in English reading, writing and comprehension skills to partner with international teams
- Fluent in another language then English (reading, writing, and comprehension skills) to support the language specific market
- Expert linguistic and cultural skills for a non English market to accurately represent user experience in early development cycles
Preferred Qualifications
- Ability to analyze complex issues, and identify potential problems with LLM outputs to improve quality with keen attention to detail
- Effective collaboration with cross-functional teams to define ML/LLM evaluation requirements
- Experience crafting, conducting, analyzing, and interpreting experiments and investigations
- Excellent communication skills
- Flexibility to work early morning or late night shift patterns required
These jobs might be a good fit
