The point where experts and best companies meet
Apple AIML - ML Engineer Safety Human Evaluation United States, Washington, Seattle 475057187
Today
Share
Apple Intelligence is powered by thoughtful data sampling, creation, and curation; high quality, detailed annotations; and application of these data to evaluate and mitigate safety concerns of new generative AI features. This role heavily draws on applied data science, scientific investigation and interpretation, cross-functional communication and collaboration, and metrics reporting and presentation to stakeholders & decision-makers. Responsibilities include:- Develop metrics for evaluation of safety and fairness risks inherent to generative models and Gen-AI features- Design datasets, identify data needs, and work on creative solutions, scaling and expanding data coverage through human and synthetic generation methods- Develop sampling strategies and combine human annotation with auto-grading to deliver high-quality, high-confidence insights at a fast pace and large scale- Work with highly-sensitive content with exposure to offensive and controversial content
- MS or PhD in Computer Science, Linguistics, Cognitive Science, HCI, Psychology, Mathematics, Physics, or a similar science or technology field with a strong basis in scientific data collection and analysis + at least 4 years of relevant work experience, or BA/BS with 8+ years of relevant work experience
- Experience gathering and analyzing language data, image data, and/or multi-modal data, including LLM-generated data
- Strong experience designing human annotation projects, writing guidelines, and dealing with highly multi-labeled, nuanced, and often conflicting data
- Proficiency in data science, machine learning, analytics, and programming with Python & Pandas; strong experience with one or more plotting & visualization libraries
- Ability to collaborate with team members to prioritize competing projects, set and maintain a schedule for milestones and project completions, and communicate with all levels of team members as well as external stakeholders
- Strong skills for rigorous model quality metrics development; interpretation of experiments and evaluations; and presentation to executives
- Experience working in the Responsible AI space
- Curiosity about fairness and bias in generative AI systems, and a strong desire to help make the technology more equitable
- Prior scientific research and publication experience
- Experience working with generative models for evaluation and/or product development, and up-to-date knowledge of common challenges and failures
- Proven track record of contributing to diverse teams in a collaborative environment
- A passion for building outstanding and innovative products. This position involves a wide variety of interdisciplinary skills
Note: Apple benefit, compensation and employee stock programs are subject to eligibility requirements and other terms of the applicable plan or program.
These jobs might be a good fit
