EY AI Testing-DET-Manager 

India, Karnataka, Bengaluru 

393260605

Key Responsibilities:

• Leadership and Team Management:
  • Lead and mentor a team of QA engineers focused on AI and machine learning projects.
  • Foster a collaborative and innovative team culture.
  • Conduct performance evaluations, provide feedback, and support career development for team members.
• Testing Strategy Development:
  • Design and implement comprehensive testing strategies for AI applications, including functional, integration, regression, and performance testing.
  • Develop and maintain test plans, test cases, and test scripts tailored to AI-specific challenges.
  • Ensure compliance with industry standards and best practices in AI testing.
• Collaboration with Cross-Functional Teams:
  • Work closely with data scientists, developers, and product managers to understand requirements and provide insights on testing needs.
  • Participate in design reviews and provide input on potential testing challenges and considerations.
• AI Model Validation:
  • Define and execute methodologies for validating AI models, including bias detection, performance evaluation, and accuracy testing.
  • Establish metrics and KPIs to assess the effectiveness of AI systems and ensure they meet specified requirements.
  • Experience in testing LLM models
• Test Automation and Tools:
  • Oversee the implementation and maintenance of automated testing frameworks and tools specific to AI applications.
  • Stay updated on emerging tools and technologies in AI testing and recommend their adoption as appropriate. Knowledge on explainability testing tools (e.g., LIME, SHAP etc.)
  • Experience in any of the AI testing platforms and tools (e.g., Functionize, Applitools, Testim etc.)
  • Knowledge on cloud platforms & services for ML Ops / LLM Ops
• Documentation and Reporting:
  • Maintain detailed documentation of testing processes, results, and lessons learned.
  • Prepare and present reports to stakeholders on testing progress, risks, and outcomes.
• Continuous Improvement:
  • Identify opportunities for process improvement in testing and development methodologies.
  • Advocate for best practices in AI testing across the organization.
• Risk Management:
  • Identify potential risks in AI models and applications and develop mitigation strategies.
  • Ensure timely communication of testing issues and risks to project stakeholders.

Qualifications:

• Education: Bachelor’s degree in Computer Science, Engineering, or a related field; Master’s degree preferred.
• Experience:
  • 10+ years of experience in software testing / development with exposure to Python
  • 1-2 years of experience in defining test strategy and plan for AI/ML applications
  • 2+ years in a managerial or leadership role.
• Technical Skills:
  • Strong understanding of AI/ML concepts, algorithms, and frameworks (e.g., TensorFlow, PyTorch).
  • Knowledge on ML / LLM frameworks & libraries
  • Proficiency in programming languages commonly used in AI testing (e.g., Python, R).
  • Experience with test automation tools (e.g., Selenium, TestNG) and frameworks.
  • Familiarity with data validation, model evaluation, and statistical analysis.
• Soft Skills:
  • Excellent communication and interpersonal skills.
  • Strong problem-solving abilities and analytical thinking.
  • Ability to work collaboratively in a fast-paced, dynamic environment.

EY exists to build a better working world, helping to create long-term value for clients, people and society and build trust in the capital markets.