IBM Data Scientist - Model Evaluation 

Poland, Lesser Poland Voivodeship, Krakow 

457438069

In this position, you’ll be immersed in the forefront of the AI revolution. Your duties will include critically assessing generative AI models and processes/enhancements related to them, establishing quality benchmarks, and challenging models’ capabilities.

Your role and responsibility will encompass:

• Critically assessing and pushing the boundaries of generative AI models and tuning of such models, focusing on LLM models
• Establishing and upholding stringent quality benchmarks
• Collaborate with the research team to develop state-of-the-art NLP solutions utilizing LLM models

A successful candidate for this role should demonstrate:

• An intrinsic motivation, coupled with an insatiable appetite for continuous learning
• Proficiency in crafting and sustaining tools to amplify productivity
• Robust analytical capabilities and a keen problem-solving acumen
• A tenacity to assume responsibility, driving problems to their resolution and tasks to fruition
• Proficiency in collaborating seamlessly within diverse team dynamics

Required Technical and Professional Expertise

• Professional experience in AI/ML model development and evaluation with solid foundation in linear algebra, statistics, and optimization
• Familiar with automated ML models evaluations techniques, including Large Language Models (LLMs)
• Familiar with advanced model training methodologies including prompt tuning and fine-tuning with foundational models
• Proficient in Python, with experience in coding and debugging in Python and Jupyter Notebook
• Demonstrated ability to critically assess and challenge AI (including LLM) models to ensure optimal performance
• Fluent in written and spoken English

Preferred Technical and Professional Expertise

• Expertise & up-to-date knowledge of advances in Foundation Models area, focusing on Large Language and Multimodal Models
• Design, develop NLP models to extract insights and information from unstructured data
• Background in developing and implementing test automation frameworks and strategies
• Experience with cloud environments