Citi Group AI/ML Engineer - Lead Analyst 

Canada, Ontario 

609041696

Role Overview:

The AI/ML Engineer - Lead Analyst is a critical role with the objective of exploring, developing, testing, deploying, enhancing, and maintaining machine learning and artificial intelligence capabilities integrated with Internal Audit’s management systems.

A broad understanding of AI/ML concepts and techniques, agile methodologies, cloud computing, intelligent chatbots, and related aspects is essential for the role. Additionally, a key aspect of this role involves the design and refinement of prompts to effectively communicate with generative AI models and to guide them, optimizing their output for a variety of technical applications and producing high-quality and contextually relevant content.

Key Responsibilities:

• Design, test, deploy and maintain machine learning use cases leveraging IBM Watson SaaS environment and toolset, and integrate with the audit management systems
• Enhance existing AI/ML capabilities through ideation, exploration, testing, prioritization, and implementation
• Design, test, and refine prompts to elicit the desired outputs from generative AI models, ensuring clarity, accuracy, effectiveness, and contextual relevance.
• Perform analytics on AI/ML capabilities usage to understand trends, patterns, and pain points and implement a sustainable feedback loop to ensure positive, continually improving user experiences.
• Explain the analysis of results in a methodical, analytical, visually-appealing manner
• Collaborate with Innovation, technology, vendor, and information security teams on updates, enhancements, releases, and other procedural requirements
• Assess current capabilities and chart a path for continuous improvement of the AI/ML capabilities value and impact
• Prepare, manage, and maintain model risk management (MRM) documentation on model performance and accuracy
• Engage with MRM team on adhering to and fulfilling requirements for MRM process
• Engage with relevant stakeholders on use case ideation, planning, development, and deployment
• Apply strong project management skills to develop project plans, timelines, milestones, and demonstrate progress toward objectives
• Utilize excellent communication skills to ideate, educate, and present internally, often at senior levels (across functions); expected to communicate with external parties
• Help shape the direction of machine learning and artificial intelligence capabilities in DSI
• Apply a strong understanding of business processes, risks and controls to identify opportunities to enhance audit efficiencies and effectiveness through the development and delivery of automated processes
• Drive tool innovations based on stakeholder needs, including adoption of new technologies and techniques

Qualifications:

• 5+ years’ experience in AI, machine learning, and model development and deployment
• Extensive experience in machine learning use case design, testing and deployment in IBM SaaS/Watson environment
• Firm understanding of IBM cloud environment and its services
• Demonstrable experience with IBM OpenPages platform, coupled with implementation and integration of machine learning algorithms using Watson capabilities
• Demonstrable experience in working with Generative AI models, with a deep understanding of their capabilities, prompting strategies, model performance, and inherent limitations.
• Extensive experience testing and using APIs
• Strong statistical foundation, with in-depth knowledge of supervised, unsupervised, and semi-supervised machine learning models, with a particular focus on NLP/NLU
• Solid understanding of data structures and data models
• Understanding of conversational AI concepts, including intents, entities, dialogs trees, actions, journeys, and experiences
• Experience in application development and UI and UX considerations
• Experience in analytics, reporting, and visualization for the analysis and presentation of results. Cognos analytics experience is a plus
• Possess an innovative, forward-looking, and imaginative mindset to continually reach for new horizons in audit innovation
• Demonstrable experience working with stakeholders and exceeding their expectations
• Take responsibility and accountability for planning, execution, and delivery
• Positive attitude and a change-driven and change-oriented outlook
• Adept at developing new ideas and improving current processes
• Consider broad implications of decisions to different functions and units
• Ability to thrive in a fast-paced, results-oriented environment
• Excellent communication and interpersonal skills, with ability to develop strong working relationships and foster team collaboration
• Possess excellent writing skills to deliver clear and concise messages, able to articulate complex concepts
• Strong Presentation skills – comfortable with public speaking across various forums and be able to effectively and logically communicate when ideas are being challenged in an open forum
• Familiarity with Financial Services industry and/or auditing a plus

Knowledge and Skills:

• Analytics, AI, machine learning, natural language processing (NLP), natural language understanding (NLU), large language models (LLMs)
• Generative AI prompt engineering and tuning
• Design of AI/ML model performance metrics and optimization
• SaaS environments, cloud computing
• IBM Watson, Watsonx.ai, Watson Assistant, OpenPages, Cognos
• Agile and waterfall project management methodologies; JIRA a plus
• Intermediate to Advanced level Python coding
• Advanced SQL
• MicroStrategy, Power BI, Tableau, or other visualization software
• Intents, entities, dialogs, actions, and related chat bot concepts are a plus

Education:

• Bachelor’s degree in Data Science, Computer Science, MIS, or related discipline.
• CISA, PMP, Agile/Scrum, Cloud Computing, or related certifications are a plus.
• Master’s degree is a plus.

Time Type:

View the " " poster. View the .

View the .

View the