Td Bank AI Data Engineer 

Canada, Ontario, Toronto 

500167585

Toronto, Ontario, CanadaTechnology Solutions

Job Description:

Key Responsibilities:

Data Ingestion and Integration:

· Design, implement, and manage scalable and efficient data ingestion pipelines for various content types.

· Develop ETL (Extract, Transform, Load) processes to ensure high data quality and integrity.

· Ensure seamless integration of data from multiple sources into vector databases.

Vector Database Management:

· Configure, optimize, and maintain vector databases for effective storage and retrieval of high-dimensional content data.

· Implement and optimize indexing and search algorithms to enable efficient and accurate data retrieval.

· Monitor database performance and scalability, addressing any issues promptly.

Knowledge Management:

· Develop and maintain comprehensive documentation for data ingestion workflows, processes, and best practices.

· Implement and enforce data governance policies to ensure data security, compliance, and quality.

· Manage metadata and data catalogs to enhance data discoverability and usability.

· Work closely with cross-functional teams, including data scientists, AI researchers, and software engineers, to understand data requirements and provide technical support.

API Management:

· Understand and utilize API management services (e.g., Azure API Management) for creating, deploying, and managing APIs.

· Develop and maintain APIs to facilitate secure and efficient data access and integration.

· Monitor and optimize API performance, ensuring scalability and reliability.

Performance Optimization:

· Conduct performance tuning and optimization of data ingestion pipelines and vector databases.

· Implement techniques to enhance the speed and efficiency of data processing and retrieval.

· Monitor system performance metrics and make necessary adjustments to meet performance targets.

Security and Compliance:

· Implement security best practices for data ingestion pipelines and vector databases to protect sensitive data.

· Ensure compliance with relevant data protection regulations and industry standards.

CUSTOMER

• Perform data analysis and assess data management requirements for a specific Platform or Journey, including complex analysis involving multiple pods or products
• Maintain expert knowledge of upstream data, including knowledge provided through data profiling, data quality reporting, and via the production of metadata
• Support the acquisition and ingestion of data
• Articulate complex, large scale, and high impact technical design and development details to non-technical business partners.

SHAREHOLDER

• Coordinate with technology work teams such as ITS, ARE, Architecture, Enterprise Protect etc. to ensure overall delivery success
• Support the QA team with dataanalysis/investigationsof complex issues/ test cases as part of SIT/UAT/PAT testing
• Provide oversight on post implementation activities during the warranty period
• Execute & approve code check-in/ check-out into source code repository as part of source code management
• Work closely with ITS/ ARE teams to support code packaging & deployment (CI & CD) into higher environments.

EXPERIENCE AND / OR EDUCATION

· Bachelor’s degree in Computer Science, Data Engineering, Information Systems, or a related field.

· 5-7 years of proven experience in data engineering.

· Experience with designing and implementing large-scaled data processing pipelines.

· Strong understanding of data platform architectures, data modeling, ETL processes, and data pipeline design.

· Experience with cloud platforms (e.g., AWS, GCP, Azure (preferred) and big data technologies.

· Proficiency in programming languages such as Python, Spark, Java, or Scala.

· Experience with vector databases (e.g., Pinecone, Milvus, FAISS) and related tools.

· Understanding of API management services (e.g. Azure API Management) for creating, deploying, and managing APIs.

· In-depth knowledge of CI/CD pipelines and DevOps practices, particularly GitHub Actions.

· Understanding of machine learning concepts, MLOps, and familiarity with AI frameworks.

· Understanding of natural language processing (NLP), machine learning, and RAG.

· Excellent problem-solving skills and attention to detail.

· Strong communication and teamwork abilities.

Please be advised that this job opportunity is subject to provincial regulation for employment purposes. It is imperative to acknowledge that each province or territory within the jurisdiction of Canada may have its own set of regulations, requirements.

If you’re interested in a specific career path or are looking to build certain skills, we want to help you succeed. You’ll have regular career, development, and performance conversations with your manager, as well as access to an online learning platform and a variety of mentoring programs to help you unlock future opportunities. Whether you have a passion for helping customers and want to expand your experience, or you want to coach and inspire your colleagues, there are many different career paths within our organization at TD – and we’re committed to helping you identify opportunities that support your goals.

We will provide training and onboarding sessions to ensure that you’ve got everything you need to succeed in your new role.

Sans Objet