Nvidia Senior Deep Learning Scientist Conversational AI 

United States, California 

330416619

What you’ll be doing:

• Develop, Train, Fine-tune, and Deploy LLMs for driving embodied conversational AI systems including multimodal understanding, speech synthesis, image generation, UI and animation rendering and control, environment interaction, and dialog reasoning and tool systems.

• Apply innovative fundamental and applied research to develop products for embodied conversational artificial intelligence.

• Build novel data driven paradigms for embodied intelligence including customization recipes for different domains and enterprise use cases.

• Develop systems and framework using various data modalities (images, video, text, audio, tactile, etc) and the roles they play in different levels of embodied reasoning and decision making.

• Explore paradigms that can deliver a spectrum of embodied behaviors - from simulated characters to real robots, and from short horizon, low level to long horizon, high level.

• Enable long-horizon reasoning and facilitate low level skills for Embodied AI tasks.

• Apply alignment techniques such as instruction tuning, reinforcement learning from human feedback (RLHF), and parameter efficient fine-tuning such as p-tuning, adaptors, LoRA, and so on to improve use cases.

• Measure and benchmark model and application performance and Analyze model accuracy and bias and recommend the next course of action & Improvements.

• Drive the gathering, building, and annotation of domain specific datasets to train LLMs for different embodied tasks and applications and maintain model evaluation systems and characterize performance and quality metrics across platforms for various AI and system components.

• Collaborate and innovate with various teams on new product features, improvements of existing products and participate in developing and reviewing code, design documents, use case reviews, and test plan reviews.

What we need to see:

• Master’s degree (or equivalent experience) or PhD in Computer Science, Electrical Engineering, Artificial Intelligence, or Applied Math with 5+ years of experience.

• Excellent programming skills in Python with strong fundamentals optimizations and software design.

• Solid understanding of ML/DL techniques, algorithms and tools with exposure to CNN, RNN (LSTM), Transformers (ViT, BERT, BART, GPT/T5, Megatron, LLMs).

• Hands-on experience on conversational AI Technologies

• Experience with Training ViT, BERT, GPT and Megatron Models for different computer vision, NLP and dialog system tasks using “PyTorch” Deep Learning Frameworks and performing data wrangling and tokenization.

• Solid understanding of MLOps life cycle and experience with MLOps workflows & traceability and versioning of datasets including knowhow of database management and queries (in SQL, MongoDB etc).

• Strong collaborative and interpersonal skills, and optimally guide and influence within a dynamic matrix environment.

Ways to stand out from the crowd:

• Fluency in a non-English language - Spanish / Mandarin / German / Japanese / Russian / French / UK English / Arabic/ Korean / Italian / Portuguese.

• Familiarity with GPU based technologies like CUDA, CuDNN and TensorRT.

• Background with Dockers and Kubernetes and deploying machine learning models on data center, cloud, and embedded systems and strong C++ programming skills.

• Experience developing all aspects of large language models.

• Integrating embodied AI systems with various sensor inputs