Expoint – all jobs in one place
Finding the best job has never been easier
Limitless High-tech career opportunities - Expoint

Red hat Senior Software Engineer - Model Inferencing 
United States, North Carolina, Raleigh 
276355981

Yesterday

What You Will Do

  • Develop and maintain a high-quality, high-performing ML inference runtime platform for multi-modal and distributed model serving.

  • Contribute directly to upstream inference runtime communities such as vLLM (https://github.com/vllm-project/vllm) , TGI (https://github.com/huggingface/text-generation-inference) , PyTorch (https://github.com/pytorch) , OpenVINO (https://github.com/openvinotoolkit/openvino) , and others.

  • Maintain CI/CD build pipelines for container images that allow faster, more secure, reliable, and frequent releases

  • Coordination and communication with various stakeholders

  • Applying a growth mindset by staying up to date with AI and ML advancements

What You Will Bring

  • Highly experienced with programming in Python and PyTorch

  • Familiarity with model parallelization, quantization, and memory optimization using vLLM, TGI, and other inference libraries.

  • Experience with Python packaging, such as PyPI libraries

  • Development experience with C++, especially with the CUDA APIs, is a big plus

  • Solid understanding of the fundamentals of model inference architectures

  • Experience with Jenkins, Git, shell scripting, and related technologies

  • Experience with the development of containerized applications in Kubernetes

  • Experience with Agile development methodologies

  • Experience with Cloud Computing using at least one of the following Cloud infrastructures: AWS, GCP, Azure, or IBM Cloud

  • Ability to work across a large, distributed, hybrid engineering team

  • Experience with open-source development is a plus

The salary range for this position is $116,270.00 - $191,840.00. Actual offer will be based on your qualifications.

Pay Transparency

● Comprehensive medical, dental, and vision coverage

● Flexible Spending Account - healthcare and dependent care

● Health Savings Account - high deductible medical plan

● Retirement 401(k) with employer match

● Paid time off and holidays

● Paid parental leave plans for all new parents

● Leave benefits including disability, paid family medical leave, and paid military leave