Red hat Senior Principal Chief Architect United States, North Carolina, Raleigh 58572639
Share
What you will do
Testing the performance and accuracy of LLMs for vLLM and llama.cpp inference on different accelerators.
Making awesome test plans and cases to hit product requirements.
Doing all sorts of testing: functional, performance, regression, you name it, to make sure the product is solid.
Writing test code and frameworks to automate testing.
Monitoring, analyzing, and reporting test results and failures.
Sharing your knowledge and recommendations to help the team keep getting better.
Keeping everyone in the loop about quality efforts.
Giving good and quick code reviews.
What you will bring
At least 3 years of software testing experience.
Solid experience evaluating LLMs for performance on accelerators and accuracy (think HellaSwag, MMLU, Chatbot Arena, TruthfulQA, etc.).
Being super comfortable with Python and PyTest is a must.
Familiarity with Git, GitHub, or GitLab.
Strong experience with API and performance testing, especially for C++ and Python.
You should be a pro with Docker, Podman, and Kubernetes or Openshift.
Highly experienced in setting up CI/CD processes like Jenkins and GitHub Actions.
Understanding of core Machine Learning algorithms and basics
Bonus points if you have:
A Bachelor’s degree (or higher) in computer science, math, or a related field is cool, but practical experience and technical skills are what really matter.
Knowing how to excel in Open Source communities.
Understanding how application build pipelines work (like how code becomes binaries or Python wheels).
A track record of contributing to the vLLM community is ahugeplus!
The salary range for this position is $133,650.00 - $220,680.00. Actual offer will be based on your qualifications.
Pay Transparency
● Comprehensive medical, dental, and vision coverage
● Flexible Spending Account - healthcare and dependent care
● Health Savings Account - high deductible medical plan
● Retirement 401(k) with employer match
● Paid time off and holidays
● Paid parental leave plans for all new parents
● Leave benefits including disability, paid family medical leave, and paid military leave
These jobs might be a good fit
