Amazon Software Dev Engineer Contract Artificial General Intelligence 

United States, Washington, Bellevue 

855427464

**This is currently a temporary contract opportunity (12 to 24 months duration) based on business needs**Key job responsibilities
1. Critically analyze and evaluate code responses generated by our LLMs across various programming languages and paradigms
2. Exercise expert judgment to select the most appropriate and efficient code solutions from multiple LLM-generated options
3. Make informed decisions on behalf of our customers, ensuring that selected code meets industry standards, best practices, and specific client needs
4. Develop and write coding demonstrations to illustrate "what good looks like" in AI-generated code, setting benchmarks for quality and efficiency
5. Provide detailed feedback and explanations for your evaluations, helping to refine and improve the LLM's understanding and output7. Stay abreast of the latest developments in software engineering, coding standards, and AI to ensure our evaluations are accurateA day in the life
As a SDE in Amazon AGI, you will play a crucial role in assessing and improving our language models' coding capabilities. Your expertise will be instrumental in evaluating LLM-generated code responses, making high-level judgments, and setting the standard for what constitutes excellent AI-assisted coding.

- Bachelor's degree or equivalent
- 2+ years of non-internship design or architecture (design patterns, reliability and scaling) of new and existing systems experience
- Experience programming with at least one software programming language

- 3+ years of full software development life cycle, including coding standards, code reviews, source control management, build processes, testing, and operations experience
- Master's degree in computer science or equivalent
- Knowledge of professional software engineering & best practices for full software development life cycle, including coding standards, software architectures, code reviews, source control management, continuous deployments, testing, and operational excellence
- Previous experience working with or evaluating AI systems, particularly in the context of code generation
- Understanding of machine learning concepts, particularly as they apply to natural language processing and code generation
- Data analysis skills, writing script to analyze model performance against human ground truth