Amazon Language Data Scientist II AWS AI Transcribe 

United States, California 

291699300

We are looking for candidates who have a strong interest in applying their linguistic and data science expertise in collecting and analyzing natural language and human data to lead multiple data collection and data analysis efforts at a time in an exciting, fast-paced environment. In this role, you will curate, engineer, and analyze natural language datasets in the medical domain, as well as design experiments and surveys to elicit human-in-the-loop insights, which are critical for developing AI-powered language applications. You will partner closely with talented language engineers, program managers, clinical experts, applied scientists, engineers, and product managers to deliver data solutions that meet customer needs.Key job responsibilities
- Translate business, modeling and ethical requirements in Health AI into executable data collection projects
- Design human-in-the-loop evaluation tasks to measure the performance and usability of models in the medical domain
- Develop the materials necessary to execute successful data collection efforts such as guidelines, annotation interfaces, quality assurance workflows
- Support the sourcing and/or creation of high-quality language datasets and language artifacts for feature and language expansion
- Analyze structured and unstructured data to provide actionable recommendations to improve data quality or model performance
- Iterate and innovate on data collection methodologies to improve data turnaround time and reliability
- Incorporate LLMs, prompt engineering, and ML techniques to automate repetitive annotation and data creation workflows
A day in the life
As a Language Data Scientist on the data team, you will take the lead on a couple of critical data projects related to AWS HealthScribe and Amazon Transcribe Medical, diving deep and developing materials to drive these projects forward. You will consult with stakeholders to understand the role data plays in developing and launching specific language services that meet customer needs. You will propose data collection and annotation strategy to curate and validate datasets, and partner with stakeholders to set the quality bar for these data projects. You will lead iterative data analysis and course correction efforts to address gaps and quality issues. You will also design experiments to elicit human-in-the-loop insights (including users who are domain experts) for model evaluation and usability testing, proposing the optimal business and evaluation metrics to use.You will gradually expand your scope by applying the principles of data-centric AI to conduct experiments to drive workflow and process improvements, with the goal to optimize the cost and quality of data. Leveraging your hands-on data analytics skills and up-to-date knowledge of machine learning (ML) and Generative AI techniques, you will collaborate with other language engineers and scientists to propose new strategies for sourcing ground truth data or generating reliable metrics, automating where appropriate.
Diverse Experiences
AWS values diverse experiences. Even if you do not meet all of the qualifications and skills listed in the job description, we encourage candidates to apply. If your career is just starting, hasn’t followed a traditional path, or includes alternative experiences, don’t let it stop you from applying.Mentorship & Career Growth
We’re continuously raising our performance bar as we strive to become Earth’s Best Employer. That’s why you’ll find endless knowledge-sharing, mentorship and other career-advancing resources here to help you develop into a better-rounded professional.Work/Life Balance

- 2+ years of data scientist experience
- 3+ years of data querying languages (e.g. SQL), scripting languages (e.g. Python) or statistical/mathematical software (e.g. R, SAS, Matlab, etc.) experience
- PhD in a language and human behavior related field with a strong quantitative component (e.g., Cognitive Linguistics, Sociolinguistics, Human-Computer Interaction); or, a Master’s degree with 3+ years of field experience
- Experience in data mining and cleaning for NLP machine learning model pipelines
- Experience in language data collection for quantitative analysis, including guidelines, workflow design
- Experience in research and experimental design involving human participants
- Experience in statistical measures for data quality assessment and research hypotheses testing
- Practical knowledge of data labeling tools and techniques (e.g., Amazon SageMaker Ground Truth, brat, ELAN)
- Excellent knowledge of semantics, pragmatics, conversation analysis, and/or discourse analysis
- Ability to explain complex concepts and solutions in easy-to-understand terms

- Experience with LLMs and prompt engineering techniques and other programmatic approaches to annotation, including weak supervision and active learning
- Practical knowledge of version control systems (e.g. Git)
- Experience with spoken data collection, speech analysis, speech transcription (from scratch or ASR-assisted)
- Experience working with clinical or medical data, such as medical transcriptions, clinical notes, or electronic health records (EHRs)
- Knowledge of healthcare terminology and medical ontologies (e.g., SNOMED CT, ICD, RxNorm)