Key job responsibilities- Construct resilient and scalable data pipelines using SQL/PySpark/Airflow to effectively ingest, process, and transform substantial data volumes from diverse sources into a structured format, ensuring data quality and integrity.
- Devise and implement an efficient, scalable data warehousing solution on AWS, utilizing appropriate NoSQL/SQL storage and database technologies for both structured and unstructured data.
- Automate ETL/ELT processes to streamline data integration from diverse sources, enhancing the platform's reliability and efficiency.- Enable advanced analytics and machine learning capabilities within the platform, extracting predictive and prescriptive insights through tools like EMR/SageMaker Notebooks.
- Implement robust security measures and ensure data compliance with internal requirements, industry standards, and regulations to safeguard sensitive information.
- Collaborate closely with data scientists and business intelligence engineers to comprehend their requirements and work together on data-related projects.
- Generate comprehensive technical documentation covering the platform's architecture, data models, and APIs, promoting knowledge sharing and ease of maintainability.A day in the life
As a machine learning engineer, you will collaborate with scientists and data engineers on developing and evaluating machine learning models using large datasets to improve the customer experience through better recommendations. You will own scaling up successful prototypes and implementing a reliable automated production workflow for the model.
- 3+ years of non-internship professional software development experience
- 2+ years of non-internship design or architecture (design patterns, reliability and scaling) of new and existing systems experience
- Experience programming with at least one software programming language
- 3+ years of full software development life cycle, including coding standards, code reviews, source control management, build processes, testing, and operations experience
- Bachelor's degree in computer science or equivalent
משרות נוספות שיכולות לעניין אותך