מציאת משרת הייטק בחברות הטובות ביותר מעולם לא הייתה קלה יותר
Advanced Data Engineer
Join the industry leader to design the next generation of breakthroughs.
That means changing the way we fly, fueling jets in an eco-friendly way, keeping buildings smart and safe and even making it possible to breathe on Mars.
elevating our strategy to drive advanced analytics and visualization tools across the Commercial enterprise. In this role, Advanced Data Engineer – Commercial, you will design, implement, and manage the data architecture, systems, and processes to effectively collect, store, process and analyze high volume, high dimensional data to provide strategic insight into complex business problems. This will involve creating and maintaining scalable, efficient, and secure data pipelines, data warehouses, and data lakes. You need to ensure consistency in data quality and availability for analysis and reporting including compliance with data governance and security standards.
Duties and Responsibilities
Work in complex data science and analytics projects in support of the Commercial organization.
Work with product owner to identify the data requirements and design/ maintain/ optimize data pipeline to ingest, transform, and load structured and unstructured data from various sources into the data warehouse or data lake.
Design and implement data models and schemas to support analytical and reporting requirements.
Develop and maintain ETL (Extract, Transform, Load) processes.
Administer, optimize, and manage databases, data warehouses, and data lakes to ensure performance, reliability, and scalability.
Create and maintain comprehensive documentation for data architecture, processes, and systems.
Troubleshoot and resolve data-related problems and optimize system performance.
YOU MUST HAVE
6 or more years of relevant experience in Data Engineering, ETL Development, Database Administration.
Experience in Azure Databricks, CI/CD & Dev Ops Process
Expert in scripting and querying languages, such as Python, SQL, PySpark
Experience with both Structured and Unstructured data
Experience in Snowflake
Knowledge of Agile development methodology
WE VALUE
Working with at least one NoSQL system (HBase, Cassandra, MongoDB)
Knowledge of databases, data warehouse platforms (Snowflake) and Cloud based tools.
Experience in using data integration tools for ETL processes.
Knowledge of Data Modelling techniques including schema design for both rational and NoSQL databases
Understanding of Hadoop's ecosystem (including HDFS) and Spark for processing and analyzing large-scale datasets.
Demonstrated experience in cutting-edge packages such as SciKit, TensorFlow, Pytorch, GPT, PySpark, Bit bucket etc.
Proven mentoring ability to drive results and technical growth in peers.
Demonstrated application of statistics, statistical modeling, and statistical process control.
Additional Informationמשרות נוספות שיכולות לעניין אותך