Design, develop and maintain scalable data pipelines and related solutions.
Build and manage Database solutions to meet data access and reporting Platforms.
Evaluate and recommend tools, technologies, and best practices for data engineering.
Develop comprehensive knowledge of how areas of business, such as architecture and infrastructure, integrate to accomplish business goals
Demonstrating deep technical knowledge and expertise in software development, including programming languages, frameworks, and best practices. Providing guidance and mentorship to junior team members
Optimize data architecture for scalability, performance and reliability.
Monitor, troubleshoot and resolve issues in data pipelines and systems.
Integrate disparate data sources(files, Databases, APIs) into centralized data access platform.
Qualifications:
8 to 12 years of experience in Technology domain
5+ years of relevant experience in Technology implementing data intensive solutions using agile methodologies.
Experience with real time data processing and data virtualization technologies such asStarburst, Snowflake, Flink will be preferred.
Strong experience with data processing (e.g. Spark, Starburst, Snowflake, RedShift,), storage(e.g. Hadoop, MongoDB, Oracle) and ETL(e.g. Airflow, Ab-Initio, Talend) solutions.
Experience with data engineering techniques like building data lakes and data warehouses, data mesh, data pipelines, ETL vs. ELT
Experience with designing and developing distributed systems, handling structured and unstructured data to store, analyze and report.
Experience with developing and maintaining large and complex database systems using relational databases like Oracle, and other big data technologies like Hadoop, Spark etc.
Proficient in PL/SQL and Unix Shell scripts. Strong SQL, PL/SQL development and database tuning experience.
Experience with Data Quality and data lineage.
Experience in designing Online Transaction Processing (OLTP), Operational Data Store (ODS) and Data Warehouse applications.
Consistently demonstrates clear and concise written and verbal communication.
Proven ability to manage multiple priorities and work effectively under pressure.
Education:
Bachelor’s degree/University degree or equivalent experience