Master’s degree or foreign equivalent in Information Technology, Information Science, Computer Science or related field and 2 years of experience in the job offered or related occupation.
2 year of experience with each of the following skills is required:
Building concurrent and parallel processing applications using Scala
ScalaTest testing framework to write unit tests to handle real time user engagement streaming data
Spark core libraries for distributed data processing
Spark SQL and data frames
Design and implement data storage solutions on Hadoop Distributed File System (HDFS).
Monitor, troubleshoot and ensure the performance and reliability of Hadoop environments.
Design ETL (Extract, Transform, Load) processes for data ingestion into Hive
Design and implement high-throughput, fault-tolerant data streaming solutions using Apache Kafka.
Develop and maintain data ingestion pipelines for real-time data processing.