Hadoop, Spark, Scala

esired Competencies (Technical/Behavioral Competency)

Must-Have • Big Data/Hadoop Experience particularly in ingesting data and implementing Data ingestion pipelines, SQOOP, HADOOP, HDFS, HIVE, IMPALA, Java, Scala, Spark • Scala is Mandatory Data Engineer with below responsibilities: • Lead Data Engineer to build data pipelines to support implementation of data science and analytics use cases. • Candidate needs to be able to develop code in the corresponding language (see technical skills), test it, and follow up with the implementation into Production environment. • Candidate will also take part in the solution design phase, so experience in analysis requirements is desirable. • Technical expert to lead a squad of engineers for a Data Product and implement data transformation projects in Hadoop. • Candidate will lead a team of 3-5 data engineers with minimum supervision and integrate with wider IT and business project teams.

Good-to-Have Good exposure to Unix and HDFS commands Experience on Pyspark will be an added advantage Experience working on Data Analytics will be an added advantage Experience on SAS will be an added advantage