AWS Data Enigneer (Spark and Scala)

Role & responsibilities

1. Should be able to design strategies and programs to collect, store, analyse and visualize data from various sources. 

2) Should be able to develop big data solution recommendations and ensure implementation of the chosen big data solution. 

3) Needs to be able to program, preferably in different programming/scripting languages such as Spark, Python, Java, Pig or SQL. 

4) Proficient knowledge in Big data frameworks Spark, Map Reduce, 

5) Should have an understanding of Hadoop, Hive, HBase, MongoDB and/or MapReduce.

6) Should also have experience with one of the large cloud-computing infrastructure solutions like Amazon Web Services or Elastic MapReduce. 

7) Tuning the Spark Engine for high volume of data ( approx billion records) processing using BDM. 

8) Troubleshoot data issues, deep dive into root cause analysis of any performance issue.