Lead Data Engineer - Spark/Python

Responsibilities for this position include :

- Provides technical leadership in Big Data space (Hadoop Stack like M/R, HDFS, Pig, Hive, HBase, Flume, Sqoop, NoSQL stores like Cassandra, HBase etc) across Fractal and contributes to open source Big Data technologies

- Write and tune complex Java, MapReduce, Pig and Hive jobs

- Adapt quickly to change in requirements and be willing to work with different technologies if required

- Experience leading a Backend/Distributed Data Systems team while remaining hands-on is very important

- Lead the effort to build, implement and support the data infrastructure

- Manage the business intelligence team and vendor partners, ensuring to prioritize projects according to customer and internal needs, and develops top-quality dashboards using industry best practices

- Manage team of data engineers (both full-time associates and/or third party resources)

- Own the majority of deliverables for the Big Data team from a delivery perspective

- Analyzes and confirms the integrity of source data to be evaluated

- Leads in deployment and auditing models and attributes for accuracy

Education for Lead Data Engineer :

Have a relevant degree such as Bachelor's and Master's Degree in Computer Science, Engineering, Statistics, Education, Technical, Information Technology, Information Systems, Mathematics, Computer Engineering, Management

Skills for Lead Data Engineer :

Desired skills for lead data engineer include :

- Python

- Spark

- Java

- Hive

- SQL

- Hadoop architecture

- Large scale search applications and building high volume data pipelines

- Message queuing

- NoSQL

- Scala

Desired experience for lead data engineer includes :

- Experience in development utilizing C# .NET 4.5

- Experience in managing a live service that customers depend

- Experience in coaching and managing other engineers

- Be a team player and enjoy collaboration with other engineers and teams

- Experience with software version management systems

- Experience with task/bug tracking software