Big Data Architect - Data Engineering

Job Summary/Your Role in a Nutshell:

The ideal candidate would be an experienced Big Data Architect with expertise in analytics, data engineering, architecture, governance, and framework design. They should approach business problems with creative thinking and guide teams to success by spearheading design and implementation processes while fostering long-term client relationships. Team handling experience is a must.

What you'll do:

- Define client needs and oversee project milestones to ensure expectations, timelines, and budgets are met.

- Define data platform architecture, design, and review code and make required changes.

- Responsible for the overall quality of project deliverables and the successful implementation of defined solutions for the customer.

- Build long-term, superior client relationships, proactively manage client expectations, and ensure that change control is used when scope boundaries are exceeded.

- Communicate a compelling and inspired vision to all customer levels, from CTOs/CIOs to engineering managers and programmers.

- Leverage extensive knowledge of distributed systems, process flows, and procedures to aid analysis and recommendations for solution offerings.

- Maintain a strong network and promote the organization at various meetings, forums, panels, publications, and conferences to establish thought leadership in the industry.

- Responsible for POCs on any tech stacks introduced, or possible version upgrades.

What you need:

- BE/B.Tech/MCA with at least 12+ years of experience in design and development using various ETL, database technologies with at least 5+ years associated Hadoop technology stack and programming languages.

Hands -on experience in 2 or more areas:

1. Hadoop, HDFS, MR

2. High Availability architecture and DR setup

3. Spark Streaming, Spark SQL, Spark ML

4. Kafka/Flume

5. Apache NiFi

6. Worked with Hortonworks Data Platform as Architect CDH (Cloudera Distribution for Hadoop) as developer/administrator

7. Hive / Pig / Sqoop

8. NoSQL Databases HBase/Cassandra/Neo4j/MongoDB

9. Visualization and Reporting frameworks like D3.js, Zeppelin, Grafana, Kibana Tableau, Pentaho

10. Scrapy for crawling websites

11. ElasticSearch

12. Google Analytics data streaming

13. Data security (Kerberos/OpenLDAP/Knox/Ranger)

- Extensive knowledge of the current landscape and the ability to visualize technology and industry trends.

- Working knowledge of Big Data Integration with Third-party/in-house built Metadata Management, Data Quality, structured/unstructured data Master Data Management solutions.

- Ability to successfully lead a team of big data engineers and big data developers through all phases of the development life cycle, including requirements definition, architecture, design, development, implementation, and testing.

- Active involvement in the community in terms of articles, blogs or speaking engagements at conferences.

About TO THE NEW:

TO THE NEW is a digital technology services company providing innovative product engineering solutions to diverse industries across the globe. We design and build digital platforms with Cloud, Data, and AI as the main pillars. Global analyst firms like Gartner, Forrester, Everest, ISG, and Zinnov have recognized TO THE NEW for its capabilities across Digital Engineering, Cloud, Media & Entertainment Solutions, Data & AI, and Digital Marketing. We have also been recognized as a 'Great Place to Work' consecutively for many years and have a team of 2000+ passionate 'Newers' spread across North America, EMEA, India, SEA, and ANZ with our headquarters in Singapore.