Hadoop, Databricks, PySpark, Spark SQL, Python - Immediate or 15 days

Mandatory Skills : Apache Spark, Python, Python for DATA, SparkSQL

The primary requirement is for candidates with strong experience in Databricks, PySpark, Spark SQL, Python.

Candidates must have strong hands-on experience in Databricks, PySpark, Spark SQL, Python and cloud.

Profiles with limited Databricks exposure or primarily support-based experience will not be considered.

Preference will be given to candidates who have worked on large-scale data platforms and Azure cloud environments.

Detailed JD

About the Role

As a Data Asset Developer you will play a critical role in designing robust scalable data architectures that power business intelligence and analytics across the cybersecurity landscape You will develop data models that meet the reporting and analytical needs of the cyber team while working closely with data scientists and analyticsBI engineers to optimize the use of the data models ensuring they are efficient and scalable

You will collaborate with data consumers to understand the requirements design and deliver the data models implement best practices for data governance and help set standards for data pipelines and solutions ensuring seamless integration

Principal Accountabilities

• Data Architecture and Modelling
• Design and implement highquality scalable data models that ensure the integrity security and accessibility of Cybersecurity Data Assets
• Collaborate with analyticsBI engineers to ensure these models are used effectively
• Produce high quality documentation and metadata
• Operate the data governance processes to publish and manage the data assets in accordance with HSBCs data controls
• Databricks Expertise
• Design and implement data architectures using Databricks notebooks and Unity Catalog
• Build robust scalable data pipelines within Databricks

Required Skills and Qualifications

• The successful candidate will be a credible leader possess skills and experience within one or more of the following areas and demonstrate a willingness to learn additional skills via certification andor onthejob learning where required
• Overall 5 years of proven experience in data architecture modelling and database design including star snowflake schemas
• 3 years of experience with Databricks notebooks Unity Catalog and building scalable data pipelines using PySparkSpark MySQL
• Handson experience working with multiterabyte data sets optimizing for performance and costefficiency
• Indepth understanding of ETL processes and data warehouse architectures
• Familiarity with Azure cloud platform and data solutions
• Experience with data governance security and compliance practices
• Excellent communication problemsolving and collaboration skills
• Preferred Qualifications
• Experience with data Lakehouse architectures and modern data engineering practices
• Knowledge of machine learning pipelines and advanced analytics using Databricks
• BTech MCA or MTech in Computer Science Information Technology Engineering or related fields is a plus

Nice to Have

Relevant certifications in Databricks or cloud platforms

Broad knowledge of cybersecurity principles and disciplines

Experience working in or with cybersecurity teams