Databricks Consultant

Responsibilities :

Design, develop, and maintain scalable Databricks-based data engineering solutions. Build and optimize complex ETL/ELT pipelines using Databricks, PySpark, Spark SQL, and Python. Develop robust data ingestion and transformation frameworks for structured and unstructured data. Implement and manage Delta Lake and Lakehouse Architecture solutions. Design, monitor, and optimize enterprise-grade data pipelines for performance, scalability, and reliability. Perform performance tuning and troubleshooting of Databricks and Apache Spark workloads. Develop reusable frameworks and best practices for Data Engineering and Data Integration. Collaborate with architects, analysts, and business stakeholders to translate business requirements into technical solutions. Ensure data quality, governance, security, and compliance standards are followed.

Technical and Professional Requirements:

6â€8 years of experience in Data Engineering and Big Data technologies. Strong hands-on experience with Databricks and PySpark. Expertise in Apache Spark, Spark SQL, and distributed data processing. Strong programming skills in Python and SQL. Experience building and managing enterprise-scale ETL/ELT pipelines. Hands-on experience with Delta Lake and Lakehouse Architecture. Strong understanding of Data Warehousing, Data Modeling, and Data Integration concepts. Experience working with large-scale structured and unstructured datasets.