Databricks Engineer

We're looking for

Databricks Engineers (India)! Qualifications

• Proficiency in Pyspark, Cloud Platform and SQL.
• Proficiency in implementing data solutions on the Databricks platform with hands on experience in setting up Databricks cluster, working in Databricks modules, data pipelines for ingestingand transforming data from various sources into Databricks.
• Spark-based ETL workflows and data pipelines for data preparation and transformation within Databricks.
• Bachelors or Master's degree
• Proficiency in implementing data solutions on the Databricks platform.
• Solid understanding of Databricks fundamentals and have hands on experience in setting up Databricks cluster, working in Databricks modules (Data Engineering, ML and SQL warehouse).
• Configure, set up, and manage Databricks clusters, workspaces, and notebooks to ensure optimal performance, scalability, and resource allocation.
• Implement data pipelines for ingesting, cleansing, and transforming data from various sources into Databricks, ensuring data quality, consistency, and reliability. Develop ETL (Extract, Transform, Load) processes as needed.
• Develop, optimize, and maintain Spark-based ETL workflows and data pipelines for data preparation and transformation within Databricks.
• Knowledge of Performance Optimization, Monitoring and Automation would be a plus.
• Understanding of data governance, compliance, and security best practices.
• Strong Proficiency in Pyspark, Databricks Notebooks, SQL, Python, Scala, or Java.
• Proficiency in SQL for data querying and manipulation.
• Familiarity with data modeling, ETL processes, and data warehousing concepts.
• Strong problem-solving and troubleshooting skills.
• Excellent communication and collaboration skills.
• Certifications in Databricks would be beneficial. ( *Note: Scoutit is an independent candidate sourcing partner for this hiring. All applications, candidate screening, and hiring-related actions are conducted directly by the respective employer’s authorized recruitment team. Relevant candidates will be contacted via email and asked to fill in additional details as part of the next stage of the application process.) Skills: pipelines,sql,etl