Databricks Data Engineer

Role & responsibilities

Key Responsibilities: 
- Develop and maintain data pipelines using Databricks, Spark, and other data processing technologies. 
- Collaborate with data scientists, analysts, and other stakeholders to understand data requirements and ensure data quality and accessibility. 
- Implement and optimize ETL processes to support analytics and reporting initiatives. 
- Monitor and troubleshoot data pipeline performance, ensuring timely delivery and reliability. 
- Contribute to the design and implementation of data architecture and data models. 
- Utilize Databricks notebooks for data exploration, visualization, and sharing insights with team members. 
- Stay updated with the latest trends and advancements in data engineering and the Databricks platform. 

Skills and Tools Required: 
- Proficiency in Databricks and Apache Spark. 
- Strong programming skills in Python or Scala for data processing. 
- Experience with ETL processes and tools. 
- Knowledge of data modeling and data warehousing concepts. 
- Familiarity with cloud platforms such as AWS, Azure, or Google Cloud. 
- Experience with SQL and database technologies (e.g., MySQL, PostgreSQL, or others). 
- Familiarity with data visualization tools (e.g., Tableau, Power BI) is a plus. 
- Strong analytical and problem-solving skills. 
- Excellent communication and collaboration skills to work effectively within a team.