EFS FCRM Professional

Responsibilities

• Design, build, and maintain robust ETL/ELT data pipelines using Apache Spark on Databricks.
• Implement data Lakehouse architecture using Delta Lake for cost-effective data storage and analytics.
• Use Databricks Workflows for orchestrating batch and streaming pipelines.
• Develop and maintain CI/CD pipelines for data applications using tools such as AWS, GitHub Actions or Databricks Repos.
• Monitor pipeline performance and troubleshoot data issues in real-time and batch environments.
• Document solutions, workflows, and technical standards.

Requirements

• Experience in data engineering with a strong focus on AWS Databricks and Apache Spark.
• Proficiency in PySpark, SQL, and Python.
• Experience with Delta Lake, Databricks SQL, and Unity Catalog.
• Hands-on experience with cloud platforms
• Familiarity with data lakehouse architecture, data warehousing and streaming data
• Strong understanding of ETL best practices, data partitioning, and performance tuning.
• Experience with CI/CD for data pipelines.
• Experience in Machine learning and Data Analytics is added advantage.
• Strong exposure and understanding on the AI and Agentic AI technologies.
• Excellent problem-solving and communication skills