Lead AWS Data Engineer (Python & Scala)

• Develop & Optimize Data Pipelines Architect, build, and enhance scalable data pipelines for high-performance processing.
• Troubleshoot & Sustain Identify, diagnose, and resolve data pipeline issues to ensure operational efficiency.
• Data Architecture & Storage Design efficient data storage and retrieval strategies using Postgres, Redshift, and other databases.
• CI/CD Pipeline Management Implement and maintain continuous integration and deployment strategies for smooth workflow automation.
• Scalability & Performance Tuning Ensure the robustness of data solutions while optimizing performance at scale.
• Collaboration & Leadership Work closely with cross-functional teams to ensure seamless data flow and lead engineering best practices.
• Security & Reliability – Establish governance protocols and ensure data integrity across all pipelines.

Technical Skills Required:

• Programming: Expert in Python and Scala
• Big Data Technologies: Proficient in Spark, Kafka
• DevOps & Cloud Infrastructure: Strong understanding of Kubernetes
• SQL & Database Management: Skilled in SQL administration, Postgres, Redshift
• CI/CD Implementation: Experience in automating deployment processes for efficient workflow