Solution Architect

Responsibilities:

• Responsible for leading a team of talented data engineers responsible for designing, building, and maintaining scalable data pipelines and infrastructure.
• Work closely with cross-functional teams to ensure client data systems meet the highest standards of quality and performance.
• Lead, mentor, and develop a team of data engineers, fostering a collaborative and inclusive team environment.
• Conduct regular performance reviews, provide feedback, and set goals for team members.
• Identify and address skill gaps and provide opportunities for professional development.
• Plan, execute, and deliver data engineering projects on time and within scope.
• Coordinate with stakeholders to gather requirements, set priorities, and define project timelines.
• Ensure projects align with overall business objectives and data strategy.
• Oversee the design, development, and maintenance of data pipelines, ETL processes, and data warehouses.
• Ensure data quality, integrity, and security across all data engineering projects.
• Identify opportunities for process improvements and drive initiatives to enhance the efficiency and effectiveness of data operations.
• Has a strong conceptual understanding of data warehousing and ETL, data governance and security, cloud computing, and batch and real-time data processing
• Ability to build/drive reusable frameworks that can drive efficiency of the overall data system
• Has executed and led multiple projects, including streaming, batch, large data pipelines, etc.
• Manages conversations with the client stakeholders to understand the requirements and translate them into technical outcomes.

Requirements:

• Strong experience with Databricks, Spark, and cloud platforms (Azure, AWS, GCP).
• Architect and deploy cloud-based data solutions (Azure, AWS, GCP).
• Define CI/CD strategies for data pipelines using Terraform, Azure DevOps, or GitHub Actions.
• Implement data cataloging, lineage tracking, and access control (Unity Catalog, Collibra, Alation).
• Ensure compliance with GDPR, CCPA, and industry-specific data security policies.
• Develop strategies for distributed computing, parallel processing, and caching mechanisms.