IN Data Engineer - Data Platforms-Azure

Key Responsibilities - Design, develop, and maintain robust data pipelines and architectures using Azure Data Factory and Azure Databricks. - Collaborate with data scientists and analysts to understand data needs and implement solutions that support analytics and reporting requirements. - Ensure data quality and integrity by implementing data governance and standardization processes across all data platforms. - Optimize data processing workflows to enhance performance and scalability, ensuring efficient handling of large data sets. - Develop and manage PySpark notebooks for data transformation and analysis, ensuring code is scalable and maintainable. - Work with cross-functional teams to integrate data from multiple sources, ensuring seamless data flow and accessibility. - Mentor junior data engineers, providing technical guidance and fostering a culture of continuous learning and improvement. - Stay up-to-date with the latest advancements in Azure technologies and data engineering best practices to ensure the team is leveraging the most effective tools and techniques.

Overview - The Senior Data Engineer Azure role is pivotal within our consulting sector, focusing on leveraging data platforms to drive business insights and solutions. - Based in Kolkata, this position is designed for experienced professionals looking to utilize their expertise in Azure to enhance data engineering capabilities. - The successful candidate will collaborate with cross-functional teams to design, implement, and optimize data solutions, ensuring they meet the strategic needs of our clients. - This role requires deep technical knowledge and the ability to work across various data platforms, primarily focusing on Azure services. - The Senior Data Engineer will contribute to the development of scalable data architectures and ensure the efficient processing of large volumes of data. - As a Level 3 position, this opportunity is tailored for individuals with 3-6 years of professional experience in data engineering, particularly within consulting environments. - Working within a dynamic team, the Senior Data Engineer will be responsible for implementing best practices in data management and analytics. - This includes the development of data pipelines, integration of diverse data sources, and deployment of advanced analytics solutions. - The role demands a strong understanding of Azure and its associated technologies, such as Azure Data Factory and Azure Databricks. - The position offers a unique opportunity to work on cutting-edge projects that transform raw data into actionable insights, directly impacting business outcomes. - As a key player in the data engineering team, the Senior Data Engineer will also mentor junior engineers and contribute to the continuous improvement of data engineering methodologies.

Requirements - Minimum of 7 years of experience in data engineering with a strong focus on Azure data platforms. - Proficient in using Azure Data Factory, Azure Databricks, and PySpark for data processing and analytics. - Demonstrated experience in building and optimizing data pipelines and architectures for large-scale data solutions. - Strong understanding of data governance, data quality, and data management principles. - Excellent problem-solving skills and the ability to work collaboratively in a team environment. - Experience in consulting environments is preferred, with a proven track record of delivering data-driven solutions to clients. - Strong communication skills with the ability to articulate complex data concepts to technical and non-technical stakeholders.