Databricks

Title: Databricks

Location: Bangalore, Pune, Mumbai, Kolkata, Delhi, Hyderabad, Bhubhaneshwar, Coimbatore

MOI: F2F

Mandatory skills:

Databricks, Pyspark, SQL, Azure Functions, Live / Streaming / batch processing data, Azure Data Factory (ADF), Logic Apps.

ELT / ETL, Data Integration & Ingestion, Data Manipulation and Processing, GITHUB, Action, Azure DevOps

Optional skills: Synapse, Data Modelling.

Experience in project management, running a scrum team.

Exposure to working with external technical ecosystem.

Key Skills:

• Design and implement scalable, secure, and high-performance data architectures on Azure.

• Design, develop, and optimize ETL/ELT data pipelines using Azure Data Factory (ADF), Databricks, and Synapse Analytics.

• Implement data ingestion, transformation, and storage solutions using Azure Data Lake, Azure SQL, and Azure Synapse.

• Develop batch and real-time data processing solutions using Azure services like Delta Live Tables, Event Hubs, Stream Analytics, and Azure Functions.

• Development experience as a data engineer with focus on core tools and technologies like ADF, Databricks, Key Vault, ADLS, SQL Server / Any RDMS, Fabric, Azure Synapse. 

• Strong SQL and Python skills for data transformation and analysis.

• Design and implement data models, schemas, and performance-optimized queries.

• Work with structured, semi-structured, and unstructured data sources.

• Ensure data quality, governance, and security in all solutions.

• Deploy data solutions using Azure DevOps CI/CD pipelines.

• Work closely with data scientists, analysts, and business users to support data-driven decision-making.

• Troubleshoot and optimize data workflows for scalability and performance.

• Ensure data solutions are cost-optimized, scalable, and secure on Azure.

• Implement logging, monitoring, and alerting solutions using Azure Monitor and Log Analytics

Secondary skills:

• Project management – status reports, stand up, knowledge on Agile methodology

• Stake holder management

• Mentor data engineers and analysts

• Provide technical direction and design reviews

• A determined focus on the user and user experience when problem- solving

• Team player and able to collaborate, proactively communicates and shares understanding and experience within a team 

• Clear communicator and able to work in a multicultural/lingual environment 

• Uses initiative and owns their deliverables end-to-end 

• Brings ideas to the table, is inquisitive and excited by new technology  

• Used to working in a flexible, independent manner on a mixture of small unstructured and large structured items 

• Provides regular, considered feedback and regularly strives to improve their own ways of working

Soft Skills:

• Engage with business leaders, developers, infra, and security teams

• Understand business requirements deeply

• Manage expectations and negotiate priorities

• self-starter and proactive to provide cost and performance-oriented solutions.                                                               

Preferred Qualifications / Certifications

• Minimum 5+ years of experience

• Any graduation, Btech, BE, MBA, BSC, MSC

• Experience working in regulated environments and with internal systems quality policies and procedures.

• Experience in development and deployment on cloud infrastructure.

• Azure Certifications (Azure Data Engineer Associate, Azure Solutions Architect).

• Databricks Certification