Senior Data Engineer

Notice Period: 0-30 days

Qualification: B.Tech/M.Tech/MCA

We are seeking a skilled Data Engineer to design, build, and maintain scalable data pipelines and infrastructure that support analytics, reporting, and machine learning initiatives. The ideal candidate has strong experience with SQL, Python, cloud platforms, and modern data engineering tools. You will collaborate with data analysts, data scientists, and software engineers to ensure reliable, high-quality data is available across the organization.

Job Description:

• Design, develop, and maintain scalable ETL/ELT data pipelines.
• Build and optimize data warehouses, data lakes, and data models.
• Integrate data from multiple sources, including APIs, databases, and third-party systems.
• Ensure data quality, consistency, security, and governance.
• Monitor and troubleshoot data pipelines and resolve performance issues.
• Optimize SQL queries and database performance.
• Collaborate with cross-functional teams to understand data requirements and deliver solutions.

Requirements

• Core Data Engineering & Architecture

• Design and build end-to-end data pipelines (ETL/ELT) for structured and unstructured data
• Develop scalable data platforms handling terabytes of data with high reliability and low latency
• Strong expertise in data modeling, warehousing, and lakehouse architectures
• Own data quality, lineage, governance, and observability frameworks
• Hands-on experience with Spark, Hadoop, Kafka, Flink (batch + real-time processing)
• Build and optimize high-throughput, distributed data systems
• Experience in streaming + event-driven architectures for large-scale financial data
• Deep expertise in AWS / Azure / GCP (Data Lakes, Warehouses, Compute, Storage)
• Tools: Databricks, Snowflake, Redshift, Synapse, BigQuery
• Pipeline orchestration using Airflow, Prefect, or similar frameworks
• Strong coding in Python, SQL, Scala
• Focus on performance optimization, reliability, and production-grade systems
• Experience with CI/CD, DevOps, and infrastructure-as-code
• Ability to design architectures and make technology decisions at scale
• Strong understanding of:
• Trade lifecycle, transactions, risk, compliance, regulatory reporting
• Customer 360, payments, lending, capital markets data and Wealth Management Business
• GenAI / LLM integration (RAG, embeddings, vector stores)
• Lead legacy-to-cloud data platform migrations
• Drive data re-engineering, system decomposition, and modernization initiatives
• Drive resilience, failover, and incident response frameworks
• Work across engineering, data science, risk, compliance, and product teams
• Ability to translate business needs (risk, reporting, revenue) into data solutions
• Mentor engineers and drive engineering best practices and standards.