Big Data Engineer
Naukri Assist · Mumbai · onsite · Posted 48d ago
Up to ₹24.0L PA5–7 yrsfulltimeSourced from
About a minute. You'll make an account on the way through.
About this role
Roles and Responsibilities :
- Design, develop, test, deploy and maintain large-scale data pipelines using Python and Spark on Snowflake platform.
- Collaborate with cross-functional teams to gather requirements and deliver high-quality solutions that meet business needs.
- Develop complex SQL queries to optimize database performance and troubleshoot issues in Snowflake databases.
- Participate in code reviews to ensure adherence to coding standards, best practices, and quality guidelines.
Job Requirements :
- Build data pipelines to ingest, transform, and integrate client data into the platform.
- Partner with AI/ML Engineers to ensure ML-ready, high-quality datasets for the POD's use cases.
- Implement data validation, quality checks, and monitoring for client-specific pipelines.
- Support data integration with client systems, including secure data exchange protocols.
- Troubleshoot data issues within the POD''s scope and coordinate fixes with Core Platform teams.
- Maintain documentation of data flows, transformations, and lineage for audit purposes.
Must have skills:
- Strong experience with data pipeline tools (Airflow, dbt, Spark, or equivalent).
- Proficiency in SQL and Python for data transformation and validation.
- Experience integrating with diverse client data sources and APIs.
- Understanding of data quality, governance, and lineage practices.
- Familiarity with cloud data platforms (Snowflake, BigQuery, Databricks, or equivalent).
Must have Finance / BFSI Exposure:
- Experience handling financial or transactional client data with strict quality and security requirements.
- Understanding of data privacy and compliance requirements for financial services clients.
- Familiarity with common financial data formats and integration standards.
GOOD TO HAVE:
- Experience with real-time/streaming data integration for client use cases
- Exposure to data anonymization or masking techniques for sensitive financial data
Sourced from naukri
Skills
Apache SparkPythonSnowflake