MUMBAI · FULLTIME
Senior Data Engineer
Crisil
Mumbai · onsite · Posted 42d ago
Sourced from
Undisclosed6–9 yrsfulltimeMumbai
About a minute. You'll make an account on the way through.
agent · working
- Read this posting0.7s
- Found the form on Naukri · 5 questions to answer1.6s
- Decided the answers for all of them2.3s
- Tailor your résumé to this role
- Waiting for you
Section · 01
About this role
- Implement Data Architecture: Implement scalable, secure, and efficient data architecture on on-prem and cloud platforms (Azure/GCP/AWS) to support business growth and data-driven decision-making.
- Collaborate with data engineers, and product teams to identify data requirements and develop data models that meet business needs.
- Data Ingestion and Integration:
- Develop and maintain data ingestion pipelines using various tools and technologies, such as Apache Spark, PySpark, Kafka, and Flume.
- Experience with GenAI tooling: LangChain, OpenAI API, Amazon Bedrock, or Vertex AI integrations, Claude, Copilot
- Integrate data from multiple sources, including relational databases, NoSQL databases, APIs, and files.
- Batch and Stream Processing:
- Develop and maintain batch and stream processing pipelines using tools like Apache Spark & databricks.
- Integrate with messaging systems, such as Apache Kafka, Amazon Kinesis, and Google Cloud Pub/Sub.
- SQL Knowledge:
- Very strong SQL knowledge, including query optimization, indexing, and database design.
- Delta Lake and Data Warehouse:
- Design and implement Delta Lake and data warehouse/mart solutions to support business intelligence, reporting, and analytics.
- Develop and maintain data pipelines to ingest, process, and store data in Delta Lake and data warehouses.
- Distributed Databases and Data Warehousing:
- Implement and maintain data warehouses, such as Amazon Redshift, Google BigQuery, and Azure Synapse Analytics.
- Database Design and Development:
- Design, develop, and maintain efficient and scalable database systems across different platforms of relational databases (such as Oracle, MySQL, PostgreSQL, SQL Server).
- Collaborate with cross-functional teams to understand data requirements and translate them into effective database solutions.
- Implement and design data models and database schemas that align with business needs, ensuring data integrity and efficient data retrieval.
- Develop and optimize database queries, stored procedures, and functions for maximum performance and responsiveness.
- Performance Tuning and Optimization:
- Analyze and monitor database performance using diagnostic tools, identifying and resolving performance bottlenecks and inefficiencies.
- Optimize data processing workflows and queries to improve performance, reduce latency, and increase throughput.
- Data Management: Implement data archival mechanisms and data retention policies to ensure efficient data storage.
- Ensure the security and integrity of data by implementing access controls, data encryption, and backup and recovery strategies.
- Automation and Integration:
- Identify and implement automation solutions for data workflows.
- Collaborate with the development team to integrate database solutions into software applications effectively.
- Data Mart and Data Lake: Design and implement data marts and data lakes to support business intelligence, reporting, and analytics.
- Develop and maintain data pipelines to ingest, process, and store data in data lakes, such as Apache Hadoop, Amazon S3, and Azure Data Lake Storage.
- CI/CD and Automation:
- Develop and maintain automated testing, deployment, and monitoring scripts using tools like Jenkins, GitLab CI/CD, or similar.
- Ensure continuous integration and delivery of data pipelines and applications.
- Data Analysis and Modeling:
- Strong data analysis skills, including data modeling, data mining, and data visualization.
- Collaborate with data modelers to develop and implement data models to drive business insights and decision-making.
- Analyze complex data sets to identify trends, patterns, and correlations.
- Exploration of New Tools:
- Ability to explore new tools and technologies, and quickly develop proof-of-concepts (POCs) for data engineering open-source tools.
- Documentation: Document database design, configurations, and technical specifications.
Sourced from naukri
Section · 02
Skills
JenkinsApache SparkScalaApache Kafkaaws kinesisDeltaApachecddata miningcloud platformgoogle cloud platformamazon web servicesapache airflowgitlab ciFlumeApache Hadoopci/cd pipelineDatabricksPysparkData EngineeringPythonSQLData VisualizationData WarehousingAPIdata analysisMicrosoft AzureData Modeling
Section · Company
About Crisil
Crisil
Analytics & KPO
4.7k+
employees
1987
39 years old
Mumbai,Maharashtra
India
About
Crisil is an agile and innovative, global analytics company driven by its mission of making markets function better.
Industries
Analytics & KPOInternet
Employee ratings
1,936 reviews
3.5/ 5
Culture
3.6
Career growth
2.8
Work-life
3.6
Employees rate it well for
Work Life Balance385Salary355Work Culture280Senior Management209Learning167
Find them on