COIMBATORE · FULLTIME
Lead Data Engineer
Vijay Venkatesh ( Proprietor of Cynosure Corporate Solutions )
Coimbatore · onsite · Posted 30d ago
Sourced from
Not disclosed7–15 yrsfulltimeCoimbatore
About a minute. You'll make an account on the way through.
agent · working
- Read this posting0.3s
- Found the form on Foundit · 7 questions to answer1.5s
- Decided the answers for all of them3.0s
- Tailor your résumé to this role
- Waiting for you
Section · 01
About this role
We are looking for a highly skilled Senior Data Engineer with strong expertise in Google Cloud Platform (GCP), Apache Beam/Dataflow, Java, and BigQuery. The ideal candidate will be responsible for designing, building, and optimizing scalable data pipelines and cloud-based data processing solutions for enterprise applications.
Key Responsibilities
- Design and develop scalable ETL/ELT pipelines using GCP services.
- Build and maintain real-time and batch data processing pipelines using Dataflow and Apache Beam.
- Develop robust backend processing solutions using Java.
- Work extensively with BigQuery for data warehousing, analytics, and performance optimization.
- Integrate data from multiple sources including APIs, databases, and streaming platforms.
- Optimize pipeline performance, cost, scalability, and reliability on GCP.
- Collaborate with cross-functional teams including Architects, Analysts, DevOps, and Business stakeholders.
- Ensure data quality, governance, monitoring, and security best practices.
- Troubleshoot production issues and provide long-term scalable solutions.
Required Skills
- Strong hands-on experience with GCP services.
- Expertise in Google Dataflow / Apache Beam.
- Strong programming skills in Java.
- Hands-on experience with BigQuery.
- Experience in building batch and streaming data pipelines.
- Good knowledge of SQL and data modeling concepts.
- Experience with CI/CD, Git, and Agile methodologies.
- Strong problem-solving and debugging skills.
Good to Have
- Experience with Pub/Sub, Cloud Composer, Cloud Storage, Dataproc, or Kafka.
- Exposure to Python or Spark.
- Experience in handling large-scale enterprise data platforms.
Preferred Qualifications
- Bachelor's/Master's degree in Computer Science, Engineering, or related field.
- GCP certifications are an added advantage.
Sourced from foundit
Section · 02
Skills
data qualityAgileApache SparkBatch processingETL/ELTApache Kafkaperformance optimizationgoogle cloud platformCloud StoragebigqueryCloud ComposerApache BeamDataprocStreaming data pipelinesGoogle DataFlowGoogle Cloud Pub SubPythonSQLCloud SecurityJavaData GovernanceData ModelingGITCi/Cd