Data Engineer

Position Name: Data Engineer

Location: Noida

Experience:4-8 years

About BinQle
BinQle is a global delivery and execution partner supporting enterprises across the US, India, Singapore and Australia. We specialize in structured hiring, managed teams, and long-term delivery support.

Role Summary:
Design, develop, and maintain scalable data integration and processing pipelines using Apache NiFi and Apache Kafka for the Canopy Automation Platform. Integrate and process data from telecom OSS/NMS and operational systems, enabling reliable data delivery for automation, monitoring, and reporting.

What Youll Do:
- Design, develop, and maintain data flows using Apache NiFi processors and related components
- Implement event-driven data pipelines with Apache Kafka, including topic, producer, and consumer management
- Ingest and transform data from APIs, databases, files, message queues, and telecom network management systems
- Develop ETL/ELT pipelines using Python, SQL, and PySpark for data extraction, transformation, and loading
- Build pipelines for KPI computation and service reporting
- Implement data quality mechanisms including validation, deduplication, error handling, and recovery
- Optimize performance for NiFi flows, Kafka pipelines, and data processing workloads
- Monitor pipelines, manage logging, alerting, back-pressure, and failure recovery
- Integrate processed data with Canopy applications, APIs, databases, and analytics platforms
- Follow Git-based development, code reviews, automated testing, QA validation, CI/CD, and production deployment practices
- Maintain documentation for architecture, data mappings, configurations, runbooks, and troubleshooting

What We re Looking For (Must-Haves):
- Strong hands-on experience with Apache NiFi, including flow design, processor configuration, error handling, and performance tuning
- Strong hands-on experience with Apache Kafka, including topic design, producer/consumer management, and delivery semantics
- Proficiency in Python and SQL for data transformation and processing
- Experience developing ETL/ELT pipelines and integrating heterogeneous data sources
- Understanding of JSON, Avro, CSV, REST APIs, and schema management
- Experience with relational databases such as PostgreSQL or MySQL
- Knowledge of data quality, idempotency, fault tolerance, monitoring, and recovery patterns
- Familiarity with Git, Linux, CI/CD, automated testing, and production support

Good to Have (Optional - only if mentioned):
- Apache Spark/PySpark for batch and distributed processing
- Kafka Connect, Schema Registry, and stream-processing frameworks
- Docker, Kubernetes, and cloud-native environments
- Telecom OSS/NMS integration, network alarms, MoP execution data, trouble tickets, and KPI platforms
- Experience with high-volume event ingestion and near-real-time data processing

Qualifications:
As per requirement

Disclaimer: This job posting has been aggregated from external source. Role details, content, and availability are subject to change. Applicants are advised to confirm the latest information directly on the company website before applying.