Data Enginner

We are looking for an experienced Data Engineer with strong hands-on expertise in Python, SQL, Spark/PySpark, and geospatial data processing to build and maintain scalable, production-grade data pipelines. The ideal candidate should have experience working withgeospatial/spatial data such as latitude/longitude coordinates, GPS/location data, maps, spatial datasets, or location-based datasets. Experience in geospatial data processing is mandatory for this role. You will work closely with Data Scientists, Analysts, and Engineering teams to build reliable data platforms and prepare high-quality datasets for analytics and downstream applications.

Responsibilities:

• Design, build, and maintain scalable ETL/ELT data pipelines using Python, SQL, and Spark/PySpark.
• Process and transform large-scale geospatial and non-geospatial datasets from multiple sources.
• Work extensively with latitude/longitude, GPS, coordinates, maps, location data, and spatial datasets.
• Perform geospatial data processing and analysis, including: Spatial joins, Proximity analysis, Distance calculations, Location-based aggregations, and coordinate and location transformations.
• Integrate geospatial data with structured and unstructured data sources.
• Build and optimise data pipelines for performance, reliability, scalability, and maintainability.
• Develop cloud-based data solutions using AWS, Azure, or GCP.
• Work with data lakes, data warehouses, distributed processing frameworks, and modern data architectures.
• Implement workflow orchestration using tools such as Apache Airflow, AWS Step Functions, Azure Data Factory, Google Cloud Composer, or equivalent.
• Implement data quality checks, testing, monitoring, and pipeline observability.
• Troubleshoot pipeline failures, data inconsistencies, and performance issues.
• Collaborate with Data Scientists, Data Analysts, and Engineering teams to prepare high-quality datasets for analytics and downstream applications.
• Follow best practices around data security, governance, deployment, and production operations.
• Participate in Agile development practices, code reviews, and technical discussions.

Requirements:

• 5-10 years of experience in Data Engineering or a closely related role.
• Strong hands-on experience with Python and SQL for data processing and pipeline development.
• Experience working with geospatial/location datasets, coordinates, latitude/longitude, GPS, maps, satellite imagery, spatial datasets, etc.
• Hands-on experience with Apache Spark/PySpark for processing large-scale datasets.
• Hands-on experience working with geospatial/spatial data is mandatory.
• Experience building and maintaining production-grade ETL/ELT data pipelines.
• Experience with at least one major cloud platform - AWS, Azure, or GCP.
• Good understanding of data lakes, data warehouses, distributed data processing, and modern data architectures.
• Experience with at least one workflow orchestration tool such as Airflow, AWS Step Functions, Azure Data Factory, Google Cloud Composer, or equivalent.
• Understanding of data quality, testing, monitoring, security, and governance practices.
• Strong problem-solving and communication skills, with the ability to work effectively in an Agile engineering environment.
• In terms of geospatial data, experience should be in latitude/longitude and maps.