Full Stack Python Engineer – GCP & Data Engineering

Key Responsibilities

Design and develop enterprise applications and backend services using Python.

Design and implement REST APIs and reusable service components for application and data integrations.

Build data ingestion, transformation and processing solutions on GCP.

Develop and optimize solutions using BigQuery for large-scale data processing and analytical workloads.

Develop Apache Beam pipelines for batch and streaming data-processing use cases. Beam provides native BigQuery connectors for reading from and writing to BigQuery.

Develop and support distributed data-processing workloads using Dataproc / Spark where applicable.

Integrate Python services with databases, cloud services, APIs and enterprise applications.

Design reusable frameworks for data ingestion, transformation, validation and publishing.

Implement application and pipeline-level logging, monitoring, exception handling and data-quality controls.

Develop unit and integration tests and participate in peer code reviews.

Troubleshoot application, API, data-pipeline and production issues.

Work with architecture, data engineering, DevOps, security and application teams to deliver production-ready solutions.

Follow enterprise engineering standards around security, scalability, maintainability and deployment.

 

Required Technical Skills

 

Must Have

Strong hands-on programming experience with Python.

Strong experience building RESTful APIs and backend services.

Strong knowledge of SQL and data-processing concepts.

Hands-on experience with Google Cloud Platform (GCP).

Strong experience working with BigQuery.

Experience building data-processing solutions with Apache Beam.

Experience with Dataproc / Spark / PySpark for distributed data processing.

Understanding of both batch and streaming data-processing patterns.

Experience integrating applications with databases, files, APIs and cloud services.

Understanding of software engineering practices including modular design, testing, version control and CI/CD.

 

Good to Have

Experience with Google Cloud Dataflow, particularly for running Apache Beam pipelines. Google Cloud documents Dataflow as supporting pipelines developed with the Apache Beam Python SDK.

Experience with Cloud Storage and Pub/Sub.

Experience with Python API frameworks such as FastAPI or Flask.

Knowledge of Docker and Kubernetes/GKE.

Experience with GitHub, GitHub Actions and/or Harness.

Exposure to workflow orchestration tools such as Airflow / Cloud Composer.

Experience implementing data-quality and observability frameworks.

Exposure to data governance, metadata and lineage concepts.

Prior experience working within banking, financial services or other regulated enterprise environments.

Expected Technical Profile