Python Software Developer

Role & responsibilities

GCP Engineering Skills

• Google Cloud Platform (GCP)
• BigQuery
• Dataproc / Apache Spark
• Bigtable
• Dataplex / Data Catalog
• Python (ETL/ELT pipelines and automation)
• SQL (data transformation, validation, reconciliation)
• Databases:
• MySQL
• Hive
• MongoDB
• Cloud Composer / Apache Airflow
• Kafka and Google Pub/Sub
• Apigee / REST APIs (data extraction and integration)
• GenAI (LLMs, embeddings, prompt engineering) and integration with data platforms
• Experience with Vertex AI, BigQuery ML, or GenAI-enabled GCP services (preferred)

Key Responsibilities

Migration & Modernization

• Design and execute comprehensive migration strategies from on-premises, legacy platforms, or other cloud environments to GCP.
• Migrate structured and semi-structured data to BigQuery, Bigtable, and other GCP storage solutions.
• Build, optimize, and maintain scalable migration pipelines using Python, SQL, and Dataproc (Apache Spark).
• Implement batch and streaming migration patterns using:
• Cloud Composer (Apache Airflow)
• Kafka
• Google Pub/Sub

Data Quality & Governance

• Perform schema mapping, data transformation, reconciliation, and validation to ensure accuracy, consistency, and completeness of migrated datasets.
• Enable metadata management, lineage, and governance using Dataplex and Data Catalog.
• Handle large-volume data transfers using best practices including:
• Parallel loading
• Incremental loads
• CDC (Change Data Capture)

GenAI Adoption

• Leverage GenAI and automation techniques to accelerate:
• Data discovery
• Schema understanding
• Mapping recommendations
• Migration assessments
• Apply GenAI-assisted:
• Data transformation
• Code generation
• Migration pattern optimization
• SQL conversion
• Pipeline scaffolding
• Test case generation

Preferred candidate profile