Azure Data Engineer with PostgreSQL

Skills- Azure Data Factory, Python, PostgreSQL, Azure Blob Storage, and SQL

Key Responsibilities

• Develop data ingestion and migration pipelines using Azure Data Factory and Python.
• Extract data from SharePoint lists, document libraries, Excel workbooks, XML files, and file repositories.
• Build transformation logic to standardize heterogeneous tracker structures and align with the target data model.
• Implement data cleansing, validation, reconciliation, and error-handling routines.
• Develop bulk-load and delta migration processes for active case migration.
• Migrate attachments to Azure Blob Storage and maintain metadata references within PostgreSQL.
• Create SQL scripts and load processes for PostgreSQL data migration.
• Support testing, UAT, reconciliation, cutover, and post-migration validation activities.
• Monitor migration performance and optimize pipeline execution.
• Prepare technical documentation and migration deployment artifacts.

Required Skills & Experience

• 4+ years of experience in Data Engineering, ETL development, or Data Migration projects.
• Hands-on experience with:
• Azure Data Factory (ADF)
• Python
• PostgreSQL
• Azure Blob Storage
• SQL
• Experience working with SharePoint APIs, Excel ingestion, XML parsing, and document migration.
• Knowledge of data quality validation, reconciliation processes, and ETL best practices.
• Experience with Azure DevOps, Git, and CI/CD tools is desirable.