Senior Data Engineer

Responsible for database administration, ETL/data pipeline development, migration, monitoring, performance optimization and operational support for CBA's & One Data.

Key Responsibilities

• Analyse requirements for new ETL/data engineering requests.
• Design and develop ETL pipelines.
• Develop approximately 1015 new pipelines per month as required.
• Work with SnapLogic and/or Databricks pipelines.
• Develop and support One Data migration pipelines.
• Develop new CBA data pipelines.
• Monitor existing 1,000+ jobs in coordination with the shared 127 support model.
• Perform database administration activities.
• Monitor Redshift/PostgreSQL database health and performance.
• Perform query optimization and performance tuning.
• Analyse database bottlenecks and recommend improvements.
• Support schema changes and data migration activities.
• Perform data validation and reconciliation.
• Troubleshoot ETL and database failures.
• Support production incidents and critical data issues.
• Establish appropriate monitoring and observability metrics.
• Document database processes, migration procedures and operational runbooks.
• Work closely with other ADP full-time employees and engineering teams.
• Ensure data operations are stable, reliable and maintainable.

Required Technical Skills

Mandatory

• 7–8 years of experience in Data Engineering/DBA.
• Strong PostgreSQL experience.
• Strong Amazon Redshift experience.
• Strong SQL.
• Database performance tuning.
• Query optimization.
• ETL/ELT development.
• Data migration.
• Data validation and reconciliation.
• Production database support.
• Job monitoring and incident troubleshooting.
• Experience with large-volume data environments.
• Strong understanding of database security and access controls.

Strongly Preferred

• Databricks
• SnapLogic
• AWS data ecosystem.
• ETL orchestration.
• Data pipeline monitoring.
• Schema evolution.