Apache Airflow Engineer

Job Summary

• We are seeking Apache Airflow migration resources to support a client initiative focused on migrating existing Cybermation-scheduled scripts to Apache Airflow. The resources will analyze the current Cybermation schedules, understand the existing shell/API invocation patterns, convert the schedules into Airflow DAGs, validate execution behavior, and support production deployment and handover.
• The migration objective is to preserve the existing business execution behavior while improving schedule control, monitoring, logging, retry handling, operational visibility, and supportability through Apache Airflow.

Key Responsibilities

• Analyze existing Cybermation job schedules, dependency patterns, runtime parameters, execution frequency, owners, success/failure behavior, and operational support expectations.
• Review existing shell scripts and API/cURL invocation logic to determine the appropriate Airflow DAG design, task structure, error handling, and retry configuration.
• Design, develop, and maintain Apache Airflow DAGs to orchestrate shell scripts, API calls, and dependent jobs with appropriate scheduling and operational controls.
• Configure Airflow schedules, task dependencies, retries, logging, alerting hooks, variables, connections, parameters, and environment-specific configuration as per client standards.
• Perform dry runs, unit testing, schedule validation, API response validation, failure/retry validation, and production-readiness checks for migrated jobs.
• Create migration mapping, validation evidence, deployment notes, operational runbooks, and support handover documentation. Coordinate with client SMEs, application owners, infrastructure teams, and release teams to support deployment, issue triage, initial run monitoring, and hypercare.