Data Engineer

Permanent role with APAR Technologies deployed to client project

Detailed Responsibilities

Data Quality (DQ) Rule Development:

• Design, document, and implement automated data quality test scripts (rules/checks) using SQL, Python, Open Metadata or other specialized DQ tools to monitor data accuracy, completeness, consistency, and conformity.

Proactive Monitoring Reporting:

• Configure and manage data quality dashboards and alerting systems to proactively detect anomalies, breaches of DQ rules, and data drift in near real-time.
• Generate regular data quality scorecards and reports detailing data health, trends in issues, and the effectiveness of remediation efforts for management review.

Issue Triage and Remediation:

• Serve as the primary responder to reported data quality issues, performing root cause analysis to identify systemic problems versus user errors.
• Work cross-functionally with the business tools teams and data owners to implement longer-term remediation plans, track fixes, and ensure controls are in place to prevent recurrence.
• Adhere to a privileged access management process for accessing live data

Metadata Management:

• Maintain and update metadata and data dictionaries related to data quality rules, metrics, data lineage and critical data elements (CDEs).
• Deliver reports/outputs of the data dictionary for the use of AI Agents to learn about what data is where and which is the best sources of data
• Ensure DQ tests and metrics are consistently mapped to business processes and regulatory requirements.

Skills Experience

• Knowledge of key RDBMS concepts and SQL
• Knowledge of MS SQLServer and MS Fabric
• Experience with directing AI agents to develop code
• Knowledge of Open MetaData
• Experience with AWS cloud platform
• Experience working in an IT support environment
• Knowledge of ITIL best practices for IT support