Pyspark Data Engineer

Job Title PySpark Data Engineer – HL7 Integration Specialist

Job Summary We are seeking a highly skilled PySpark Data Engineer with strong experience in HL7 healthcare data standards and large-scale data processing. The ideal candidate will be responsible for designing, developing, and optimizing healthcare data integration pipelines, transforming HL7 messages, and enabling efficient data exchange across healthcare systems. Key Responsibilities Design, develop, and maintain scalable data pipelines using PySpark. Process, transform, and validate healthcare data based on HL7 standards (any version such as HL7 v2.x, HL7 v3, or FHIR). Build and optimize ETL/ELT workflows for healthcare interoperability solutions. Collaborate with business stakeholders, architects, and healthcare domain teams to understand data integration requirements. Ensure data quality, integrity, security, and compliance with healthcare regulations. Troubleshoot and resolve data processing and integration issues. Develop reusable frameworks and best practices for healthcare data ingestion and transformation. Monitor pipeline performance and implement optimization strategies. Create and maintain technical documentation and data mapping specifications.

Required Skills (Must Have) Strong hands-on experience with PySpark and distributed data processing Solid understanding of HL7 standards (any version) and healthcare data exchange concepts Experience designing and implementing data transformation and integration solutions Proficiency in SQL and data modeling concepts Experience working with large-scale datasets and ETL processes Strong problem-solving and analytical skills Excellent communication and collaboration abilities

Preferred Skills (Good To Have) Experience with SMILE CDR. Knowledge of Google Cloud Platform (GCP) services Experience with Informatica (PowerCenter, IICS, or related tools) Familiarity with healthcare interoperability standards such as FHIR Exposure to cloud-based data engineering architectures

Qualifications Bachelor's or Master's degree in Computer Science, Information Technology, Engineering, or a related field. Relevant experience in data engineering, healthcare interoperability, or healthcare data integration projects.

Experience 4+ years of experience in Data Engineering 2+ years of hands-on experience with PySpark and healthcare data integration using HL7