Data Architecture || Open-Source Data Platforms

Must-Have Skills

• 7+ years of experience in Data Architecture / Data Engineering with strong focus on open-source data platforms.
• Proven expertise designing and implementing Data Lakehouse architectures using Delta Lake or Apache Iceberg.
• Deep hands-on experience with:
• Apache Airflow (DAG design, best practices, scaling)
• dbt Core (dimensional modeling, Kimball/ Data Vault, incremental models)
• Airbyte or similar open-source ingestion tools
• Trino or Presto for federated querying
• Strong experience with MySQL, PostgreSQL, and Salesforce data integration (especially CDC).
• Expertise in handling skewed data, data cleaning, standardization, and building reliable fact/dimension models.
• Experience migrating from legacy ETL tools (SSIS, Pentaho, Informatica, etc.).
• Strong Linux administration and self-hosted deployment knowledge.
• Proficiency in Python and Spark/PySpark for complex transformations.

Strongly Preferred

• Experience with S3-compatible object storage (MinIO, Ceph, etc.).
• Knowledge of data cataloging tools (Amundsen, DataHub – open-source versions).
• Familiarity with containerization (Docker, Kubernetes) and GitOps.
• Experience reducing infrastructure footprint (from multi-OS to single Linux environment).

Qualifications

• Bachelor’s or master’s degree in computer science, Engineering, or related field (or equivalent experience).
• Demonstrated ability to design scalable systems without relying on proprietary cloud services. - Strong communication skills and ability to explain complex architectural decisions to technical and non- technical stakeholders.
• Portfolio of open-source data platform implementations or contributions is a big plus. Skills: pyspark,dbt,data architecture,presto,spark,python,data lakehouse architecture,trino,airflow,airbyte,open source data platform