Senior Data Engineer (AI Data Platforms & Automation)

Title: Senior Data Engineer (AI Data Platforms & Automation)

Experience: 8–12 Years (Preferred: 9–10 Years)

About the Role We are looking for an experienced and hands-on Senior Data Engineer to build scalable data platforms and AI-ready data pipelines. The ideal candidate will be responsible for designing data architectures, metadata frameworks, ETL pipelines, and automation that support AI and data initiatives across the organization.

Key Responsibilities Data Engineering

• Design and develop scalable ETL/ELT pipelines.
• Build robust data ingestion frameworks.
• Design enterprise data models.
• Implement data transformation and validation processes.
• Optimize data storage and retrieval.

AI Data Platforms

• Build AI-ready datasets.
• Design metadata frameworks and taxonomies.
• Develop dataset versioning strategies.
• Manage structured and unstructured datasets.
• Support AI model training and evaluation pipelines.

Data Quality & Governance

• Implement data quality checks.
• Build validation and monitoring frameworks.
• Define data standards and best practices.
• Ensure data integrity and consistency.
• Develop documentation and data lineage.

Automation

• Automate repetitive data workflows.
• Build Python-based automation tools.
• Develop integrations with internal and external systems.
• Create dashboards and monitoring utilities.

Platform Engineering

• Work with SQL and NoSQL databases.
• Optimize database performance.
• Manage cloud-based data platforms.
• Support scalable enterprise data architecture.

Required Technical Skills Programming

• Python (Expert)
• SQL (Expert)
• PySpark (Preferred)

Databases

• PostgreSQL
• MySQL
• MongoDB
• BigQuery / Snowflake
• Vector databases (Preferred)

Data Engineering

• ETL/ELT Development
• Data Warehousing
• Data Modeling
• Metadata Management
• Data Quality Frameworks

Cloud

• Google Cloud Platform or AWS
• Cloud Storage
• BigQuery / Redshift
• Airflow (Preferred)

Tools

• Docker
• Git
• CI/CD
• API Integration

Preferred Experience

• AI data platforms
• Metadata management
• Dataset curation
• Workflow automation
• Enterprise data engineering
• Cloud-native data platforms
• AI data preparation

Soft Skills

• Strong analytical thinking
• Excellent problem-solving skills
• Ability to design scalable systems
• Hands-on implementation mindset
• Strong communication skills Ability to work with cross-functional teams