RE : Openings for Data Engineer – AI Implementation - Contract to Hire Position - Remote
Career Path Solutions Private Limited · Remote · remote · Posted 4d ago
About a minute. You'll make an account on the way through.
About this role
Role: Data Engineer – AI Implementation
Experience: Relevant experience in Data Engineering and AI data preparation
Technical Skills: Python, SQL, ETL/ELT, RAG, Qdrant, Neo4j, MongoDB
Role Overview
We are looking for a Data Engineer to support enterprise AI implementations by preparing, transforming, validating, and managing data for AI agents, automated workflows, and Retrieval-Augmented Generation (RAG) knowledge systems.
The candidate will work closely with Product Managers, AI Implementation Engineers, and client teams to ensure that structured data, semi-structured data, and enterprise documents are accurate, accessible, and ready for AI applications.
Key Responsibilities
1. Data Engineering & AI Workflows
- Define data requirements, schemas, field mappings, and transformation rules.
- Build and maintain ETL/ELT pipelines for data cleaning, transformation, and validation.
- Process structured and semi-structured data from APIs, databases, files, and integration sources.
- Identify and resolve missing values, duplicate records, inconsistent formats, and data quality issues.
- Prepare test datasets and validate data outputs for AI agents and automated workflows.
2. Knowledge Systems & RAG
- Extract, parse, and prepare enterprise documents for ingestion into AI knowledge bases.
- Preserve document structure, tables, metadata, and source references during processing.
- Implement document chunking, metadata extraction, and content versioning for RAG applications.
- Work with vector databases such as Qdrant to support embeddings, indexing, similarity search, and metadata filtering.
- Build and maintain knowledge graphs using Neo4j, data modelling, entity relationships, and Cypher queries.
- Manage document ingestion, updates, deletions, and troubleshooting of missing or outdated content.
- Apply OCR techniques when processing scanned documents.
3. Data Operations, Security & Governance
- Monitor pipeline performance, data quality, and content freshness.
- Implement error handling, retries, reconciliation checks, and recovery mechanisms.
- Follow data privacy, access control, credential management, retention, and deletion requirements.
- Maintain data lineage, version-controlled pipeline code, technical documentation, and operational procedures.
- Support release testing, production troubleshooting, and operational handover.
Required Skills & Qualifications
- Experience in data engineering or data-intensive software implementations.
- Strong proficiency in Python and SQL.
- Hands-on experience building ETL/ELT pipelines and data processing workflows.
- Understanding of APIs, JSON, databases, schemas, data mapping, and file-based data exchange.
- Experience with data cleansing, transformation, validation, and quality management.
- Practical knowledge of RAG data preparation, document parsing, chunking, and metadata management.
- Experience with vector databases, preferably Qdrant, including embeddings and similarity search.
- Knowledge of Neo4j, knowledge graphs, data modelling, and Cypher query language.
- Familiarity with MongoDB and structured or semi-structured data.
- Understanding of pipeline monitoring, error handling, version control, data lineage, and data governance.
- Strong analytical, troubleshooting, communication, and collaboration skills.
Technology Stack
- Programming & Querying: Python, SQL
- Databases: MongoDB
- Vector Database: Qdrant
- Knowledge Graph: Neo4j, Cypher
- AI & Knowledge Systems: RAG, embeddings, document chunking, metadata extraction, semantic search
- Data Engineering: ETL/ELT, data pipelines, API integration, data validation, OCR
- Additional Technologies: Node.js, React.js
Interested can share resume to this Mail ID - [HIDDEN TEXT]
Sourced from foundit