Data Engineer

Summary: We are looking for a highly motivated and experienced Data Engineer to join our data engineering team. The ideal candidate will have a strong background in building scalable data pipelines using the AWS cloud stack and extensive hands-on experience with Snowflake. Proficiency in Python and SQL, along with graph and vector database technologies, is essential. This role requires strong problem-solving abilities and a proactive mindset to deliver efficient, scalable, and reliable data solutions.

Responsibilities:

• Design, develop, and maintain scalable data pipelines on AWS using services such as S3, Glue, Lambda, Redshift, and EMR.
• Build and optimize data warehousing solutions using Snowflake, including performance tuning and data modeling.
• Write efficient and reusable code in Python and SQL for data transformation and processing.
• Collaborate with cross-functional teams, including data scientists, analysts, and business stakeholders, to understand data requirements.
• Develop and optimize solutions using graph databases (e.g., Neo4j, Company Neptune), including query design and performance tuning.
• Design, build, and operate vector database solutions (e.g., Milvus, Company OpenSearch) to support semantic search, recommendations, RAG, and AI-driven use cases.
• Integrate vector databases with LLM-based applications and AI workflows.
• Monitor, troubleshoot, and improve pipeline performance and reliability.
• Ensure data quality, integrity, and security across all stages of the pipeline.
• Participate in code reviews, architecture discussions, and continuous improvement initiatives.

Requirements:

• 5-8 years of experience in data engineering or related roles.
• Strong hands-on experience with AWS cloud services, including data and AI workloads.
• Deep understanding of Snowflake architecture, performance tuning, and best practices.
• Advanced proficiency in Python and SQL for data pipelines, transformations, and services.
• Strong understanding of graph and vector data modeling concepts and their practical applications.
• Hands-on experience with graph databases (e.g., Neo4j, Neptune) and vector databases (e.g., Milvus, Company OpenSearch).
• Experience with version control systems (e.g., Git) and Git workflows.
• Experience working with Azure DevOps (AzDO) boards for backlog management in Agile environments.
• Excellent analytical and problem-solving skills.
• Strong communication and collaboration abilities.
• Bachelor’s or Master’s degree in Computer Science, Engineering, or a related field.

Required Skills:

• Knowledge of the NVIDIA ecosystem and its applications in data and AI.
• Experience with orchestration tools such as AWS Step Functions.
• Familiarity with data governance and compliance practices.
• Exposure to real-time data processing frameworks (e.g., Kafka, Spark Streaming). #AditiConsulting # 26-04677