ML Engineer I

Job Summary

We are seeking an experienced AI Developer + Testing Specialist with 6+ years of hands-on experience in AI/ML development, testing, and implementation across multiple AI platforms including Copilot, Claude, and other major AI tools. This role requires expertise in developing robust AI applications, comprehensive testing methodologies, quality assurance, and integration with enterprise systems.

Key Responsibilities

Development & Implementation

• Design, develop, and deploy AI-powered applications and solutions using Copilot, Claude, and other major AI platforms
• Create custom AI workflows, prompts, and integrations tailored to business requirements
• Develop RAG (Retrieval Augmented Generation) systems, chatbots, and conversational AI applications
• Implement API integrations with various AI models and services
• Write clean, maintainable, and well-documented code following best practices and coding standards
• Collaborate with product teams to understand requirements and translate them into technical specifications

Testing & Quality Assurance

• Develop comprehensive test strategies and test plans for AI applications
• Perform functional, integration, performance, and security testing of AI-powered solutions
• Conduct prompt engineering validation and optimization testing
• Create and maintain automated test suites for AI/ML pipelines
• Perform edge case and adversarial testing to identify model limitations and vulnerabilities
• Validate output quality, accuracy, latency, and consistency across different AI models
• Document test results, defects, and recommendations with detailed analysis

AI Model Evaluation & Optimization

• Evaluate and compare different AI models (GPT-4, Claude, Copilot, LLaMA, etc.) for specific use cases
• Assess model performance, accuracy, cost-effectiveness, and suitability
• Fine-tune prompts and configurations for optimal performance
• Monitor model performance in production and recommend improvements
• Conduct A/B testing and performance benchmarking

Documentation & Knowledge Management

• Create technical documentation for AI implementations and testing procedures
• Develop training materials and best practice guides for AI tool usage
• Maintain comprehensive records of testing methodologies and results
• Document lessons learned and recommendations for future projects

Collaboration & Communication

• Work cross-functionally with data scientists, engineers, product managers, and stakeholders
• Present findings, recommendations, and test results to technical and non-technical audiences
• Participate in code reviews and peer testing sessions
• Contribute to team knowledge sharing and continuous improvement initiatives

Required Qualifications

Experience

• 6+ years of professional experience in AI/ML development or testing
• 3+ years of hands-on experience with Copilot, Claude, or similar LLM-based platforms
• 2+ years of dedicated testing and QA experience with AI/ML systems
• Proven experience developing production-grade AI applications
• Experience with multiple AI platforms and models (OpenAI, Anthropic, Microsoft, Google, Meta, etc.)

Technical Skills

• Programming Languages: Python, JavaScript/TypeScript, Java, or similar (Python preferred)
• AI/ML Frameworks: LangChain, LlamaIndex, OpenAI API, Anthropic API, Azure OpenAI, Copilot
• Testing Frameworks: pytest, Selenium, Jest, or similar automated testing tools
• API Development & Integration: RESTful APIs, webhooks, real-time integrations
• Version Control: Git, GitHub, GitLab
• Cloud Platforms: Azure, AWS, Google Cloud (at least one)
• Prompt Engineering: Advanced knowledge of prompt design, optimization, and validation

Domain Knowledge

• Deep understanding of Large Language Models (LLMs) and their capabilities/limitations
• Knowledge of RAG (Retrieval Augmented Generation) systems and vector databases
• Understanding of AI ethics, bias detection, and responsible AI practices
• Familiarity with AI security, data privacy, and compliance requirements
• Knowledge of NLP/NLU concepts and applications

Education & Certifications

• Bachelors degree in Computer Science, Software Engineering, Computer Engineering, or related field
• OR equivalent professional experience (6+ years) in place of formal education
• Relevant certifications (AI/ML, Cloud, QA) are a plus

Preferred Qualifications

• Advanced Certifications: AWS Certified Machine Learning, Microsoft Azure AI Engineer, Google Cloud AI Engineer
• Masters degree in Computer Science, Artificial Intelligence, or related field
• Experience with advanced testing methodologies (chaos engineering, security testing, load testing)
• Hands-on experience with enterprise AI platforms and tools
• Experience with Agile/DevOps methodologies and CI/CD pipelines
• Published work, patents, or contributions to open-source AI projects
• Experience with vector databases (Pinecone, Milvus, Weaviate, Chroma)
• Knowledge of machine learning operations (MLOps) and model monitoring
• Experience in financial services, healthcare, or regulated industries
• Track record of building scalable, fault-tolerant systems

Required Technical Competencies

AI/LLM Expertise

• Proficiency with Claude (Opus, Sonnet, Haiku models)
• Proficiency with Copilot (Microsoft Copilot, GitHub Copilot, Copilot Pro)
• Experience with OpenAI models (GPT-4, GPT-3.5, fine-tuning)
• Understanding of model selection criteria and trade-offs
• Expertise in prompt engineering and few-shot learning

Testing & QA

• Test planning and strategy development
• Automated testing and continuous integration
• Performance and load testing
• Security and penetration testing basics
• Regression testing and test maintenance
• Quality metrics and reporting

Development Tools & Practices

• Proficiency in relevant programming languages
• Knowledge of software architecture patterns
• Experience with microservices and distributed systems
• Database design and optimization (SQL, NoSQL)
• API design and development

Cloud & DevOps

• Cloud platform experience (Azure, AWS, or GCP)
• Containerization (Docker, Kubernetes)
• CI/CD pipeline implementation and management
• Infrastructure as Code (IaC)
• Monitoring and logging solutions

Soft Skills Required

• Problem-Solving: Ability to debug complex AI systems and identify root causes
• Attention to Detail: Meticulous in testing and quality assurance
• Communication: Clear documentation and ability to explain technical concepts to non-technical stakeholders
• Collaboration: Strong teamwork and ability to work effectively in cross-functional teams
• Adaptability: Ability to quickly learn new AI tools, models, and frameworks
• Ownership: Takes responsibility for project outcomes and quality
• Initiative: Self-motivated with ability to identify improvements and take action
• Time Management: Ability to manage multiple projects and priorities
• Continuous Learning: Passion for staying updated with AI/ML advancements

Responsibilities Summary

• Development: AI app development, API integration, RAG systems, workflow automation
• Testing: Test strategy, automated testing, performance testing, prompt validation
• Evaluation: Model comparison, performance assessment, A/B testing, benchmarking
• Documentation: Technical specs, test reports, best practices, training materials
• Collaboration: Cross-team coordination, stakeholder communication, code reviews

Disclaimer: This job posting has been aggregated from external source. Role details, content, and availability are subject to change. Applicants are advised to confirm the latest information directly on the company website before applying.