Senior AI Security Engineer

Key Responsibilities
AI Security & Guardrails

• Design, implement, and maintain enterprise AI security guardrails.
• Develop controls to mitigate prompt injection, jailbreaks, data leakage, hallucinations, model abuse, and adversarial attacks.
• Define AI trust, safety, governance, and security standards.
• Evaluate AI applications against organizational security, privacy, and compliance requirements.
• Establish policies and enforcement mechanisms for responsible AI usage.

AI Testing & Evaluation

• Design and develop AI testing harnesses to assess model performance, reliability, resiliency, and security.
• Create automated validation frameworks to test AI systems against approved success criteria.
• Develop benchmark testing methodologies for LLMs, agents, RAG platforms, and generative AI applications.
• Build continuous testing pipelines that validate changes before production deployment.
• Measure model quality, accuracy, consistency, safety, and business effectiveness.

AI Red Teaming

• Conduct AI red team assessments against enterprise AI applications.
• Develop adversarial testing scenarios and attack simulations.
• Identify and document vulnerabilities and weaknesses within AI systems.
• Perform threat modeling for AI-enabled business processes.
• Define remediation recommendations and compensating controls.

Engineering & Automation

• Develop Python-based automation for AI security and testing workflows.
• Build APIs, integrations, and orchestration frameworks supporting AI evaluation efforts.
• Integrate testing and governance controls into CI/CD pipelines.
• Create dashboards and reporting capabilities that provide visibility into AI risk and performance.

Collaboration & Leadership

• Partner with AI and application development teams to embed security throughout the AI lifecycle.
• Provide guidance on secure AI architecture patterns and implementation approaches.
• Mentor teams on AI security best practices.
• Stay current on emerging AI threats, frameworks, and regulatory developments.

Required Skills
AI/ML Technologies

• Large Language Models (GPT, Claude, Gemini, Llama, Mistral)
• Retrieval Augmented Generation (RAG)
• Agentic AI architectures
• Multi-agent systems
• Vector databases and embeddings
• Prompt engineering
• AI model evaluation methodologies
• Fine-tuning concepts and LLM optimization

Programming & Development

• Advanced Python development
• API development and integration
• Git/GitHub
• Automated testing frameworks
• CI/CD pipelines
• Containerization (Docker)
• Software Development Lifecycle (SDLC)

AI Security

• Prompt injection testing
• AI red teaming
• LLM security assessment
• Threat modeling
• Adversarial AI testing
• Secure AI architecture
• Data protection and privacy controls
• Responsible AI frameworks

AI Testing Frameworks

• Promptfoo
• DeepEval
• Ragas
• MLflow
• LangSmith
• Automated evaluation platforms

Cloud & Infrastructure

• Microsoft Azure
• Azure AI Foundry
• Azure OpenAI
• Azure Kubernetes Service (AKS)
• Kubernetes
• Monitoring and observability tooling

Security Frameworks

• OWASP Top 10 for LLM Applications
• NIST AI Risk Management Framework
• Secure coding standards
• Security assessments and vulnerability management

Qualifications

• Bachelor's degree in Computer Science, Engineering, Cybersecurity, or related discipline.
• 7+ years of software engineering, cybersecurity, or platform engineering experience.
• 3+ years working with AI/ML or Generative AI technologies.
• Demonstrated experience designing AI testing frameworks and security controls.
• Strong written and verbal communication skills.
• Ability to work effectively with both technical and business stakeholders.