AI / LLM Systems Engineer (Consultant/Freelance)

Key Responsibilities

• Set up and optimize LLM inference infrastructure, including migrating from Ollama to vLLM
• Benchmark and optimize open-weight models such as Gemma, Qwen, Llama, Mistral, etc.
• Optimize GPU utilization, latency, throughput, memory usage, and inference cost
• Develop and experiment with model quantization and calibration techniques to minimize quality degradation
• Build fine-tuning pipelines using techniques such as LoRA/QLoRA for specific use cases
• Establish automated LLM and RAG evaluation frameworks
• Benchmark models across quality, latency, memory, and throughput
• Profile and optimize workloads on NVIDIA RTX 4090 / RTX 5090 and future GPU infrastructure
• Evaluate emerging open-source models, inference frameworks, and optimization techniques
• Build reusable foundations for model serving, evaluation, experimentation, and deployment

What We're Looking For

• Strong Python and Linux engineering skills
• Hands-on experience with LLMs and Hugging Face Transformers
• Experience with vLLM or other LLM inference frameworks
• Strong understanding of LLM inference, GPU memory, attention, KV cache, batching, etc.
• Experience with quantization, model optimization, or compression
• Experience with LoRA/QLoRA/fine-tuning
• Experience designing LLM evaluation and benchmarking systems
• Strong debugging, profiling, and performance optimization mindset

Good to Have

• CUDA / Triton / FlashAttention
• GPTQ / AWQ / FP8 / INT8 / INT4
• SGLang / TensorRT-LLM / llama.cpp
• Distributed or multi-GPU inference
• RAG evaluation and retrieval optimization
• Experience working with production LLM infrastructure